Posts by C. A. Zamfir

Nothing more, nothing less.
green, red, and white high voltage circuit breaker

Automate code-review chores with a simple Dangerfile

One of the most annoying chores in code-reviewing is having to say: “You forgot to [unwritten rule]”. It feels arcane, it annoys the code-reviewer and the guy doing the work, it builds frustration, sparks useless arguments, some edging on the brink of flame wars. Well, not unless the rules are written, the rules themselves codereviewed and agreed upon in the team. One idea to automate-away some of the most common concerns and focus on the interesting logic is to automate the chores away using the Danger bot.

In one work situation, I had the possibility to implement the first pipelines the team has used and to provide some standardization going forward. Our Git-flow evolved around 3 branches (integration, dev and master) with the direction of code flowing being from left to right, passing through QA as it went from integration to “development” branch (tied to the specific cloud account) and on to “master” from which release branches were triggered, later to be deployed on the production cloud account. The existence of the “integration” branch basically is to check-in the code somewhere until we are ready to “cut the cord” and release a new version for QA and extensive development testing (with custom data and a sample of anonymous production data).

Continue reading →
An open empty notebook on a white desk next to an iPhone and a MacBook

Orchestrating notebooks with Camel K, an integration-based approach

When you achieve somewhat of a level of maturity in your data analytics pipelines you also tend to start exploring various and flexible ways to orchestrate the ETL processes of data you have and derive various tables for different access patterns, as required by the business downstream. However, similar to the “object impedance mismatch” in the object vs. relation database worlds, there’s an “impedance mismatch” between data-engineers and business folks when it comes to the expectations they have on the speed of delivery, quality, correctness, maintainability.

In the mindset of the business folk, the derivative work or data that he requires is just a simple SQL that is run on the “raw data” which has infinite amounts of CPU power, and infinite amounts of memory, probably running on GPUs anyway and thus just needs to be written/queried as such and it will return results in an instant.

Continue reading →
round green clock at 2:20

Writing software is a creative endeavor

A few books I’ve read and in fact the majority if not all of them (from decades old Mythical Man Month to Agile Estimation and Planning or more recent) all repeat the idea that the act of creation of software is a creative endeavor in the artistic sense of the word. My years in this domain for sure confirm it and furthermore, as with any “art” if it’s not done with pleasure, it’s not worth doing.

I wanted to write this post based on a discussion with former colleagues from a previous common employer which I shall not name who had the nerves of calling to the office a big batch of employees in full pandemic rise of COVID-19 only for the reason of a top-management visit, though the same employer had 2 recent infections in the ranks of other fellow colleagues. Of course, my friends were angry for being imposed to take a risk they do not want to take and because of the “slave” mentality of the employer.

Continue reading →
closeup photography of brown analog scale

Scaleway Kapsule and Rancher-managed Hetzner Kubernetes clusters

I honestly think this is the 3rd time I’m moving things around. First I was on Proxmox managing myself a cluster of bare-metal Hetzner (from the server auction page). Then I was torn between a home server and one on Hetzner. I was using many images from the TurnKey Linux project. Nice, interesting but required immense investments of time.

Then I decided I wanted managed Kubernetes services. Not long ago (half-a-month) did I went over to Digital Ocean’s managed Kubernetes service. For not a small price, got a 3 node cluster with 24GB of RAM and 12 cores. Started installing my stuff and quickly ran out of resources only to be forced to pay more.

Continue reading →
close-up photo of gray stone pile

The dynamic equilibrium of code, flexibility in software development

It must be considered that there is nothing more difficult to carry out nor more doubtful of success nor more dangerous to handle than to initiate a new order of things; for the reformer has enemies in all those who profit by the old order, and only lukewarm defenders in all those who would profit by the new order; this lukewarmness arising partly from the incredulity of mankind who does not truly believe in anything new until they actually have experience of it.

Niccolò Machiavelli (1469 – 1572)

The above words are as true for software development as they are true for politics and human nature. Though unlike politics which requires a shift of mentality of the majority of the population and unlike the human nature which usually takes years to change or a very aggressive and disturbing moment of life (death, pain, illness) … we are “blessed” let’s say, in software development to have a choice.

Continue reading →
white ceramic mug near tea pot on table

It’s not just about learning a programming language

It’s actually teaching yourself a way to think … I found myself today in a situation at work that’s outlined perfectly in this article on strategic domain driven design. In fact, the original idea I remembered it from Eric Evan’s great DDD book, but I couldn’t quickly find the page and paragraphs that contained the idea of contexts and upstream/downstream systems so I relied on Google to find me a relevant authority to invoke. Luckily, the above article came about and it captured the idea I wanted to express to my co-workers perfectly.

The situation was that an upstream data producer made a deployment, to which we scaled effectively and nicely all in an automated fashion but then a day later, they rolled-back the change as it produced 50% extra data. Then at the roll-back, 50% of the data was gone.

Continue reading →

Ivideon on Kubernetes, the simplest form of HA video surveillance

The decision to go to Kubernetes (managed deployment on Digital Ocean) was so that I could also free my home-server and eventually shut it down. There I hosted the Windows-based Ivideon server, on a Windows VM running on Proxmox. The internet in Romania being broadband helped a lot. I’m actively using Ivideon to record activity around the properties we hold.

Though in need of one for a quick deployment, I couldn’t find a HELM chart or something already done. On the other hand, the server itself is pretty stateless and just needs a configuration file and a PV to hold the archive, so not too big of a challenge to write a deployment file for it.

Continue reading →

Swithing oceans, from Heztner to Digital Ocean

I think I’ve ignored the Kubernetes movement for many years now. I used to maintain Docker-based infrastructures a couple of years ago, over bare-metal, mostly for work purposes. It was an interesting learning experience back then in the details of the containers foundation.

Still, I am still prudent about infrastructure in general and for a long time have favored pure and simple bare-metal or bare-metal + VM solutions in favor of containers for most of the critical data workloads (a.k.a. Big Data). Even in work deployments, we bypassed the usual performance penalties of containers by bind mounting the disk or using IPVLAN for networking when pure performance was needed. My favoritism for bare-metal is based on the fact that you can’t just ignore +50 years of evolution and documentation (if we intent on saying that we consider the “birth” of 1st operating system (UNIX) in 1969). I don’t want to go earlier than that …

Continue reading →

On “Fundamentals of Software Architecture”

In all honesty, I am somewhere near my 30s now and besides knowing to program in pretty much any language (or to that end, able to learn one in a few hours) I have to admit I have lost the passion towards high-frequency coding, in the sense of LOCs (lines of code) per hour, but I’m more and more attracted to the art of building software and systems architectures, with the least amount of glue code possible.

I believe that for the past years I’ve used and abused former Safari Books Online (now O’Reilly learning) plowing through every imaginable book on architecture, starting with the great reference book “Software Architecture In Practice, 3rd edition” by Len Bass, Paul Clements and Rick Kazman. Most of these authors and the authors that have dedicated a part of their time to book-writing have been members of the SEI (Software Engineering Institute).

Continue reading →

DIY surveillance with Ivideon and PNI

I seem to have a crush on Ivideon in combination with ONVIF-compatible devices (most under the PNI brand which has proved reliable in the past years for my offline DVR that I keep at our work-in-progress home at the country-side, on my father’s land, donated a few years ago to me).

These days I went off on a small house project to mount a few speed dome cameras at my in-laws house, at the country side also. The reasons I wanted to do the project was because my in-laws recently contracted a FTTH service (Fiber To The Home) from one well known provider in Romania (Digi/RCS&RDS).

Continue reading →