Another midnight catches me tinkering away at my own little cloud on the Internet. Since renting these 6 machines in Hetzner I’ve been literally hacking away for the past month on setting things up fully automated (that means also dns-01 challenges for all my domains automated with Ansible + FreeIPA for the core services, DNS included and tightening up security).
Out of all 6 machines, given 6TB per machine accumulated 36TB on top of which I put a few volumes of Gluster, two of them as “backup” and “ha-vms-root-fs” as I called it. Understandably, one is for (local, fast recovery) backups, doh, one is to host HA VMs declared as “resources” in Proxmox so the cluster takes care of making them HA if one machine fails). The one for backups in addition to TLK (TurnKey Linux) which provides the “tlkbam-backup” cron.
For the past few years I’ve been paying around €60/month at Google Cloud to host the equivalent of 4 cores and 8 GB of RAM in total on all my instances. Recently I converted my home i7-3770 to an Proxmox based server and found it super simple to work with it. Through a combination of No-IP, DNS CNAMEs and an HAproxy instance forwarded through my router I was able to get many applications easily installed (and backed-up to S3 through TurnKeyLinux TKLBAM/backup which runs Duplicity) around every single day. So much for complexity as in about 3 days I had pretty much everything up (Nexus, Go.CD and agents, this blog, Mattermost, Nextcloud, etc.)
In the past 3 days I rediscovered Hetzner. I knew them for a long time but I wasn’t so keen on renting “dedis” (dedicated servers). Up until I discovered their server auction going around the €30 per i7-3770 with 2x3TB of HDD and 32GB of RAM.
Initially I just fooled around and played around with 1 machine and the “installimage” script, trying out to see if it’s easy to set-up Debian 10 and PVE. It went smooth. Then I explored the networking part, trying to see if I could get an private subnet on the same VM to be routed in the so-called “single IP, routed configuration” that Proxmox suggests.
I’m undeniably an OCD kind of guy and a control freak by definition. Why else would I abandon the easiness of WordPress.com for hosting this blog on my own (backed-up, d’oh) LXC container on some random machine?! For control.
It’s 00:00 midnight and I’m writing this after a full week of migrating off-cloud to my own machine. I’ve recently found out about Proxmox VE, a mostly Debian-based virtualization engine based on KVM/qemu and LXC and since I’m a Debian fan, I quite jumped-in on the fact that it could be what I was potentially looking for.
For the past months I’ve been trying to put an old i7/32GB of RAM machine of mine to work. Since I have a growing kid and not much of time, I wanted something dead-easy for a homelab slash DIY/Wordpress hosting slash CI/CD machine with an Nexus (binary artifact repository) slash anything that can run in isolation. So I went off, bought 3 Western Digital HDDs, installed the Proxmox ISO making myself an RAIDZ ZFS-backed array for the upcoming VMs or containers and there you go.
I tried to kind of ignore this problem and not write about it. But it kept appearing and reappearing on my daily Slashdot feed up to the point I’d also want to say something. We seem to have a problem that about defining what it means to be open-source and what is expected of the developers behind these projects.
Let me first start by saying I’m an advocate of open-source. I’ve tried my best to use open-source software in all my architecture designs, respecting the licenses of the products I’ve used, trying to also give feedback where it mattered (eg. Docker/moby on IPVLAN, Grafana’s Elasticsearch support and other tickets). I did this directly or through my fellow peers encouraging them to take action and feedback the community. I’ve always been thankful of the hard-work some people (not me) are putting in. I made myself small contributions, bug reports I could call them, from using the open-source software. I was thrilled to be a part of that and to contribute something that got into the next change-log as a fix (eg. Cassandra).
When I was young and deciding what career path would I take and when our teachers would ask what do we want to pursue, I’d always answer something along the lines of: “software engineer”. Twenty or so years ago I had much esteem for the trade. The principles of automating repetitive tasks, freeing someone from the day to day burden of repetitive chores and of course, of their salary, leaving them naked in the street, unable to feed their family, hunting for mice to survive seemed such a good investment of my time and skills.
Of course, I’m joking! Jesus! But the software industry is not and the reason that is happening is because pretty much any other industry, be it automotive, manufacturing, even sending satellites to space (and trust me, I know) is full of repetitive tasks that can be easily coded to perfection.
I haven’t had time these days to blog anymore. Since leaving my previous employer, a former “prestigious” mobile game creation company I’ve searched for other jobs, mainly going through an French telecom business, where the frustration were so high, from day one, that I quickly sought new challenges (searching again) after the first week.
Anyway, while I can’t say names and I will try not to tie this blog to any employer, directly, I’ve found a good (and exciting) job at a contractor in the space business. For which travel was included. The two weeks I’m here in Toulouse should be enough to understand 2 projects formerly under development here and to bring them over for upgrade back in my mother country (Romania).
The 2 weeks here meant that I’ve had the chance of a weekend over here. While rough at first, since the chosen hotel (MetrHotel Basso Cambo) is really well placed near the metro, the bus station directly to work, a Lidl, McDonald’s and KFC near, it took me a few days to accommodate so I didn’t have the urge to visit the center (old town) of the city.
Today I’ve left work and one of my colleagues who’s a father of two was playing Destiny 2 on Windows. Well, I’m a Linux fan and the only games I have available are the ones on the Steam OS + Linux category. Which fall short of Counter-Strike: Global Offensive (which I play in Competitive mode because public is sooo much full of d-bags) and Bioshock.
In all honesty, I just got the wake-up call that I haven’t been playing in the last 3 or 4 years pretty much anything. Funny of all, I worked in a gaming company, so games should’ve been somewhat in my target, no? Well, they haven’t, as I was too much concentrated on the day-to-day activity of the project. Which is a bummer. The last 30 minutes or so playing a T vs. CT competitive match, allthough we lost, were the most fun moments at the computer I had in a while.
Nate Kupp currently holds the position of Director of Infrastructure and Data Science at Thumbtack and has presented this year his talk and success story entitled: “From humble beginnings: building the data stack at Thumbtack”. This is one of the presentations I’ve enjoyed much because it was similar to one of the pains I’ve also experienced in my day-to-day work.
A difference between Nate’s approach and mine is the executivesponsors (and a bit of luck of being in the right place, right time and the right management mentality). My experience on the other hand is, from my perspective a failure, but for others a small success against overwhelming odds.
Yesterday my wife proposed we do a quick Sunday walk to Bucharest’s Botanical Garden in the early part of the day, right before noon around 12:00/13:00 when our little exactly 1.7 year old (today) bundle of joy goes for the mid-day nap.
I agreed for two reasons: (1) the botanical garden isn’t all that crowded and (2) it’s big enough for the baby to run around. It’s not the best of parks or recreational areas, in terms of arangements or venues, but it’s the most diverse in terms of colors and the display of plants, in comparison to other parks.
As stated earlier, Andrey Sharapov’s presentation on “Building data products: from zero to hero!” has given me many motives to talk. And I can’t seem to stop with the ideas of things to write about. Maybe probably because I’ve been, at my current employer, at the time of writing, through the same pains as the guys at Lidl did. And those pains are centered around the management of data people, be them engineers or scientists.
The featured image of this article present a young female manager asking: “How are you?” and getting a cryptic reply: “About half a standard deviation below the mean …” which in some languages goes by as swearing or rude behaviour. But in all honesty, although English, these people come from very different backgrounds, with slight variations of vocabulary (and understanding, not that one is less of an educated person from the other as management itself is a hard discipline also as it involves a few areas of sociology, economics and more).