closeup photography of brown analog scale

Scaleway Kapsule and Rancher-managed Hetzner Kubernetes clusters

I honestly think this is the 3rd time I’m moving things around. First I was on Proxmox managing myself a cluster of bare-metal Hetzner (from the server auction page). Then I was torn between a home server and one on Hetzner. I was using many images from the TurnKey Linux project. Nice, interesting but required immense investments of time.

Then I decided I wanted managed Kubernetes services. Not long ago (half-a-month) did I went over to Digital Ocean’s managed Kubernetes service. For not a small price, got a 3 node cluster with 24GB of RAM and 12 cores. Started installing my stuff and quickly ran out of resources only to be forced to pay more.

Continue reading →

Ivideon on Kubernetes, the simplest form of HA video surveillance

The decision to go to Kubernetes (managed deployment on Digital Ocean) was so that I could also free my home-server and eventually shut it down. There I hosted the Windows-based Ivideon server, on a Windows VM running on Proxmox. The internet in Romania being broadband helped a lot. I’m actively using Ivideon to record activity around the properties we hold.

Though in need of one for a quick deployment, I couldn’t find a HELM chart or something already done. On the other hand, the server itself is pretty stateless and just needs a configuration file and a PV to hold the archive, so not too big of a challenge to write a deployment file for it.

Continue reading →

Swithing oceans, from Heztner to Digital Ocean

I think I’ve ignored the Kubernetes movement for many years now. I used to maintain Docker-based infrastructures a couple of years ago, over bare-metal, mostly for work purposes. It was an interesting learning experience back then in the details of the containers foundation.

Still, I am still prudent about infrastructure in general and for a long time have favored pure and simple bare-metal or bare-metal + VM solutions in favor of containers for most of the critical data workloads (a.k.a. Big Data). Even in work deployments, we bypassed the usual performance penalties of containers by bind mounting the disk or using IPVLAN for networking when pure performance was needed. My favoritism for bare-metal is based on the fact that you can’t just ignore +50 years of evolution and documentation (if we intent on saying that we consider the “birth” of 1st operating system (UNIX) in 1969). I don’t want to go earlier than that …

Continue reading →

Wildcard DNS in Let’s Encrypt with Go.CD, Ansible, FreeIPA and S3

When I started working on my own home-cloud (a weird term for a small self-sustained, bare-metal paid cloud on Hetzner) I needed a way to have trusted SSL certificates. I really, really hate the warning messages of the browsers when entering an self-signed site. One of my goals was to use Let’s Encrypt, put HAproxy in front of any and all services and have HAproxy do the SSL termination (and even internally, to have all services use Let’s Encrypt signed certificates).

As part of this small architecture (based on Proxmox in a cluster configuration) it was chosen also to deploy a 5-node FreeIPA cluster to manage DNS mostly but also I took advantage of other IdM features. Another goal was to implement the wildcard DNS challenge so that I wouldn’t have to configure each and every sub-domain I required (there were a couple of TLDs and a miriad of sub-domains which I already forgot their names).

Continue reading →

Continuous delivery of infrastructure as code using Go.CD and Ansible

I’m fond of the CI/CD movement, mostly because I can quickly see the value in automating the build and deployment pipeline and getting a quick feedback and if all tests pass, a good feeling of reliability of the service I’m deploying. A few years ago I would’ve used Go.CD for both CI and CD pipelines and I have yet to see a project that does not benefit from this ideology in some way or form.

The history of Go.CD starts as CruisteControl, probably the first CI software that was built in this industry, long before Jenkins became popular. Born in ThoughtWorks, backed by Folwer & friends, originally named Cruise in homage to the original CI tool, but quickly renamed to “Go” to avoid the confusion.

Continue reading →

Idempotent LXC with Ansible and Proxmox using “pvesh”

Back a few months when I started my Hetzner deployment of a small Proxmox cluster I checked to see if there was an Proxmox module for Ansible. And indeed there is one on the official documentation but as I was soon to discover, it didn’t work with my Proxmox 6 installation due to issue #59164 which got resolved (but is only available in 2.9.2 which my Debian-based Go.CD agents can’t see right now). Of course, I could install from “pip” sources and that would solve the versioning issue, but back then this was still an issue.

So what I wanted is an idempotent way of creating mostly LXC containers using Proxmox. Initially I wanted to go the REST API way but it was kind of complicated (in the sense of doing that from Ansible code). Secondly, there was the ‘pvesh’ CLI tool that we could use and based on the available “nextid” command I was able to “test” if the declared “vmid” existed:

Continue reading →

The €154, 5-node, HA, hyper-converged Proxmox private cloud on Hetzner

For the past few years I’ve been paying around €60/month at Google Cloud to host the equivalent of 4 cores and 8 GB of RAM in total on all my instances. Recently I converted my home i7-3770 to an Proxmox based server and found it super simple to work with it. Through a combination of No-IP, DNS CNAMEs and an HAproxy instance forwarded through my router I was able to get many applications easily installed (and backed-up to S3 through TurnKeyLinux TKLBAM/backup which runs Duplicity) around every single day. So much for complexity as in about 3 days I had pretty much everything up (Nexus, Go.CD and agents, this blog, Mattermost, Nextcloud, etc.)

In the past 3 days I rediscovered Hetzner. I knew them for a long time but I wasn’t so keen on renting “dedis” (dedicated servers). Up until I discovered their server auction going around the €30 per i7-3770 with 2x3TB of HDD and 32GB of RAM.

Initially I just fooled around and played around with 1 machine and the “installimage” script, trying out to see if it’s easy to set-up Debian 10 and PVE. It went smooth. Then I explored the networking part, trying to see if I could get an private subnet on the same VM to be routed in the so-called “single IP, routed configuration” that Proxmox suggests.

Continue reading →