Wildcard DNS in Let’s Encrypt with Go.CD, Ansible, FreeIPA and S3

When I started working on my own home-cloud (a weird term for a small self-sustained, bare-metal paid cloud on Hetzner) I needed a way to have trusted SSL certificates. I really, really hate the warning messages of the browsers when entering an self-signed site. One of my goals was to use Let’s Encrypt, put HAproxy in front of any and all services and have HAproxy do the SSL termination (and even internally, to have all services use Let’s Encrypt signed certificates).

As part of this small architecture (based on Proxmox in a cluster configuration) it was chosen also to deploy a 5-node FreeIPA cluster to manage DNS mostly but also I took advantage of other IdM features. Another goal was to implement the wildcard DNS challenge so that I wouldn’t have to configure each and every sub-domain I required (there were a couple of TLDs and a miriad of sub-domains which I already forgot their names).

Continue reading →

Continuous delivery of infrastructure as code using Go.CD and Ansible

I’m fond of the CI/CD movement, mostly because I can quickly see the value in automating the build and deployment pipeline and getting a quick feedback and if all tests pass, a good feeling of reliability of the service I’m deploying. A few years ago I would’ve used Go.CD for both CI and CD pipelines and I have yet to see a project that does not benefit from this ideology in some way or form.

The history of Go.CD starts as CruisteControl, probably the first CI software that was built in this industry, long before Jenkins became popular. Born in ThoughtWorks, backed by Folwer & friends, originally named Cruise in homage to the original CI tool, but quickly renamed to “Go” to avoid the confusion.

Continue reading →

Idempotent LXC with Ansible and Proxmox using “pvesh”

Back a few months when I started my Hetzner deployment of a small Proxmox cluster I checked to see if there was an Proxmox module for Ansible. And indeed there is one on the official documentation but as I was soon to discover, it didn’t work with my Proxmox 6 installation due to issue #59164 which got resolved (but is only available in 2.9.2 which my Debian-based Go.CD agents can’t see right now). Of course, I could install from “pip” sources and that would solve the versioning issue, but back then this was still an issue.

So what I wanted is an idempotent way of creating mostly LXC containers using Proxmox. Initially I wanted to go the REST API way but it was kind of complicated (in the sense of doing that from Ansible code). Secondly, there was the ‘pvesh’ CLI tool that we could use and based on the available “nextid” command I was able to “test” if the declared “vmid” existed:

Continue reading →

Getting there …

Another midnight catches me tinkering away at my own little cloud on the Internet. Since renting these 6 machines in Hetzner I’ve been literally hacking away for the past month on setting things up fully automated (that means also dns-01 challenges for all my domains automated with Ansible + FreeIPA for the core services, DNS included and tightening up security).

Out of all 6 machines, given 6TB per machine accumulated 36TB on top of which I put a few volumes of Gluster, two of them as “backup” and “ha-vms-root-fs” as I called it. Understandably, one is for (local, fast recovery) backups, doh, one is to host HA VMs declared as “resources” in Proxmox so the cluster takes care of making them HA if one machine fails). The one for backups in addition to TLK (TurnKey Linux) which provides the “tlkbam-backup” cron.

Continue reading →