blue and brown concrete stairs

Continuous integration hits 30 years, though some still ignore it

It still puzzles me how is it possible that an 1991 idea, is still ignored by some software engineering teams worldwide in their advent to “deliver” at any cost, fail to invest the adequate time into ensuring they have all the pipelines in order. Don’t know if to attribute that to bad management or complacency inside the entire team.

Sadly to say, I’ve had my share of this situation. Though on the projects under my team’s ownership, we’ve been all the rage with tests, code quality metrics and documentation published directly from the source code, not the same I can say for the other half and for the artifacts produced in the last years of work, which are hand built, locally on the developer’s laptop which is releasing the artifact.

Continue reading →

Continuous delivery of infrastructure as code using Go.CD and Ansible

I’m fond of the CI/CD movement, mostly because I can quickly see the value in automating the build and deployment pipeline and getting a quick feedback and if all tests pass, a good feeling of reliability of the service I’m deploying. A few years ago I would’ve used Go.CD for both CI and CD pipelines and I have yet to see a project that does not benefit from this ideology in some way or form.

The history of Go.CD starts as CruisteControl, probably the first CI software that was built in this industry, long before Jenkins became popular. Born in ThoughtWorks, backed by Folwer & friends, originally named Cruise in homage to the original CI tool, but quickly renamed to “Go” to avoid the confusion.

Continue reading →

On workflow engines and where Airflow fits in

With the occasion of the CrunchConf 2018 there was a presentation on “Operating data pipeline using Airflow @ Slack” fromĀ Ananth Packkildurai. If you don’t know what Airflow is, it’s an workflow engine of the similar likes of Oozie and Azkaban. It’s based on the concept of a DAG which you write in Python and execute on a cluster.

As in the case of the Kafka presentation by Tim Berglund, we’ve asked the hard questions and they got popular pretty soon. In the case of Airflow, in the eco-system of workflow engines, we had quite a heavy question.

Continue reading →