Dear open-source, stop being free

I tried to kind of ignore this problem and not write about it. But it kept appearing and reappearing on my daily Slashdot feed up to the point I’d also want to say something. We seem to have a problem that about defining what it means to be open-source and what is expected of the developers behind these projects.

Let me first start by saying I’m an advocate of open-source. I’ve tried my best to use open-source software in all my architecture designs, respecting the licenses of the products I’ve used, trying to also give feedback where it mattered (eg. Docker/moby on IPVLAN, Grafana’s Elasticsearch support and other tickets). I did this directly or through my fellow peers encouraging them to take action and feedback the community. I’ve always been thankful of the hard-work some people (not me) are putting in. I made myself small contributions, bug reports I could call them, from using the open-source software. I was thrilled to be a part of that and to contribute something that got into the next change-log as a fix (eg. Cassandra).

Continue reading →

Unicorn data engineers & scientists, a guide to catch, keep and sh*t rainbows

This year at CrunchConf 2018 there was an interesting talk by Andrey Sharapov an Data Engineer & Scientist at Lidl. Yes, Lidl. The store in your back alley or in your neighbourhood. Did you know it does Big Data? I assumed, yes, given one wants to optimize both the idea of minimizing waste and increasing profits (eg. how much of X do one store needs to order to ensure it’s gone by EOD).

Andrey’s talk was centered around “Building data products: from zero to hero!” and I would personally want to apraise the realism of his presentation which gives me content for more than one article on the subject. He’s one in a series of presenters at this year’s conference that has called out to the strategy of companies of investing too much in data scientists, then finding out they don’t have an infrastructure those scientists need, then trying to find data engineers a bit too late in the game (which are even more scarce than scientists).

Continue reading →