Today I’ve left work and one of my colleagues who’s a father of two was playing Destiny 2 on Windows. Well, I’m a Linux fan and the only games I have available are the ones on the Steam OS + Linux category. Which fall short of Counter-Strike: Global Offensive (which I play in Competitive mode because public is sooo much full of d-bags) and Bioshock.
In all honesty, I just got the wake-up call that I haven’t been playing in the last 3 or 4 years pretty much anything. Funny of all, I worked in a gaming company, so games should’ve been somewhat in my target, no? Well, they haven’t, as I was too much concentrated on the day-to-day activity of the project. Which is a bummer. The last 30 minutes or so playing a T vs. CT competitive match, allthough we lost, were the most fun moments at the computer I had in a while.
Nate Kupp currently holds the position of Director of Infrastructure and Data Science at Thumbtack and has presented this year his talk and success story entitled: “From humble beginnings: building the data stack at Thumbtack”. This is one of the presentations I’ve enjoyed much because it was similar to one of the pains I’ve also experienced in my day-to-day work.
A difference between Nate’s approach and mine is the executivesponsors (and a bit of luck of being in the right place, right time and the right management mentality). My experience on the other hand is, from my perspective a failure, but for others a small success against overwhelming odds.
Yesterday my wife proposed we do a quick Sunday walk to Bucharest’s Botanical Garden in the early part of the day, right before noon around 12:00/13:00 when our little exactly 1.7 year old (today) bundle of joy goes for the mid-day nap.
I agreed for two reasons: (1) the botanical garden isn’t all that crowded and (2) it’s big enough for the baby to run around. It’s not the best of parks or recreational areas, in terms of arangements or venues, but it’s the most diverse in terms of colors and the display of plants, in comparison to other parks.
As stated earlier, Andrey Sharapov’s presentation on “Building data products: from zero to hero!” has given me many motives to talk. And I can’t seem to stop with the ideas of things to write about. Maybe probably because I’ve been, at my current employer, at the time of writing, through the same pains as the guys at Lidl did. And those pains are centered around the management of data people, be them engineers or scientists.
The featured image of this article present a young female manager asking: “How are you?” and getting a cryptic reply: “About half a standard deviation below the mean …” which in some languages goes by as swearing or rude behaviour. But in all honesty, although English, these people come from very different backgrounds, with slight variations of vocabulary (and understanding, not that one is less of an educated person from the other as management itself is a hard discipline also as it involves a few areas of sociology, economics and more).
This year at CrunchConf 2018 there was an interesting talk by Andrey Sharapov an Data Engineer & Scientist at Lidl. Yes, Lidl. The store in your back alley or in your neighbourhood. Did you know it does Big Data? I assumed, yes, given one wants to optimize both the idea of minimizingwaste and increasing profits (eg. how much of X do one store needs to order to ensure it’s gone by EOD).
Andrey’s talk was centered around “Building data products: from zero to hero!” and I would personally want to apraise the realism of his presentation which gives me content for more than one article on the subject. He’s one in a series of presenters at this year’s conference that has called out to the strategy of companies of investing too much in data scientists, then finding out they don’t have an infrastructure those scientists need, then trying to find data engineers a bit too late in the game (which are even more scarce than scientists).
With the occasion of the CrunchConf 2018 there was a presentation on “Operating data pipeline using Airflow @ Slack” from AnanthPackkildurai. If you don’t know what Airflow is, it’s an workflow engine of the similar likes of Oozie and Azkaban. It’s based on the concept of a DAG which you write in Python and execute on a cluster.
As in the case of the Kafka presentation by Tim Berglund, we’ve asked the hard questions and they got popular pretty soon. In the case of Airflow, in the eco-system of workflow engines, we had quite a heavy question.
Just got back from CrunchConf 2018. A good panel of speakers and an interesting conference. Lots of food and drinks. Good atmosphere, helpful organizers. Fun times, good memories. The conference was a blast with most of my questions hitting the top votes with a little help from the community.
I decided in the context of the conference that I will share my thoughts on the presentations, at least for those that were intriguing and for those that my questions got the top votes. All in all, I would like to appraise good presentations, devoid of hype and commercialism. There seems to be some hype in today’s world around the Big Data projects, with the naive jumping ship to the next cool project.