![]() Our endless race towards database integrity - Enforced data relationships (such as via foreign keys) meant that data was often stuck partway through the pipeline (for instance, if a service was sold before its master data had been propagated through the data warehouse, then all subsequent sales transactions would be stuck or if a customer moves to an address yet to be updated on the post office's databases, then their customer data would struggle to be pushed into tables and data sets).Some of such discoveries - about our data landscapes - included: The period of Radical Candor that followed was, however, to deliver a number of insightful revelations - about our landscape - which we had, at the time, thought absolutely normal and taken for granted. Some of these steps had effectively become hurdles - or at times even brick-wall-like barriers - to both data flows and data products engineering. ![]() While on the surface this might have come across somewhat castigating, the underlying message was clear: as while by no means the intention at the start of its journey, our data landscapes have indeed become riddled with pace annihilating steps. "The trouble is, we get data flying in at such speeds - from a variety of sources - and we know this speed is increasing all the time with the increase in the use of digital channels and with the increasingly demanding customer, who wants it all now! And what do we do? We slow it all down - with our myriad of tediously tortuous processes, with our spaghetti ETL, and with our complex byzantine data warehouses!". ![]() ![]() It is yet another one of those days in the office: my director had just endured a severe dressing down from the CEO and as the usual cascading ritual was moving around the hierarchy like a ricochet, I had the unfortunate situation of our coffee rounds coinciding.Īfter getting a generous helping of bitter medicine, my director goes on to add, "Do you know the honest problem with Data departments, like ours?", a question in no way positioned like it expected a verbal response. ![]()
0 Comments
Leave a Reply. |