Humans are arguably the most adaptable creatures on the planet — on both a micro and macro scale, we’re able to evolve to meet new challenges or circumstances all the time. It makes sense, then, that we want the tools we build and use every day to mimic our flexibility. If we are able to adapt to environmental changes, take on new challenges, and stay relevant, doesn’t it make sense that we would have the same expectation from our technology platforms?
But when it comes to implementing a data pipeline, technologists struggle with building those attributes into the design, resulting in a pipeline that may be effective at the time of its creation but unsustainable and less relevant for the future.
The key to solving this problem is having flexibility in the data pipeline itself. The world of data is constantly changing and expanding, and without that flexibility in a data pipeline design, businesses are stymied in what they can truly achieve with their data.
Read the full article here