Podcast #7 : Data Tools

Dave Kruse (CEO) and John Hwang (Head of Data Engineering) break down how to evaluate and choose the right data tools for your platform, covering what they do, why they matter, and the key selection criteria. In this episode, the duo explores various types of data tools like all-in-one data tools, virtualization tools, and ETL tools and assesses them at different levels of performance and operational limitations.

All-in-one data tools are often expensive and make hiring skilled engineers a challenge, thereby extending build times rather than reducing them. Virtualization data tools come with significant drawbacks, too. Since they do not store processed data, reprocessing becomes inevitable every time a query runs. This slows queries from seconds to hours, and it may not be feasible with third-party vendors or APIs that restrict frequent querying. ETL tools are often widely used, but also have some limitations. These traditional systems struggle to process large data volumes, and only a few developers are allowed to work on the development pipeline at a time. Testing is hard and inefficient. Every time a test is run, it requires the data to be fully processed, making it even more complex.

So, what is the best alternative? Tune in to our latest podcast to explore data tools and gain deeper insights.