How to Simplify Data Pipelines with DBT and Airflow?
Reading Time: 7 minutes In today’s data-driven world, efficient data pipelines have become the backbone of successful organizations. These pipelines ensure that data flows smoothly from various sources to its intended destinations, enabling businesses to make informed decisions and gain valuable insights. Two powerful tools that have emerged to simplify the management of data pipelines are DBT (Data Build […]
How to ETL API data to AWS S3 Bucket using Apache Airflow?
Reading Time: 7 minutes 2.5 quintillion bytes of data are produced every day with 90% of it generated solely in the last 2 years (Source: Forbes). Data is pulled, cleaned, transfigured & then presented for analytical purposes & put to use in thousands of applications to fulfill consumer needs & more. While generating insights from the data is important, […]
Easily build ETL Pipeline using Python and Airflow
Reading Time: 5 minutes Apache Airflow is an open-source workflow management platform for authoring, scheduling, and monitoring workflows or data pipelines programmatically. Python is used to write Airflow, and Python scripts are used to create workflows. It was created by Airbnb. In this blog, we will show how to configure airflow on our machine as well as write a […]