Data Pipelines & ETL Software Directory

Pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

63.1K stars open source

AI & Machine Learning Data Pipelines & ETL

Apache Airflow

Platform to programmatically author, schedule, and monitor workflows.

45.8K stars open source

Cron & Job Scheduling Data Pipelines & ETL

Data Engineering Zoomcamp

Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. The next cohort starts in January 2026. Join the course here 👇🏼

42.3K stars open source

LMS Data Pipelines & ETL

Jaeger

CNCF Jaeger, a Distributed Tracing Platform

22.9K stars open source

Monitoring & Observability Data Pipelines & ETL

Prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

22.6K stars open source

Workflow Automation Data Pipelines & ETL

Airbyte

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

21.4K stars open source

Data Pipelines & ETL iPaaS & API Integration

Zipkin

Zipkin is a distributed tracing system

17.4K stars open source

Monitoring & Observability Data Pipelines & ETL

Unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. V... Read More

14.9K stars open source

AI & Machine Learning Data Pipelines & ETL

Dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

14.3K stars open source

Cron & Job Scheduling Data Pipelines & ETL

Dbt Core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

13K stars open source

BI Dashboards Data Pipelines & ETL

Debezium

Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.

12.8K stars open source

NoSQL Databases Data Pipelines & ETL

Redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

12.2K stars open source

Search Engines Data Pipelines & ETL

Kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducib... Read More

10.9K stars open source

ML Frameworks Data Pipelines & ETL

Apollo

Sunshine fork - The easiest way to stream with the native resolution of your client device

9.7K stars open source

API Tools Container Management Crypto & Blockchain

Seatunnel

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

9.4K stars open source

AI & Machine Learning Data Pipelines & ETL

Risingwave

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

9.1K stars open source

Data Pipelines & ETL Data Warehouses

Elasticsearch Dump

Import and export tools for elasticsearch & opensearch

7.9K stars open source

DB Admin & GUI Clients Data Pipelines & ETL

Snowplow

The leader in Customer Data Infrastructure

7K stars open source

Event Tracking & CDP Data Pipelines & ETL

Unstract

LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows

6.6K stars open source

AI Agents Data Pipelines & ETL

Ibis

the portable Python dataframe library

6.6K stars open source

Workflow Automation NoSQL Databases Wiki & Knowledge Base

Data Juicer

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

6.5K stars open source

Data Labeling Data Pipelines & ETL

Pgloader

Migrate to PostgreSQL in a single command!

6.4K stars open source

Relational Databases Data Pipelines & ETL

Flink Cdc

Flink CDC is a streaming data integration tool

6.4K stars open source

Data Pipelines & ETL Data Warehouses

Cloudquery

Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud an... Read More

6.4K stars open source

Data Pipelines & ETL

Data Pipelines & ETL Software & Alternatives

About Data Pipelines & ETL Software