7 Real-Time Intelligence: Kafka Streaming and Event-Driven Agent Triggers
Series: Building an Agentic Data Platform | Part 7 of 17Reading time: ⏳ ~12 minutesTags: 🏷️ Kafka event streaming real-time data CDC change data capture event-driven agents…
6 Governing with Intelligence: OPA Policies and Agent-Driven Compliance
Series: Building an Agentic Data Platform | Part 6 of 17Reading time: ⏳ ~10 minutes 📌 TL;DR Governance is layered: OPA enforces data rules; middleware applies consistent…
5 Building the Engine: Core ETL Stages with Agent Instrumentation
Series: Building an Agentic Data Platform | Part 5 of 17Reading time: ⏳ ~13 minutesTags: 🏷️ ETL stages data ingestion PII scrubbing audit logging pipeline instrumentation PostgreSQL…
4 Synthetic Data Engineering: Teaching Your Platform What Real Data Looks Like
Series: Building an Agentic Data Platform | Part 4 of 17Reading time: ⏳ ~11 minutesTags: 🏷️ synthetic data data generation GDPR financial data Faker Pydantic pipeline testing…
3 Designing for Intelligence: The Agentic Data Pipeline Architecture
Series: Building an Agentic Data Platform | Part 3 of 17Reading time: ⏳ ~13 minutesTags: 🏷️ pipeline architecture stage-based design agentic ETL TOGAF OPA OpenLineage LangGraph 📌…
2. Engineering the Foundation: A Production-Grade Development Environment
Series: Building an Agentic Data Platform | Part 2 of 17Reading time: ⏳ ~11 minutesTags: 🏷️ Docker Compose development environment 12-Factor App Kafka MinIO OPA Prometheus Grafana…
1. From Scripts to Sentience: Building an Agentic Data Platform
Series: Building an Agentic Data Platform | Part 1 of 17Reading time: ⏳ ~12 minutes 📌 TL;DR Most data engineering tutorials teach you to move data from…