Case study2024
Telecom data platform
500+ Pentaho pipelines that keep a telecom operator's 15+ internal systems in sync with one PostgreSQL database.
problem A telecom operator's data lives in 15+ internal systems and has to stay in sync with one central PostgreSQL database, millions of records a day.
What I built
- I develop and maintain 500+ Pentaho Data Integration pipelines into a central PostgreSQL database, handling millions of records daily.
- I built custom Java components that stream Apache Kafka data into PDI, beyond its built-in steps.
- I build Power Apps Canvas apps for the business users on top of the data.
| pipelines | 500+ |
|---|---|
| sources | 15+ internal systems |
| target | PostgreSQL |
| streaming | Kafka via custom Java steps |
| dates | Oct 2024 – present |
| outcome | 15+ internal systems kept in sync with one PostgreSQL database, millions of records a day. |
- Pentaho PDI
- PostgreSQL
- Apache Kafka
- Java
- Power Apps