Data platform
Veyt2022 – presentvia Cefalo
200+pipelines in production
The pipelines, APIs and agent tooling behind Veyt's published curves for green-energy markets.
100%50%
runtime, slowest jobs
100%60%
build time
99.4 ms0.193 ms
core lookup on the timeseries API
- Product
- Veyt sells market intelligence across every green-energy market it covers — carbon, guarantees of origin, power purchase agreements and renewable fuels. Traders and analysts act on the published curves the same day they land.
- What I did
- I architected the ingestion layer that reaches dozens of external registries and APIs, and I build and run the pipelines feeding all of it: Python ETL publishing timeseries, timeseries groups and forecast curves from upstream sources that each define a schedule and a format differently, orchestrated with Prefect and shipped to Cloud Run through GitHub Actions — including the sources that publish nothing and have to be scraped with BeautifulSoup and Playwright. I cut processing time on the slowest jobs by 50% with multiprocessing and by removing the code paths that forced them to run serially, and took 40% off build times by reworking the Docker layers. On the timeseries API I rewrote the queries and indexes — one core lookup went from reading 1,141 rows to 19, and from 99.4 ms to 0.193 ms — which took database compute from ~1.75 to ~0.91 CU-h an hour. I also build the REST APIs the platform exposes, and the MCP servers that give agents authenticated, typed access to the same systems.
- Python
- FastAPI
- ETL
- Prefect
- Pandas
- Playwright
- Timeseries
- MCP
- PostgreSQL
- Redis
- Docker
- GCP Cloud Run





