Sector: Aviation
ProductionData & BI platform — European airline
From zero to the company's single source of data, declared by the presidency.
A European airline with ten departments and executive management made decisions on data that lived scattered across files, source systems and spreadsheets. There was no institutional figure: every area came to committee with its own.
I designed and built the complete platform: 36 dashboards in production over 136 data sources, ten departments plus management, ~50 users with access through the client's corporate directory. The presidency declared it the company's single source of data. It closed on time, with no pending functionality, after twelve months.
Key metrics
- Dashboards in production, each with its runbook · 10 departments + management
- 36Dashboards in production, each with its runbook · 10 departments + management
- Operational sources —SFTP, APIs and databases— across 400 artifacts in three layers
- 136Operational sources —SFTP, APIs and databases— across 400 artifacts in three layers
- Infrastructure cost per user per month ($4,767 over twelve months, 50 users)
- ~€9.5Infrastructure cost per user per month ($4,767 over twelve months, 50 users)
- Annual saving against conventional alternatives, depending on scenario (2–7× more expensive, official AWS prices)
- $4,700–$30,700Annual saving against conventional alternatives, depending on scenario (2–7× more expensive, official AWS prices)
Stack
- AWS
- S3 Parquet
- Lambda
- Step Functions
- RDS PostgreSQL
- Superset
- Grafana
- ECS Fargate
- Terraform
Decisions and trade-offs
The data warehouse that turned out unnecessary
The client came asking for a data warehouse. I measured the real volume, modelled the alternatives against official AWS prices across seven scenarios, and brought them the numbers: the platform cost $4,767 a year; a minimum viable Redshift, twice that; a comfortably sized one, up to seven times more. With that comparison in front of us, the decision was made together. The cost: these are models with declared assumptions, not quotes — what I stand behind is the order of magnitude, not the exact figure. And sizing to real usage leaves less headroom for load spikes.
The data lake, and the door it leaves open
Medallion on S3 with Parquet, and PostgreSQL only as the serving layer. Without the data lake the same data lives entirely in the database: $9,844 a year against $4,767, and the instance goes from $61 to $401 a month. The storage layer alone costs 18.3 times more —$81 against $1,480 a year— through unit price and through losing Parquet compression: 10:1 observed, against ~3:1 in the database. What does not show up on the invoice is what weighs most for the future: the data stays in an open format on object storage, so a lakehouse is an engine on top, not a migration — and with it, AI-driven analytics over the same storage, without moving a byte.
Orchestration on Lambdas, not Glue
Glue bills per minute of reserved work; Lambda, per execution. Measured at the migration over the Silver/Gold layer: ~$550 a month on Glue against ~$29 on Lambda, about nineteen times less. The platform has grown since —more sources, more memory per function— and Lambda now runs at ~$72 a month: at the same ratio, Glue would be around $1,374, roughly $15,600 a year apart. The saving does not thin out with scale: it grows with it, because you pay per execution and not per clock. The cost: packaging the framework into Docker images to get past the 50 MB Lambda limit.
Infrastructure as code, in full, with no exceptions
23 Terraform modules and not one configuration done by hand in the console: every architecture decision stays versioned and the platform rebuilds from the repository. The cost is discipline sustained over twelve months — any console shortcut would have been faster that afternoon and would have broken reproducibility forever.
What the client said at the close of Phase 1
I just wanted to say congratulations on the excellent work done throughout this year. I have gone over the dashboards and the tool, the manuals — honestly, every detail shows everyone's care and professionalism.
On top of that, the attentiveness and willingness: it is real teamwork, where you can even see the contribution to improve what already existed manually.
Still a few little things to mesh and adjust, of course, but the result is incredible. Be proud of the work done, because it is high quality.
With the day-to-day madness I had not been able to thank you for all the effort. We value it enormously, and I did not want the close of Phase 1 to slip by unnoticed.
I think the tool speaks for itself about all the potential still to be developed!