Projects

Sector: Aviation

Production

BI platform — European airline

From zero to the single source of institutional data, on AWS, with a team of 3.

Architecture diagram

A European airline had no single source of data: every department lived in its own spreadsheet. With a team of 3, I designed and led an end-to-end data platform on AWS that went from 0 to institutional source in ~6 months.

The outcome was not just technical: the president mandated it as the company-wide single source of truth. The entire infrastructure runs in production for ~$360/month, with cost efficiency as a design criterion from day one, not a final cut.

Key metrics

Entire infra in production
~$360/mo Entire infra in production
Dashboards (36 Superset + 2 Grafana) · 11 depts
38 Dashboards (36 Superset + 2 Grafana) · 11 depts
Compression: 16.6 GB → 0.82 GB
95% Compression: 16.6 GB → 0.82 GB
Records · 136 integrated sources
22M Records · 136 integrated sources

Stack

  • AWS
  • S3 Parquet
  • Lambda
  • Step Functions
  • RDS PostgreSQL
  • Superset
  • Grafana
  • ECS Fargate
  • Terraform

Decisions and trade-offs

  1. Orchestration on Lambdas, not Glue

    Orchestration runs on Lambdas at ~$7/month instead of AWS Glue (~$550/month): ~$6.5k/year avoided in orchestration alone, with the same functional result for the actual workload volume.

  2. PostgreSQL data mart, not a data warehouse

    Entire infra at ~$360/month versus ~$2,000/month for an equivalent Redshift (~$19.7k/year avoided). At this scale, a well-modeled PostgreSQL data mart covers analytical demand without a DW’s fixed cost.

  3. Silver framework: 95% compression

    16.6 GB of raw Avro dropped to 0.82 GB of Parquet in the Silver layer: 95% less storage and cheaper downstream reads, thanks to columnar format and strict typing.

  4. Everything as infrastructure as code

    17-19 Terraform modules: the whole platform is reproducible and auditable. Grafana embedded with JWT RS256 SSO, so there’s no second identity to manage.