The problem
Teams want reporting, audit trails and analytics, but a traditional data warehouse is expensive to run and needs its own ETL. Most of the data already exists as events flowing through the platform.
Key design decisions
- 01
Capture events at the bus
An EventBridge rule forwards matching domain events to Amazon Data Firehose, so producers do not change to feed the lake.
- 02
Columnar, partitioned storage
Firehose converts events to compressed Parquet using a table schema defined in the Glue Data Catalog and writes them to date-partitioned prefixes in S3, which keeps Athena scans, and therefore costs, small.
- 03
A catalog instead of a warehouse
Athena runs standard SQL directly against S3 using the catalog's table definitions, and partition projection finds new date partitions without scheduled crawlers or servers to manage.
- 04
Lifecycle rules for old data
Older partitions move automatically to cheaper storage classes that remain queryable, and only data you no longer need to query moves to archive tiers.