You'll own our data lake - the system of record for millions of financial events every day, including bets, wallet movements, and live odds, across 12M+ active users.
You'll be responsible for how data is ingested, stored, retained, and governed across our data infrastructure, built on S3 and Snowflake/Databricks. Your work will ensure that analytics, finance, and regulatory teams have access to accurate, reliable, and fully reconciled data.
What You'll Do:
- Lakehouse Architecture: Own the lakehouse architecture, including bronze/silver/gold layers, Iceberg/Delta tables, and schema evolution.
- Data Ingestion: Land operational data via CDC streaming (Kafka, Debezium), handling late and duplicate events.
- Performance & Cost Optimization: Design data layouts for speed and cost efficiency, including partitioning, compaction, file sizing, and query performance on Trino/Athena/Snowflake.
- Data Retention & Archival: Own storage tiering, regulatory retention, immutability, and GDPR deletion.
- Data Quality & Reliability: Ensure data correctness through freshness SLAs, drift detection, and reconciliation against source wallet and ledger systems.
- Data Governance: Own catalog and lineage, row/column access control, PII masking, encryption, and audit trails.
- Monitoring: Monitor ingestion health, data anomalies, and cloud storage/compute costs.
Requirements:
- 5+ years of experience in Data Engineering, with proven ownership of large-scale data lake or lakehouse architectures.
- Production experience with cloud object storage (S3/GCS) and Snowflake, Databricks, or BigQuery.
- Hands-on experience with an open table format (Iceberg, Delta, or Hudi).
- Deep understanding of partitioning, file layout, and query optimization at terabyte-plus scale.
- Experience with CDC and streaming ingestion (Kafka + Debezium or equivalent).
- Strong SQL and data modeling skills, with a solid understanding of relational databases and OLTP systems, particularly in the context of financial ledgers.
- Experience with data lake governance, including catalogs, lineage, access control, PII, and retention.
- Python and an orchestration tool (Airflow/Dagster) as working tools. This is an architecture and ownership role, not a heads-down coding position.
- Proven hands-on experience designing, building, and deploying AI agents and LLM-powered workflows in production. This goes beyond using existing AI tools or APIs and requires actual development and implementation experience.
Nice to have:
- Experience in fintech, iGaming, or another regulated, audit-heavy environment.