Description
We are seeking an experienced Senior Data Engineer.
The ideal candidate is a self-motivated, multi-tasking team player with a proven track record of collaboration.
You will be responsible for designing, developing, managing, and maintaining our open-source data platform, including our Data Lakehouse (S3, Spark, Iceberg, and ClickHouse), ETL processes, and orchestration tool (Temporal Workflow).
What You Will Do
- Develop a scalable data platform that integrates multiple sources for easy access.
- Design and enhance data tools, including orchestration, governance, Data Lakehouse, BI, and more.
- Ensure the smooth operation of data systems for analysts, data scientists, and engineers.
- Optimize data pipelines—ingestion, processing, and output—within a microservices environment.
Requirements
- 5+ years of experience in a data engineering-related position.
- Experience with Spark.
- Experience with Iceberg.
- SQL expertise, including working with various databases, data warehouses, third-party data sources, and AWS cloud services.
- Proficient in Python (OOP + Processing packages like Polars).
- Experience in building, designing, and optimizing data pipelines.
- Self-driven, can-do attitude.
Nice To Have
- Experience with ClickHouse.
- Experience with Glue Data Catalogue.
- Experience with K8S.
- Experience with Temporal Workflow.
- Familiarity with ERP systems (big advantage).
- Familiarity with supply chain systems (advantage).