Description
We are seeking for experienced Data Engineer,
The ideal candidate is a self-motivated, multi-tasker, and demonstrated team-player.
You will be responsible for designing, developing, managing and maintaining our open-source data platform, including our Data-Lakehouse (S3 & Delta Lake & Clickhouse), ETL processes and orchestration tool (Temporal Workflow).
What You Will Do
- Develop a scalable data platform integrating multiple sources for easy access.
- Design and enhance data tools (orchestration, governance, Data-Lakehouse, BI, etc.).
- Ensure smooth operation of data systems for analysts, scientists, and engineers.
- Optimize data pipelines (ingestion, processing, and output) in a microservices environment.
Requirements
Must:
- 3+ years of experience in a data engineering-related position
- SQL expertise, including working with various databases, data warehouses, third-party data sources, and AWS cloud services
- Proficient in Python
- Experience in building, designing, and optimizing data pipelines
- Self-driven, can-do attitude
Nice To Have
- Familiarity with Ruby or other relevant languages
- Experience with Spark
- Experience with Delta Lake
- Experience with ClickHouse
- Experience with Temporal Workflow
- Familiarity with ERP systems (big advantage)
- Familiarity with supply chain systems (advantage)
- Passion for open-source tools