DevJobs

Data Engineer

Overview
Skills
  • Python Python ꞏ 3y
  • SQL SQL ꞏ 3y
  • Ruby Ruby
  • Spark Spark
  • ClickHouse
  • Delta Lake
  • Temporal Workflow
Description

We are seeking for experienced Data Engineer,

The ideal candidate is a self-motivated, multi-tasker, and demonstrated team-player.

You will be responsible for designing, developing, managing and maintaining our open-source data platform, including our Data-Lakehouse (S3 & Delta Lake & Clickhouse), ETL processes and orchestration tool (Temporal Workflow).

What You Will Do

  • Develop a scalable data platform integrating multiple sources for easy access.
  • Design and enhance data tools (orchestration, governance, Data-Lakehouse, BI, etc.).
  • Ensure smooth operation of data systems for analysts, scientists, and engineers.
  • Optimize data pipelines (ingestion, processing, and output) in a microservices environment.

Requirements

Must:

  • 3+ years of experience in a data engineering-related position
  • SQL expertise, including working with various databases, data warehouses, third-party data sources, and AWS cloud services
  • Proficient in Python
  • Experience in building, designing, and optimizing data pipelines
  • Self-driven, can-do attitude

Nice To Have

  • Familiarity with Ruby or other relevant languages
  • Experience with Spark
  • Experience with Delta Lake
  • Experience with ClickHouse
  • Experience with Temporal Workflow
  • Familiarity with ERP systems (big advantage)
  • Familiarity with supply chain systems (advantage)
  • Passion for open-source tools
Onebeat