As a Senior Data Platform Engineer, you will:
- Be a part of a cross-functional team of data, backend, and DevOps engineers.
- Work closely with Data Scientists to understand needs and design for optimized data layout and infrastructure used in data ingestion for ML and analytics use-cases.
- Design and implement scalable data serving capabilities for various stakeholders within the company.
- Monitor and improve data pipelines to enforce high data quality.
- Experiment and learn various technologies in the domain of big data and high throughput computing.
- Drive for continuous innovation and improvement in Final’s data stack.
Requirements:
- At least 5 years of server-side development experience working in a high-scale environment.
- Proven understanding in designing, developing, and optimizing complex solutions that move and/or manipulate large volumes of data.
- Proficient in one or more general-purpose programming languages, including but not limited to C++, Java, .Net, Rust, Python.
- Python data analytics libraries experience (pandas, polars, etc.).
- Comfortable with learning new languages and technologies.
- Strong sense of ownership.
- Excellent problem-solving skills.
- BSc/MSc in Computer Science/Engineering.
Nice to have:
- Sound understanding of different big data file and table formats such as Parquet, Delta Lake, AVRO, Iceberg, HDF5.
- SQL experience.
- Hands-on experience with distributed query engines such as Spark and Hadoop.
- Experience with implementing open-source solutions on existing large-scale systems.
- Experience working on large-scale and complex on-premises systems.
- Familiarity with cloud data platforms (Databricks, AWS, Azure, etc.).
- Experience with data pipelining tools such as Airflow, Kubeflow, or similar.
- Experience with ML concepts and processes.