We’re looking for a strong Senior Software Engineer with Data Engineering experience.
Someone who is proficient in building robust, scalable, maintainable and thoroughly monitored data pipelines on cloud environments.
Responsibilities:
- Design and develop the data's team micro services - Java services running on K8S.
- Tackle data duplication, velocity, schema adherence (and schema versioning), high availability, data governance, and more.
- Develop, design, and maintain end-to-end ETL workflows, including data ingestion and transformation logic, involving different data sources.
- Enrich financial data through third-party data integrations.
- Develop and maintain our data pipeline written mostly in Java and running on K8S in a micro-service architecture.
- Plan and communicate integrations with other teams that consume the data and use it for insights creation.
- Ongoing improvement of the way data is stored and served. Improve queries and data formats to make sure the data is optimized for consumption by a variety of clients.
Requirements:
- Strong background as a software engineer with at least 2+ years experience with Java.
- 6+ years in data engineering, and data pipeline development on high-volume production environments.
- 2+ years experience with monitoring systems (Prometheus, Grafana, Zabbix, Datadog).
- Experience in object-oriented development. Should have strong software engineering foundations.
- Experience with data-engineering cloud technologies as Apache Airflow, K8S, Clickhouse, Snowflake, Redis, cache technologies and Kafka.
- Experience with relational and non-relational DBs. Proficient in SQL and query optimizations.
- Experience with designing infrastructure to maintain high availability SLAs.