Moonshot is seeking a Senior Data Engineer with a strong background in building scalable data architectures and pipelines. The ideal candidate will have a passion for data and a proven track record of managing complex data systems. In this role, you will collaborate with cross-functional teams to enhance data quality, availability, and accessibility, enabling data-driven decision-making across the organization.
Requirements:
- Minimum of 5 years of experience in data engineering, with a focus on data pipeline development.
- Strong proficiency in SQL and experience with NoSQL databases.
- Experience with data processing frameworks such as Apache Spark or Apache Kafka.
- Familiarity with cloud platforms like AWS, Google Cloud, or Azure for deploying data solutions.
- At least 3 years of experience with ETL tools and data integration frameworks.
- Solid understanding of data warehousing concepts and experience with tools like Snowflake or Redshift.
- Excellent problem-solving skills and the ability to work in a collaborative team environment.
Responsibilities:
- Design, develop, and maintain robust and scalable big data pipelines and ETL processes.
- Define and implement data architectures and infrastructures that support analytics and reporting needs.
- Automate client data collection and transformation workflows to ensure high data quality and consistency.
- Manage and optimize cloud-based databases and tables.
- Build APIs for seamless integration with various data sources.
- Implement and maintain logging and monitoring systems (e.g., Elasticsearch clusters).
- Collaborate with cross-functional teams to deliver end-to-end data solutions.
- Ensure data security, reliability, (integrity)and compliance with best practices.