DevJobs

Senior Data Engineer

Overview
Skills
  • Python Python
  • Kafka Kafka
  • Spark Spark
  • Linux Linux
  • Docker Docker
  • Kubernetes Kubernetes
  • Airflow Airflow
  • Hadoop
  • Nifi
  • VMs

We are looking for a talented big data software engineer to join our cyber defense product group and skillfully plan and build our data pipes and processing flows.

The successful candidate should have hands-on experience with big data processing technologies and architectures.

The candidate will join our growing team of analysts, developers, data scientists, and architects who design and develop innovative solutions to the most critical needs of our customers. This is a unique opportunity to join the cyber and big-data revolution, learn, influence and work on meaningful challenges.


Responsibilities

  • Designs, architects, implements, and supports our data pipes and processing flows.
  • Work with data scientists and and analytics department in analyzing and understanding data sources.
  • Design and provide insights on database technology and data modeling best practices.


Requirements

  • Be very familiar with frameworks such as Spark, Hadoop, Kafka, Nifi and Airflow.
  • 4 years of experience as big data engineer.
  • Previous experience in configuring and managing multi-source data flows and processing of large data sets.
  • Experienced with Linux environment
  • Fluent in Python and can easily navigate in big data store technologies.
  • Maintaining data integrity and security and managing data and analysis flows requires attention to details, high responsibility and much talent, therefore we seek excellent candidates with a high sense of responsibility, accountability and details orientation, as well as strong inter-personal skills and ability to perform well as part of a professional team.


Advantages

  • Experience in architecting and configuring virtualization, processing cluster systems and micro services environments (VMs, Dockers, Kubernetes) is an advantage.
KayHut