DevJobs

Senior Data Engineer

Overview
Skills
  • Python Python
  • SQL SQL
  • Kafka Kafka
  • AWS AWS
  • GCP GCP
  • Snowflake Snowflake
  • Selenium Selenium
  • Airflow Airflow
  • Playwright
  • Scrapy
  • BigQuery
  • ClickHouse
  • Dagster
  • Prefect
  • Pub
  • Sub
About the Company

Adge is revolutionizing brand marketing by transforming performance alignment to unlock hidden potential. Over 90% of brand content currently falls short of profitability, but Adge bridges this gap through cutting-edge technology. Utilizing comprehensive data analysis and AI, we deconstruct marketing content into key attributes to create data-driven strategies that boost performance. We are replacing guesswork with advanced analytics, allowing brands to achieve greater impact and reduce costs.


About the Role

We are seeking a Senior Data Engineer with strong web scraping and data ingestion experience to design and own our data pipelines end-to-end. This role is ideal for someone who enjoys building robust systems that turn messy, high-volume external data into reliable, analytics-ready assets powering AI models and real-time insights.


Responsibilities
  • Design and maintain scalable ETL/ELT pipelines for structured and unstructured data
  • Build and operate scraping and data collection systems from diverse external sources
  • Ensure data quality, validation, enrichment, and lineage
  • Optimize data storage and query performance for analytics and ML workloads
  • Collaborate with data scientists on feature generation and model inputs
  • Support real-time and batch ingestion use cases
  • Improve reliability, monitoring, and cost efficiency of data infrastructure


Requirements
  • 6+ years of experience in Data Engineering
  • Formal education in Computer Science or a related field
  • Strong experience with Python for data pipelines and scraping
  • Proven experience with scraping frameworks (Scrapy, Playwright, Selenium, custom crawlers)
  • SQL knowledge and experience with analytical data stores (BigQuery, ClickHouse, Snowflake, or similar)
  • Experience with streaming or messaging systems (Kafka, Pub/Sub)
  • Hands-on experience with GCP or AWS
  • Experience with data orchestration tools (Airflow, Dagster, Prefect)
  • Strong understanding of data modeling and performance optimization
  • Experience in AdTech or a high-growth startup environment is a strong advantage
Adge.ai