Lead Data Engineer ID71008

Description

**About the role** We are looking for a **Lead Data Engineer** to own the data pipeline and analytical architecture layer for a large\-volume marketing analytics platform. You will make architectural decisions around partitioning strategy, file formats, schema design, and near\-real\-time processing for OLAP\-oriented workloads built on an S3\-backed data lake. You will design and govern ETL pipelines, define DAG\-based orchestration strategies using Airflow, drive the AWS data stack including Athena and EKS, and lead a team of senior developers while enforcing code quality standards. **What you will do** * Design and own ETL pipelines that extract, transform, and validate data from internal databases and external APIs at scale. * Make architectural decisions around partitioning, file formats, schema and data\-type strategy, and near\-real\-time processing for large\-volume, OLAP\-oriented data systems built on an object\-storage data lake. * Own the design of scheduled batch workflows (DAGs) on the client's Airflow setup, defining pipeline structure, dependencies, and triggering strategies, and driving architectural discussions around them. * Drive the use of the client's AWS data stack, including an S3\-backed data lake, Athena, and EKS/Kubernetes. * Partner directly with the client's DevOps team to clarify functional and non\-functional requirements. * Review pull requests and enforce code quality standards. * Guide senior developers and ensure alignment with the client's engineering practices. **Must haves** * **7\+ years of engineering experience**, with a proven track record designing and implementing ETL pipelines and making architectural decisions for large\-volume data systems. * Hands\-on experience with OLAP\-style analytical data architecture, with experience in Athena, Trino/Presto, BigQuery, Snowflake, Spark SQL, ClickHouse, or similar technologies. * Hands\-on experience designing data lakes backed by object storage such as **S3** or equivalent, including partitioning strategies, file formats such as **Parquet/ORC**, and cost/performance tradeoffs. * Deep familiarity with DAG\-style workflow definition and triggering, with substantial experience in **Airflow** or comparable orchestrators such as Dagster, Prefect, Luigi, or Step Functions. * Practical experience across the **AWS data ecosystem**, including S3\-backed data lakes, serverless query engines such as Athena or equivalent, and **EKS/Kubernetes**. * Strong backend proficiency in **Python**, with experience using FastAPI or Flask. * Comfortable working with **REST and GraphQL**. * Experience with **Docker and PostgreSQL** for transactional and application layers. * Highly comfortable working in Mac/Linux terminal\-centric environments. * Practical, hands\-on experience with AI\-assisted development tools such as Claude Code, combined with the critical judgment to challenge AI\-generated output when it compromises long\-term maintainability. * Leadership experience setting standards for responsible use of AI tooling, including identifying risky AI\-driven shortcuts during code review. * Strong communication and technical judgment, with the ability to defend technical decisions, challenge quick fixes with sound reasoning, and balance long\-term maintainability with pragmatic delivery. * Upper\-Intermediate English level. **Nice to haves** * Direct production experience with Athena. * Working knowledge of TypeScript and React, sufficient to guide integrations and review frontend\-adjacent pull requests. * Production experience building AI features using AWS Bedrock, LangChain, Pydantic AI, or similar technologies. * Experience with monorepo tooling such as Nx or modern package managers such as Poetry, UV, or Yarn. * Experience with Redis and caching layers or SageMaker. * Experience with marketing data structures, campaign management APIs, or digital advertising metrics. **Perks and benefits** * **Growth without limits:** build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget * **Competitive compensation:** get recognition that reflects your skills and impact, with regular performance and compensation reviews * **Flexibility:** work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm * **Meaningful, modern projects:** build impactful products using modern technologies alongside global teams and leading brands * **Collaborative culture:** join a supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized * **Well\-being \& support:** access local well\-being programs and people\-focused support tailored to your location Job Type: Full\-time Work Location: Remote