Senior Data Engineer / Data Engineering Tech Lead

Description

Job Summary: Technical reference in Data, ensuring platform stability and scalability, leading architecture initiatives and enabling the team to perform automated operations. Key Highlights: 1. Lead the evolution of architecture and empower the Data team 2. Serve as the technical guardian of data architecture and mentor the team 3. Collaborate with business stakeholders to ensure data reliability **Job Mission** ------------------- Be the technical reference for the Data area — ensuring stability, scalability, and governance of data platforms (AWS \+ GCP), leading architectural evolution and empowering the team to achieve predictable and automated operations. **Main Responsibilities** -------------------------------- * Serve as the technical guardian of data architecture (Airflow, Airbyte, Spark/PySpark, Kubernetes/EKS, EMR, S3/Lake, Redshift, BigQuery, DataFlow). * Lead the refactoring of the critical closing pipeline (CMRR / RH \+ DP) into a more modular, automated, and observable model. * Establish development standards, versioning practices, and testing protocols (Git, DEV/STG/PRD environments, CI/CD, Data Quality, Lineage). * Support governance across multiple data lakes and warehouses, reducing ingestion duplication and promoting layer reuse. * Act as a technical reference and mentor for the current team (Analytics Engineers, DataOps, and Interns), elevating overall technical proficiency. * Engage directly with business stakeholders when necessary, understanding calculation rules and ensuring data reliability. Desired Technical Stack Area Technologies Ingestion \& Orchestration Airflow, Airbyte, Lambdas, DataFlow, ScheduledQueries Distributed Processing PySpark (strong), Spark SQL, EMR, Dataproc, DLT (desirable) Cloud \& Infrastructure AWS (EKS, EMR, Lambda, S3, Redshift), GCP (BigQuery, DataFlow, Composer) Infrastructure as Code / DevOps Terraform, Docker, Kubernetes, CI/CD Data Quality \& Lineage Great Expectations, DataHub or equivalent Governance \& Security IAM, VPN, environment/staging segregation