Senior Data Engineer

Description

Leega is a company focused on delivering efficient and innovative service to its customers. This could not be any different with our primary fuel: people! Our culture is inspiring, and our values are present in everyday life: ethics and transparency, quality excellence, teamwork, economic, social and environmental responsibility, human relations, and credibility. We seek innovative professionals who are driven by challenges and focused on results. If you are looking for a dynamic and collaborative company that invests in its employees through continuous training, Leega is the place for you! **>> LEEGA IS FOR EVERYONE — we would be thrilled to welcome you to our team. Join us in shaping our history and building our future.** Apply now to our open positions! **Responsibilities and Duties** **About the Opportunity** You will architect and evolve the data lake—the company’s central nervous system for data—serving as the foundational layer that powers, in real time, the dynamic pricing engine, ML models, and commercial intelligence across the group. This is an ownership role: you define the multi-tenant Lakehouse architecture—from streaming to the semantic layer—and are accountable for its reliability, governance, and cost efficiency. **Your Challenges** * **Lakehouse Architecture —** Design and evolve the Apache Iceberg-based data lake on S3—with well-defined layers, partitioning and compaction, time-travel capabilities, and support for DELETE/UPDATE to comply with LGPD. * **Streaming and CDC —** Build real-time ingestion (Kafka, Flink, CDC with Debezium) with controlled schema evolution (Schema Registry) and delivery guarantees. * **Transformation and Orchestration —** Model the transformation layer using dbt and orchestrate batch and quality workflows in Airflow—from crawler to backfill. * **Semantic Layer —** Maintain metric definitions in Cube.js—the single source of truth feeding BI tools and AI agents, ensuring enterprise-wide consistency. * **Access and Performance —** Operate federated and low-latency OLAP queries over the lake, with cost isolation, tenant-based access, and performant queries. * **Quality, Lineage, and FinOps —** Ensure data testing, lineage tracking, and cost efficiency, maintaining platform reliability as it scales. ***You’ll work side-by-side with data scientists, ML engineers, and business teams.*** **Requirements and Qualifications** **Stack & Tools** * **Lakehouse & Storage:** Apache Iceberg, Amazon S3, Athena + Glue (Trino + Hive in future evolution) * **Streaming & Ingestion:** Kafka (MSK), Apache Flink, Debezium (CDC), Schema Registry * **Transformation & Orchestration:** dbt, Apache Airflow * **Semantic Layer & OLAP:** Cube.js, ClickHouse, Apache Pinot, Trino * **Governance & Quality:** OpenMetadata, Lake Formation, Great Expectations * **Infrastructure & DevOps:** EKS/Kubernetes, ArgoCD, Crossplane, GitHub, Claude Code **What We’re Looking For** **Essential** * Proficiency in SQL and query optimization in distributed environments (minimum 5 years). * Python with solid experience in PySpark or distributed processing. * Orchestration (Airflow), ELT, and dbt applied at scale (minimum 4 years). * Streaming (Kafka, Flink) and Lakehouse architectures using Apache Iceberg (minimum 3 years). * Strong understanding of data governance, quality, and modeling. * Comfort with AI-assisted development (Claude Code). **Nice-to-Have** * CDC (Debezium) and low-latency OLAP (ClickHouse, Pinot, Trino/Athena). * Semantic Layers (Cube.js, dbt) and Data Mesh architectures. * Governance and catalog solutions (OpenMetadata, Lake Formation). * Vector databases (Qdrant) and ML-oriented data pipelines. **Additional Information** **Remote Work** **Project Duration:** 6 months, with possibility of extension or internalization. At Leega, we don’t just deliver lines of code or dashboards. We transform complex technological challenges into real impact. With over 15 years of experience, we combine the depth of human expertise with the speed of Artificial Intelligence to build solutions that reshape businesses—and, above all, improve people’s lives. We are a technology consultancy—agnostic and strategic. With offices in Brazil and Europe, our team of more than 580 multidisciplinary talents lives and breathes the Data Analytics, Cloud, and AI ecosystem. We master the full foundation—from Governance to Engineering—to ensure technology is not merely a tool, but a sustainable competitive advantage for the market’s top players.