Senior Data Engineer

Description

Job Summary: We are seeking a Senior Data Engineer to design, build, and evolve scalable data pipelines, ensuring data governance, observability, and reliability. Key Highlights: 1. Designing, building, and evolving scalable data pipelines. 2. Working with medallion architecture (Bronze, Silver, and Gold). 3. Using tools such as Apache Airflow, Azure Data Factory, and Microsoft Fabric. We are seeking a Senior Data Engineer to design, build, and evolve scalable data pipelines. This person will be responsible for structuring end\-to\-end data flows—from ingestion through analytical consumption—ensuring data governance, observability, and reliability. **Responsibilities and Duties** Design, build, and evolve end\-to\-end data pipelines, covering ingestion, transformation, reprocessing, and delivery for consumption. Work with medallion architecture (Bronze, Silver, and Gold), ensuring clear quality criteria and layer-specific responsibilities. Develop data ingestion and transformation processes using Python for data (e.g., Pandas, PySpark). Ensure pipelines are idempotent, resilient, and observable, with appropriate use of logs, metrics, and alerts. Define and implement strategies for: * Automated retries * Failure handling * Incremental and full reprocessing Work with orchestration and ingestion tools such as: * Apache Airflow * Azure Data Factory (triggers, linked services, integration runtimes, and Mapping Data Flows) Work with Microsoft Fabric, including: * Data Engineering * Lakehouse * Notebooks * Pipelines Perform rigorous data validations, including reconciliations and cross-checks using Excel when necessary. Investigate and resolve data incidents with root\-cause analysis, proposing improvements to prevent recurrence. Work with CI/CD practices using Azure DevOps (Repos and YAML Pipelines), ensuring: * Code versioning * Environment promotion * Secure secret management via Azure Key Vault Clearly communicate risks, impacts, and technical decisions to both technical and non\-technical audiences. Provide technical support to the team, serving as a reference for best practices in data engineering. **Requirements and Qualifications** Solid experience as a Data Engineer in analytical data environments. Proficiency in building pipelines with Python for data (Pandas, PySpark, or similar). Experience with layered data architecture (medallion). Hands\-on experience with Apache Airflow and/or Azure Data Factory. Practical experience with Microsoft Fabric (Lakehouse, Notebooks, and Pipelines). Knowledge of pipeline reliability best practices (idempotency, reprocessing, fault tolerance). Experience with data observability (logs, metrics, alerts). Hands\-on experience with CI/CD for data using Azure DevOps. Strong critical analysis skills for data validation and quality assurance. Excellent communication skills for interacting with technical and business teams. **Here, we are \#SangueLaranja!** We have been in the market for 17 years, partnering closely with our clients to deliver transformative experiences. We are a global technology and innovation ecosystem; beyond Brazil, we operate in Europe and the UK, with offices in Portugal, London, Dubai, and the Netherlands. **F for Formation: We believe in practicing a culture of knowledge sharing, community awareness, and the transformative power of knowledge!** We run initiatives and social actions that foster development, such as the Orange Juice tech community, the Training Program, our leadership academy, and numerous partnerships with NGOs and Edtechs. **At FCamara, everyone is welcome. For us, Diversity, Respect, and Ethics are non\-negotiable elements embedded in our DNA.** **So, are you ready to join an amazing team and become the protagonist of your own story?**