Senior Data Engineer

Description

FCamara is seeking a Senior Data Engineer to partner with a leading Information Technology and Services company with over 40 years of market presence. This company is part of the daily lives of millions of Brazilians, developing solutions that positively impact food, health, transportation, and culture—ensuring more time and improved quality of life for businesses, workers, and their families. You will join a high-performance team, empowered with autonomy and responsibility to design and evolve robust, scalable, and secure solutions. **Responsibilities and duties** * Modernize and migrate legacy data platforms to modern architectures on Azure Databricks and Data Lakehouse. * Conduct assessments of legacy environments to identify opportunities for modernization and improvement. * Perform reverse engineering of ETL pipelines and workflows developed in Oracle and Informatica PowerCenter. * Design, develop, and implement scalable pipelines using PySpark, Spark SQL, and Delta Lake. * Migrate traditional ETL processes to ELT architectures, ensuring higher performance, scalability, and governance. * Develop and maintain batch and near real-time pipelines in distributed environments. * Implement and manage layered architectures (Bronze, Silver, and Gold) following the Medallion Architecture pattern. * Integrate solutions with Azure ecosystem services, such as Azure Data Lake Storage Gen2, Azure Data Factory, Databricks, and Azure Key Vault. * Apply data engineering best practices, including versioning, testing, observability, monitoring, and failure handling. * Implement resilience mechanisms such as retry, idempotency, and dead-letter. * Optimize Spark workload and SQL query performance to improve operational efficiency and reduce costs. * Implement and support data governance practices, including access control, data lineage, and metadata management. * Collaborate with business areas and technical teams to translate requirements into scalable and reliable data solutions. * Support the definition of modern data architectures aligned with concepts such as Data Mesh, Data Vault, and Medallion Architecture. **Requirements and qualifications** * Proven experience as a Senior Data Engineer. * Experience in data platform migration and modernization projects. * Advanced knowledge of Oracle and Informatica PowerCenter. * Hands-on experience with Azure Databricks, PySpark, and Spark SQL. * Knowledge of Delta Lake and Lakehouse architectures. * Experience with Azure Data Lake Storage Gen2 (ADLS Gen2), Azure Data Factory (ADF), and Azure Key Vault. * Practical experience building batch, ELT, and near real-time pipelines. * Knowledge of data modeling using Data Vault, Star Schema, and Medallion Architecture. * Experience orchestrating pipelines using Azure Data Factory and Databricks Workflows. * Knowledge of data governance, Unity Catalog, access control, and data lineage. * Experience optimizing Spark application and SQL query performance. * Familiarity with CI/CD practices, Git-based versioning, automated testing, and monitoring. * Experience implementing resilience mechanisms such as retry, idempotency, and dead-letter. * Ability to operate in mission-critical environments with focus on high availability, scalability, and governance. * Strong communication skills for interaction with business areas and cross-functional teams. **Nice-to-have:** * Strong experience in legacy platform modernization. * Experience performing reverse engineering of Informatica PowerCenter workflows and mappings. * Ability to translate complex business rules into scalable, high-performance pipelines. * Knowledge of Azure Databricks cost optimization and reduction strategies. * Experience integrating open-source solutions with cloud environments. * Architectural mindset—engaging not only in development but also in solution and technical standard definition. * Knowledge of modern data architecture frameworks, such as Data Mesh and Data Vault. **Here at FCamara, we are \#SangueLaranja!** We have been in the market for 17 years, side by side with our clients, delivering transformative experiences. We are a global technology and innovation ecosystem; beyond Brazil, we operate in Europe and the UK, with offices in Portugal, London, Dubai, and the Netherlands. **F for Formation: We believe in sharing culture, community spirit, and in the transformative power of knowledge!** We run initiatives and social actions promoting development, such as the Orange Juice tech community, the Training Program, our leadership school, and numerous partnerships with NGOs and Edtechs. **At FCamara, everyone is welcome. For us, Diversity, Respect, and Ethics are non-negotiable elements deeply embedded in our DNA.** **So, are you ready to join an amazing team and become the protagonist of your own story?**