Senior Data Engineer (Azure/Databricks Specialist)

Description

Job Summary: We are looking for a talented and passionate individual to build, maintain, and manage data pipelines, supporting the development and operation of our clients' data platforms. Key Highlights: 1. Build and maintain data ingestion and processing pipelines 2. Develop ETL/ELT processes using Python, Airflow, and Azure Data Factory 3. Work with Azure Databricks, Spark/SparkSQL, and various databases ***Keyrus believes in diversity and inclusion. We encourage everyone to apply for our hiring process, regardless of gender, age, race, or religion. We do not tolerate any form of discrimination. This principle is reinforced throughout our hiring process and actively practiced within the company.*** We know great results are achieved only with a strong team; therefore, we seek talented and passionate individuals who desire professional growth and wish to build a long-term career with us. **Roles and Responsibilities:** * Build and maintain **data ingestion and processing pipelines** from multiple sources. * Develop **extraction, transformation, and loading (ETL/ELT)** processes using **Python, Apache Airflow, and Azure Data Factory**. * Structure and manage data in a **Data Lake**, ensuring information organization and governance. * Develop data transformations and processing using **Spark / SparkSQL on Databricks**. * Support, monitor, and enhance the client's **data platform**, ensuring stability and performance. * Work with various **databases and storage technologies**, such as SQL Server, Oracle, MongoDB, and Hadoop. * Collaborate with multidisciplinary teams in a dynamic, data-driven environment. **Essential Qualifications:** * Solid experience with **Azure Databricks** * Knowledge of **Azure Data Factory** and **Azure Data Lake**. * Experience developing with **Python**. * Knowledge of **Apache Spark / SparkSQL**. * Experience with **Oracle and SQL Server databases**. **Preferred Qualifications:** * Experience with **agile methodologies (Scrum or Kanban)**. * Knowledge of **Git / Azure DevOps**. * **Cloud certifications**, preferably **Microsoft Azure**. * **English** proficiency for reading and technical communication. Our mission is to help companies unlock the full potential of Data and Digital to improve performance, drive transformation, and generate new growth levers and competitiveness. Join the Keyrus Team! If you'd like to learn more about us, we invite you to visit our website: http://www.keyrus.com/br/ ***Good luck!***