Data Engineer (Mid-level)

Description

**Mission:** Act in the modernization of the data platform of one of our largest clients in the financial segment, leading the migration of solutions from the *legacy* environment to a *Lakehouse* architecture based on *Databricks*. The mission of the position is to transform business rules into scalable and high-quality data solutions, developing the consumption layer with *DBT* and contributing to the continuous evolution of the platform through the application of engineering, modeling, and governance best practices. In an environment of high technical complexity and large data volume on *Azure* infrastructure, a collaborative performance is expected, with a focus on consistent deliveries, communication with business areas, and the construction of an increasingly efficient, organized, and prepared platform for future challenges. **Simplified application: https://rh\-evoluum.web.app/opportunities/engenheira\-o\-de\-dados\-pleno** **Required knowledge/requirements:** * Experience in ***Python*****,** ***SQL*****,** ***PySpark*** **and, mainly,** ***DBT***; * Experience with ***Delta Lake*****,** ***Databricks*** and *Unity Catalog*; * Experience in ***Azure*** infrastructure; * Relational and non-relational databases. * Experience with code versioning (*Git*) and *CI/CD* processes (***Azure DevOps***); * Solid knowledge of *Lakehouse* architecture. **Desirable differentials (not mandatory):** * Previous experience in **financial institutions.** * LGPD and **GDPR** (General Data Protection Regulation). * API ingestion. **Responsibilities:** * Act in the migration of data solutions from the *legacy* environment. * Analyze and understand existing business rules, ensuring their correct implementation in the new environment. * Develop and maintain the consumption layer using *DBT*, ensuring the quality, organization, and consistency of the data provided. * Implement data models following engineering, modeling, and governance best practices. * Work in a *Databricks* (*Lakehouse*) environment, using *Unity Catalog* as the structure for data organization and governance. * Sustain the development lifecycle through versioning in ***Git*** and deployment automation via ***CI/CD*** pipelines (*Azure DevOps*), ensuring that production deliveries are safe and traceable. * Contribute to the continuous evolution of the environment. * Interact with business areas for requirements gathering and understanding of business rules. * Collaborate with other team members in the construction of a modern and scalable data environment. **What you will find here:** * 100% remote position (anywhere office). * Performance evaluation, individual development plan, Investing in You Program. * Health and well\-being program, health and dental plan. * Gamified challenges and a store to buy products with your points. * Birthday Day Off. * Flexible voucher for daily expenses. * Rewarding Referral Program. * Life Insurance. Privacy Notice \- Recruitment \& Selection: https://drive.google.com/file/d/1LV2tTa\_PeSbyiXuqrG\_6WJw0jrEqRxnA/view?usp\=sharing Payment: R$8\.000,00 \- R$9\.800,00 per month Selection question(s): * Have you worked with DBT? If so, for how long? * Have you worked with PySpark for data processing? * Do you have practical experience with Databricks and Delta Lake? * Have you used Unity Catalog for data governance? * Have you worked with Azure? * Please inform which Microsoft Azure services you have used. Example: Azure Data Factory (ADF), Azure Synapse Analytics, Azure Databricks, Azure Data Lake Storage (ADLS), Azure SQL Database, Azure Event Hubs, Azure Functions, Azure DevOps for data pipelines, among others. Work location: Remote