Senior Data Engineer | Azure

Description

Job Summary: We are seeking a Senior Data Engineer to drive data architecture evolution, implement and maintain data pipelines, and refactor the ingestion architecture within a dynamic environment. Key Highlights: 1. Participation in data architecture evolution and pipeline implementation. 2. Collaboration with technical stakeholders to deliver scalable and efficient solutions. 3. A relaxed and dynamic environment with continuous learning. We are passionate about technology, creativity, and challenges. If you enjoy challenges, continuous learning, and value personal connections, join us! \# We value diversity and believe it is fundamental to innovation and delivering value to our customers. All our positions are open to everyone, with or without disabilities, regardless of age, gender, sexual orientation, ethnicity, religion, or any other characteristic. If you identify with this role, come join our team! **WHAT WE’RE LOOKING FOR?** We seek a **Data Engineer**, at the **Senior** career level, who wants to work with us in a relaxed and dynamic environment with continuous learning while developing large-scale projects alongside major national and international clients. We have offices in Maringá, São Paulo, and Chicago (USA), but our work model is fully remote—we prefer to call it *anywhere office*. **WHAT WILL THIS PROFESSIONAL DO?** * Support defining and evolving the **data architecture** (medallion, batch, CDC, streaming). * Directly implement and maintain **pipelines** on Databricks using Python, SQL, and Spark. * Participate in the **refactoring of the ingestion architecture**, replacing current processes with direct ingestion from MongoDB. * Develop and adjust notebooks for data ingestion and transformation (e.g., fix tables, add columns, review logic). * Ensure data quality, consistency, and governance, supporting the team’s technical decision-making. * Collaborate with technical stakeholders to propose scalable, secure, and efficient solutions. **WHAT IS REQUIRED FOR THIS POSITION?** * Solid experience with **Python** and **SQL**. * Knowledge of **Spark** and **data architecture** (batch and streaming, CDC, medallion). * Experience with **Databricks**, with desirable knowledge of MongoDB and Kafka. * Ability to design ingestion and data processing architectures for batch and/or streaming scenarios. * Hands-on experience with **Azure**, especially **Cosmos DB** and **Data Factory**. * Proactive mindset to work closely with a **tech lead**, with project and architectural vision. **WHAT WOULD BE A PLUS?** * Knowledge of **shell script**. * Prior experience with complex data ingestion scenarios (e.g., streaming \-\> batch, MongoDB integrations). * Familiarity with best practices in data governance and cloud pipeline optimization. **PROCESS STEPS:** * Application * Cultural fit interview * Technical interview * Client interview * Hiring * ***Note: It is essential to ensure no conflicts of interest or affiliations that could compromise confidentiality or impartiality. This criterion will be observed from the outset of the selection process.***