Job Summary: A professional to develop end-to-end data pipelines and data models on Azure/Databricks, ensuring quality, performance, security, and predictability in a consulting environment. Key Highlights: 1. Lead the development of data pipelines and data models on Azure/Databricks. 2. Design and implement data ingestion and transformation processes. 3. Conduct code reviews and provide technical mentorship to the team. **Role in the Company:** Lead the development of end-to-end data pipelines and data models on Azure/Databricks, ensuring quality, performance, security, and predictability of deliverables in a consulting environment. **Responsibilities:** Design and implement Bronze/Silver/Gold domain-oriented data ingestion and transformations. Define and apply standards: naming conventions, partitioning, Z-Order, optimizations, Liquid Cluster, etc. Orchestrate workflows in ADF/Jobs; parameterization, Key Vault integration, and credential best practices. Implement observability (logs, metrics, alerts). Apply governance (Unity Catalog: catalogs, schemas, grants, RLS/masking where applicable). Conduct code reviews, mentor the team, and serve as the technical interface with the client. Estimate effort, plan sprints, and communicate risks/impediments. Investigate and resolve low-to-medium-complexity incidents, such as job failures, connectivity errors, etc. **Requirements:** Databricks Fundamentals. Advanced SQL (window functions, CTEs, tuning) and solid PySpark (joins, narrow/wide transformations, native functions vs UDFs). Databricks (Jobs/Workflows, Delta Lake, optimizations, managed/external tables, Unity Catalog). Azure Data Factory (Integration Runtimes, triggers, parameters/variables, integrations). Git + CI/CD (GitHub Actions/Azure Pipelines) for notebooks/Jobs/ADF. Security (RBAC, secrets management, networking) and basic FinOps. Continuous Data Quality and monitoring of drifts/anomalies. Consultative communication and translation of business requirements. **Nice-to-Haves:** DP-203 or DP-600 or DP-700. Intermediate to advanced English (B1/B2). Databricks Data Engineer Associate. Databricks Spark Developer Associate. Dimensional/vault modeling; data products/data mesh. Cost optimization (right-sizing, autoscaling, caching) and SLOs. * **Our Benefits:** * **Health Insurance Allowance**: Monthly financial support to assist with your private health insurance plan. * **Wellhub**, to keep your body and mind active — your way. * **Online Therapy**, fully covered by the company, because mental health matters. * **Online Nutrition Consultations**, up to two sessions per month to support your nutritional health. * **Life Insurance**, with a policy value of BRL 125,000, providing greater security for you and your family. * **Birthday Day Off**, because your special day deserves celebration. * **Paid Time Off**, to recharge your energy. * **Internal Gamification**, turning achievements into rewards and recognition. * **Educational Partnerships** **with universities**, including FIAP, Anhanguera, and Instituto Infnet, to support your growth and learning. * **Technical Certification Bonus**, recognizing and rewarding your efforts to learn. ***We value every voice and every individual, knowing that diversity makes us more innovative and stronger.***
João Silva
Indeed · HR
João Silva
Indeed · HR
