Senior Data Engineer

Description

Job Summary: Senior Data Engineer to build and evolve robust and scalable data pipelines, focused on large-scale data volumes and the Databricks environment. Key Highlights: 1. Building and evolving robust and scalable data pipelines 2. Focus on large-scale data volumes and the Databricks environment 3. Transforming business requirements into efficient technical solutions **Positively impacting billions of lives is our purpose—and it can be yours too!** Founded in 2010 in Brazil, Semantix is a reference in Big Data, Analytics, and Artificial Intelligence. We are a team of innovation enthusiasts from diverse backgrounds and experience levels. What unites us is our shared motivation to transform our customers’ experiences through a data-driven culture. If this resonates with you, Semantix is your place. We’re excited to welcome you to our team. After all, **the future is built together**. Requirements: We are seeking a Senior Data Engineer to build and evolve robust and scalable data pipelines, with a focus on large-scale data volumes and the Databricks environment. This professional will be responsible for transforming business requirements into efficient, optimized, and engineering- and data-governance-compliant technical solutions. **Responsibilities and Duties:** * Develop scalable data pipelines using PySpark and SparkSQL. * Optimize queries and processing jobs for large-scale data volumes. * Work with Databricks, including notebooks, clusters, Jobs, and Unity Catalog. * Implement CI/CD processes using GitHub Actions. * Participate in meetings with business areas, translating functional requirements into technical solutions. * Support initiatives for migrating and modernizing legacy data environments. **Requirements and Qualifications:** * Solid experience building data pipelines. * Proficiency in PySpark and SparkSQL, with emphasis on performance. * Hands-on experience with Databricks (Workspace, Jobs, Notebooks, Clusters, Unity Catalog). * Experience with continuous integration and delivery (CI/CD) using GitHub Actions. * Strong communication skills for collaborating with business teams and translating requirements technically. **Preferred Qualifications:** * Technical knowledge of SAS and MongoDB. Benefits Competitive market-based salary; Caju (flexible card) with R$ 1\.060/month loaded; Bradesco Health Insurance; Bradesco Dental Insurance; * ️ Preventive medicine with Dr. Alper; Life Insurance; * ️ Gympass; ️ SESC; Daycare allowance for mothers and fathers; Profit-sharing program (PLR); Learning – an area dedicated to developing hard and soft skills; Partnerships with educational institutions for technical training, MBAs, postgraduate studies, certifications, English, and Spanish; Career development plan; Discounts on products via a partner portal. We emphasize that all our job openings are open to individuals of all profiles and backgrounds, valuing diversity and fostering an inclusive and welcoming environment for everyone.