Senior Data Engineering Specialist

Description

The first thing you need to know is that here, you won’t fall into routine. Radix develops solutions for companies across diverse sectors and industries. Each project involves its own technologies, solutions, and deadlines — giving you the opportunity to work on varied engineering, data, analytics, and digital transformation challenges. Beyond our presence across Brazil — with offices in Rio de Janeiro, São Paulo, and Belo Horizonte — we also have subsidiaries in the United States, further consolidating Radix as a global company. Do you want to be part of this story and turn ideas and dreams into reality? **About the role** ---------------- We are seeking a Data Engineering Specialist with strong hands-on experience in Databricks to design, implement, optimize, and maintain modern data solutions built on the Lakehouse architecture. This person will be responsible for building scalable, reliable, and high-performance data pipelines using the Databricks platform — with emphasis on ingestion, transformation, data quality, governance, modeling, and data delivery for analytical consumption, BI, enterprise integrations, and advanced data initiatives. The ideal candidate has solid experience with Databricks, Apache Spark, PySpark, SQL, Delta Lake, Unity Catalog, Workflows, Delta Live Tables, data modeling, API-based integrations, and modern Data Quality practices — along with the ability to collaborate closely with technical teams and business units. **Key responsibilities:** --------------------------------- * Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, PySpark, SQL, and Delta Lake. * Implement layered data architectures (Bronze, Silver, Gold), ensuring data organization, traceability, quality, and governance. * Develop ingestion, transformation, validation, and data delivery processes using native Databricks capabilities — including notebooks, jobs, workflows, Delta Live Tables, Auto Loader, and Databricks SQL. * Design and implement Data Quality frameworks across ingestion, transformation, and consumption layers. * Develop automated data quality rules, validations, and monitoring processes to ensure accuracy, completeness, consistency, uniqueness, timeliness, and integrity of data. * Implement data profiling, cleansing, standardization, enrichment, reconciliation, and exception handling processes. * Define, implement, and monitor data quality metrics, KPIs, scorecards, and dashboards. * Build automated validation processes for structured and semi-structured data — including JSON data from REST APIs, SaaS platforms, and enterprise systems. * Design canonical data models and analytical models that consolidate information from multiple source systems. * Create and maintain source-to-target mappings, transformation logic, business rules, and data harmonization processes. * Design dimensional models — including star schemas — to support analytical workloads, reporting, and BI solutions. * Implement incremental loading strategies, CDC, data reconciliation, and end-to-end validation. * Work with Unity Catalog — covering governance, lineage, metadata management, catalogs, schemas, permissions, and role-based security. * Optimize Databricks workloads — including notebooks, jobs, SQL queries, clusters, compute resource usage, partitioning, Z\-Ordering, Spark tuning, and performance best practices. * Investigate, diagnose, and resolve data quality, performance, availability, and integrity issues — identifying root causes and implementing preventive solutions. * Support architectural decisions related to Databricks, Delta Lake, Lakehouse, governance, integration, automation, and observability. * Collaborate with data architects, data engineers, analysts, software engineers, and business stakeholders to deliver reliable, scalable, and high-quality solutions. * Support modern engineering practices — including Git version control, CI/CD, automated deployment, code reviews, and technical documentation. **Required qualifications:** ------------------------------- * Fluent English for conversation. * Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field — or equivalent practical experience. * Extensive experience as a Data Engineer, Data Specialist, Data Architect, or similar role. * Hands-on, advanced experience with Databricks in enterprise data engineering projects. * Solid knowledge of Apache Spark, PySpark, and Spark SQL. * Advanced SQL expertise — including complex queries, transformations, optimization, and performance tuning. * Experience with Delta Lake — including Delta tables, versioning, optimization, partitioning, and storage best practices. * Experience designing and implementing Raw/Bronze, Curated/Silver, and Business/Gold layers. * Experience orchestrating pipelines and workloads in Databricks — including jobs, workflows, and automation. * Experience implementing Data Quality frameworks, methodologies, and best practices in enterprise environments. * Experience developing and implementing data quality rules, validation frameworks, reconciliation processes, and exception handling. * Strong understanding of data profiling, cleansing, standardization, enrichment, and record matching. * Practical experience integrating and validating JSON data from REST APIs, SaaS platforms, and enterprise systems. * Solid knowledge of relational database modeling, normalization, and data integrity principles. * Experience designing canonical data models that consolidate information from multiple source systems. * Experience with source-to-target data mapping, transformations, harmonization, and business rule documentation. * Strong knowledge of dimensional modeling, star schemas, and enterprise Data Warehouse concepts. * Experience implementing incremental loads, CDC, reconciliation processes, and automated data quality validations. * Familiarity with data governance, metadata management, data lineage, master data, reference data, and role-based security. * Experience with Unity Catalog or equivalent concepts for governance, cataloging, access control, and data lineage. * Proficiency in Git, CI/CD practices, and deployment automation for data engineering solutions. * Ability to work autonomously, think critically, communicate clearly, and collaborate effectively with technical and business teams. **Nice-to-have qualifications:** ----------------------------------- * Databricks certification or accreditation related to architecture — such as Databricks Platform Architect Accreditation/Badge — preferably on Azure or AWS. * Databricks Certified Data Engineer Professional or equivalent certification. * Experience with Lakehouse architecture in large-scale enterprise environments. * Experience with Delta Live Tables, Auto Loader, Databricks Asset Bundles, Databricks CLI, or Databricks deployment automation. * Experience with cloud platforms — especially Azure, AWS, or GCP. * Experience with observability, monitoring, and alerting tools for data pipelines. * Experience with Databricks SQL, dashboards, SQL Warehouses, and BI tool integrations. * Knowledge of FinOps applied to Databricks — including cluster optimization, serverless, jobs, and workloads. * Master’s or Ph.D. in Data Science, Artificial Intelligence, Engineering, Computer Science, or disciplines requiring high analytical capability. **Benefits:** * National Health Insurance (for employee and dependents, private room included) * National Dental Insurance (for employee and dependents) * Flexible Meal/Meal Allowance * Remote Work Allowance * Birthday Day Off * Wellhub (formerly Gympass) * Extended Maternity Leave (6 months) and Paternity Leave (20 days) * Daycare Assistance for children up to 3 years old (per child) * Mental health support via Wellz * Benefits Club offering discounts with various partners * Education partnership with academic institutions and language courses * Professional Development (Corporate University) * Partnership with coworking spaces across Brazil * Quality of Life and Well\-Being Program * Consulting physician for Radix employees * Incentive Plans **Radix consistently ranks among the Best Companies to Work For because:** * We have committed, dedicated, curious, and innovative professionals. * Team spirit is our greatest strength. We cooperate and know we’re all rowing in the same direction. * We foster a diverse environment that values equity and inclusion. * Our working hours are flexible, and for nearly all projects, remote work from anywhere in Brazil is possible. * We value well\-being and care for our people — offering mental health support programs, access to psychiatrists, and consulting physicians. **Interested?** **\#JoinRadix** **Code: bb82