Senior Data Engineer (Databricks | AWS | Regulatory)

Description

Job Summary: INDT is seeking a Data Engineer to implement and maintain regulatory data pipelines, focusing on data quality, traceability, and governance. Key Highlights: 1. Collaborative environment with challenges for learning and growth 2. Opportunity to develop talents and be valued 3. Freedom to propose and develop innovative projects **About Us:** Since 2001, INDT has been a technology institute that inspires innovation and breathes technology! Development is in our DNA, and we offer a collaborative environment filled with challenges for learning and growth. We originated in Manaus (AM) through Nokia and now operate across Brazil, aiming to present our services and Culture to all Brazilian regions. Currently, we have over 200 active collaborators. For us, eliminating geographical and cultural barriers was merely a consequence of our belief that nothing is impossible. Thus, today more than ever, we can look in the mirror and think: we work at a company that lives and grows every day! Do you want the opportunity to develop your talents? Do you want to work in an excellent work environment alongside colleagues who will support you on your journey? Do you want an incredible experience of feeling valued? Do you want to enhance your daily deliverables and projects with inspiring leaders? Do you want freedom to propose and develop innovative projects, with autonomy and ownership? Then join us in making a difference! **\#VEMSERINDT** We are looking for **a Data Engineer** to implement and maintain regulatory data pipelines, with focus on CADOC 6209\. This is a strategic position with significant impact on data quality, traceability, performance, and governance, ensuring compliance with regulatory obligations. Requirements: **Key Responsibilities** * Develop and maintain data pipelines in Spark (Databricks) to comply with CADOC 6209 * Implement transformations and validations using SQL and PySpark, ensuring adherence to regulatory rules * Integrate data from sources within the AWS environment (Lake/Lakehouse) * Implement data quality controls, reconciliations, and audit trails * Ensure code versioning, artifact traceability, and engineering best practices * Contribute to the evolution of the data platform architecture and standards **Technical Requirements** * Solid experience with advanced SQL * Practical experience with Apache Spark / Databricks * Knowledge of AWS (S3, IAM, Glue) * Experience building production-grade data pipelines (ETL/ELT) * Experience in regulatory, financial, or highly critical data environments * Experience with Git and versioning best practices **Nice-to-Haves** * Experience with CADOC 6209 or other Central Bank regulatory requirements * Knowledge of data governance, auditing, and traceability * Experience with data quality frameworks * Experience migrating legacy solutions (e.g., SAS, Alteryx) to Databricks * Knowledge of CI/CD applied to data **Expected Profile** * Organization and attention to detail, especially in regulatory environments * Strong communication skills for interaction with technical and business teams * Critical mindset regarding data quality and reliability * Commitment to deadlines and deliverables