Manager | Data Engineering

Description

Job Summary: Work in Data Engineering and Data Architecture, data modeling, pipeline construction, and technical leadership in data projects, focusing on reusable and scalable components. Key Highlights: 1. Expertise in Data Engineering and Data Architecture 2. Technical leadership in data projects 3. Proficiency in Big Data technologies **Description:** Work in Data Engineering and Architecture, data modeling, and construction of ingestion and transformation pipelines. Component-oriented architecture emphasizing reusability and data products. Big Data Platforms and Technologies Apache Spark, Apache Flink, Kafka, Iceberg, Kudu, Hive. DevOps and Code Productization Version control with Git. Continuous integration and deployment using DevOps pipelines (CI/CD). Automated testing and pipeline monitoring. Data Governance and Security Access control and policies using Apache Ranger. Knowledge of LGPD/GDPR and best practices for data protection. Cloud and Infrastructure Experience with cloud-based environments (AWS, Azure, GCP). Resource management and scalability of solutions. Orchestration and Monitoring Tools Airflow or similar tools. Monitoring of jobs and performance of distributed systems. Machine Learning Knowledge Integration of predictive models into data pipelines. Supporting business squads in operationalizing models. Technical leadership in data projects Experience managing data engineering teams, with a focus on delivering reusable and scalable components. Working in multidisciplinary squads and agile methodologies. Pipeline and Data Architecture Development Hands-on experience in modeling, ingestion, transformation, and large-scale data delivery. Knowledge of data product-oriented architectures and analytical platforms. Proficiency in Big Data technologies Practical experience with tools such as Apache Spark, Flink, Kafka, Iceberg, Kudu, Hive. Equivalent platforms for job orchestration. Data Governance and Security Implementation of access control and data masking policies using tools like Apache Ranger. Ensuring compliance with data protection regulations (LGPD, GDPR). Code Productization with DevOps Experience with Git version control, continuous integration (CI/CD), and DevOps pipelines for deploying solutions to production. Application of automated testing and component monitoring. Focus on Operational Efficiency Track record of delivering solutions that reduce rework, increase business team autonomy, and improve data process performance. 2512230202491882091