Manager | Data Engineering

Description

Job Summary: Work in Data Engineering and Architecture, modeling, building pipelines, and managing multidisciplinary teams focused on scalable solutions. Key Highlights: 1. Experience with Data Architecture and Big Data (Spark, Flink, Kafka) 2. Technical leadership in data projects and team management 3. Proficiency in DevOps, Data Governance, and Cloud (AWS, Azure, GCP) **Description:** Work in Data Engineering and Architecture, data modeling, building ingestion and transformation pipelines. Component-oriented architecture with reusable components and data products. Big Data Platforms and Technologies Apache Spark, Apache Flink, Kafka, Iceberg, Kudu, Hive. DevOps and Code Productization Version control with Git. Continuous integration and deployment via DevOps pipelines (CI/CD). Automated testing and pipeline monitoring. Data Governance and Security Access control and policies using Apache Ranger. Knowledge of LGPD/GDPR and best practices for data protection. Cloud and Infrastructure Experience with cloud-based environments (AWS, Azure, GCP). Resource management and solution scalability. Orchestration and Monitoring Tools Airflow or similar tools. Monitoring of jobs and performance of distributed systems. Machine Learning Knowledge Integration of predictive models into data pipelines. Supporting business squads in operationalizing models. Technical Leadership in Data Projects Experience managing data engineering teams, focusing on delivering reusable and scalable components. Working in environments with multidisciplinary squads and agile methodologies. Pipeline and Data Architecture Development Hands-on experience in modeling, ingestion, transformation, and large-scale data delivery. Knowledge of data product–oriented architectures and analytical platforms. Proficiency in Big Data Technologies Practical experience with tools such as Apache Spark, Flink, Kafka, Iceberg, Kudu, Hive. Equivalent platforms for job orchestration. Data Governance and Security Implementation of access and masking policies using tools like Apache Ranger. Ensuring compliance with data protection regulations (LGPD, GDPR). Code Productization with DevOps Experience with Git version control, continuous integration (CI/CD), and DevOps pipelines for deploying solutions to production. Application of automated testing and component monitoring. Focus on Operational Efficiency Track record of delivering solutions that reduce rework, increase business team autonomy, and improve data process performance. 2511210202471882091