Manager | Data Engineering

Description

Job Summary: Role as a Data Engineer and Architect, focusing on data modeling, pipeline construction, data architecture, and technical leadership in Big Data projects and agile methodologies. Key Highlights: 1. Experience in Data Engineering and Data Architecture and Big Data 2. Technical leadership in data projects and engineering teams 3. Work with DevOps, CI/CD, Data Governance, and Data Security **Description:** Work in Data Engineering and Architecture, data modeling, and building ingestion and transformation pipelines. Component-oriented architecture with reusable components and data products. Big Data Platforms and Technologies Apache Spark, Apache Flink, Kafka, Iceberg, Kudu, Hive. DevOps and Code Productization Version control with Git. Continuous integration and deployment using DevOps pipelines (CI/CD). Automated testing and pipeline monitoring. Data Governance and Security Access control and policies using Apache Ranger. Knowledge of LGPD/GDPR and best practices for data protection. Cloud and Infrastructure Experience with cloud-based environments (AWS, Azure, GCP). Resource management and scalability of solutions. Orchestration and Monitoring Tools Airflow or similar tools. Monitoring of jobs and performance of distributed systems. Machine Learning Knowledge Integration of predictive models into data pipelines. Supporting business squads in operationalizing models. Technical Leadership in Data Projects Experience managing data engineering teams, with focus on delivering reusable and scalable components. Working in environments with multidisciplinary squads and agile methodologies. Building Pipelines and Data Architecture Hands-on experience in modeling, ingestion, transformation, and large-scale data delivery. Knowledge of data product-oriented architectures and analytical platforms. Proficiency in Big Data Technologies Practical experience with tools such as Apache Spark, Flink, Kafka, Iceberg, Kudu, Hive. Equivalent platforms for job orchestration. Data Governance and Security Implementation of access control and data masking policies using tools like Apache Ranger. Ensuring compliance with data protection regulations (LGPD, GDPR). Code Productization with DevOps Experience with Git version control, continuous integration (CI/CD), and DevOps pipelines for deploying solutions to production. Application of automated testing and component monitoring. Focus on Operational Efficiency Track record of delivering solutions that reduce rework, increase business team autonomy, and improve data process performance. 2511210202471882091