Senior Data Engineer

Description

Job Summary: An experienced Big Data professional to lead projects, design distributed environments, develop data pipelines, and ensure data governance and security. Key Highlights: 1. Leadership in architecture projects and Big Data solutions 2. Development of data pipelines and ETL processes 3. Ensuring high availability, security, and performance of clusters Description: * Completed undergraduate degree; * Advanced knowledge of ITIL; * Advanced proficiency in Python; * Advanced knowledge of PySpark; * Experience with Airflow and pipeline automation; * Advanced experience with Cloudera; * Knowledge of Hadoop (administration and optimization); * Advanced experience with ETL (Pentaho Data Integration/Kettle, Jasper ETL, and similar tools); * Hands-on experience with infrastructure-as-code and DevOps practices. * Lead architecture projects and Big Data solutions; * Design, administer, and optimize distributed environments (Hadoop, Cloudera, Spark); * Develop data pipelines and ETL processes; * Ensure high availability, security, and performance of clusters; * Resolve complex infrastructure issues and perform tuning; * Support and train technical teams, documenting best practices; * Collaborate with stakeholders in defining requirements and designing solutions; * Ensure data governance, quality, and security. 251110020218420281