Senior Data Engineer

Description

Job Summary: We are seeking a Senior Data Engineer to build and manage robust Big Data solutions on the Cloudera platform, integrating data and collaborating with AI teams. Key Highlights: 1. Implement and manage Big Data architectures on the Cloudera platform 2. Ensure data governance via Apache Ranger and Cloudera Navigator 3. Collaborate with Data Science and Engineering teams on AI and ML solutions Description: We are looking for a Senior Data Engineer specializing in Cloudera. If you want to become our expert building and managing robust Big Data solutions, this position is for you! Responsibilities and Duties * Implement and manage Big Data architectures on the Cloudera platform. * Configure data pipelines using Apache Spark, Livy, and NiFi. * Ensure data governance through Apache Ranger and Cloudera Navigator. * Structure and organize Data Lakes in layered architecture to optimize data processing and consumption. * Define, create, and maintain a centralized data catalog, ensuring comprehensive documentation of data assets, traceability, and compliance with governance policies; Collaborate with Data Science and Engineering teams to integrate AI and Machine Learning solutions. * Requirements and Qualifications * Advanced experience with Cloudera Data Platform (CDP), Cloudera Manager, and Cloudera Navigator in cloud and on\-premises environments. * Proficiency in building layered Data Lakes (Bronze, Silver, and Gold). * Experience with Python and Apache Spark for large-scale data processing. * Knowledge of Apache Ranger for permission management and governance. * Use of Hive and HUE for querying and interacting with data. * Experience with Apache NiFi for automating and integrating data flows. * Knowledge of Livy for managing and submitting Spark jobs in distributed environments. * Experience delivering technical training and presentations. * Business and data analysis skills to facilitate understanding client needs and defining technical solutions. 2511230202461339760