Senior Data Engineer

Description

Job Summary: We are seeking a Senior Data Engineer to build and manage robust Big Data solutions on the Cloudera platform, integrating with AI and Machine Learning teams. Key Highlights: 1. Implement and manage Big Data architectures on the Cloudera platform. 2. Configure data pipelines using Apache Spark, Livy, and NiFi. 3. Ensure data governance through Apache Ranger and Cloudera Navigator. Description: We are looking for a Senior Data Engineer specializing in Cloudera. If you want to become our expert building and managing robust Big Data solutions, this position is for you! Responsibilities and Duties * Implement and manage Big Data architectures on the Cloudera platform. * Configure data pipelines using Apache Spark, Livy, and NiFi. * Ensure data governance through Apache Ranger and Cloudera Navigator. * Structure and organize Data Lakes in layers to optimize data processing and consumption. * Define, create, and maintain a centralized data catalog, ensuring comprehensive documentation of data assets, traceability, and compliance with governance policies; Collaborate with Data Science and Engineering teams for integration of AI and Machine Learning solutions. * Requirements and Qualifications * Advanced experience with Cloudera Data Platform (CDP), Cloudera Manager, and Cloudera Navigator in cloud and on\-premises environments. * Proficiency in building layered Data Lakes (Bronze, Silver, and Gold). * Experience with Python and Apache Spark for large-scale data processing. * Knowledge of Apache Ranger for permission management and governance. * Use of Hive and HUE for querying and interacting with data. * Experience with Apache NiFi for data flow automation and integration. * Knowledge of Livy for managing and submitting Spark jobs in distributed environments. * Experience delivering technical training and presentations. * Business and data analysis skills to facilitate understanding of customer needs and definition of technical solutions. 2511230202461339760