Senior Data Engineer (1)

Description

Job Summary: We are seeking a Senior Data Engineer to design, implement, and optimize robust data pipelines, working on cloud architectures and ensuring data quality and reliability. Key Highlights: 1. Design and optimize robust, large-scale data pipelines 2. Work with Databricks, GCP, and integration of diverse data sources 3. Ensure data quality and reliability, and automate processes Description: We seek professionals with: * Completed undergraduate degree; * Experience with platforms and languages: o Databricks (Python, PySpark, SQL, Bundles) o GCP (BigQuery, Cloud Functions); * Experience in data modeling and processing: Dimensional modeling and BI-oriented modeling o Large-scale data transformations, creation and maintenance of data pipelines (batch and streaming); * Integration and version control: o Code and pipeline version control (Git) o Continuous integration and DevOps best practices applied to data. Preferred Qualifications: * Experience with pipeline orchestration (Airflow, Dataform, or similar); * Knowledge of event-driven architecture (Pub/Sub, Kafka); familiarity with DataOps practices and data observability; * Experience in large-scale or high-criticality corporate environments; * Experience in project management and stakeholder engagement with external and internal clients; * Knowledge of agile methodologies. Key Challenges for a SENIOR DATA ENGINEER: * Develop and Scale Data Pipelines: Design, implement, and optimize robust data pipelines, ensuring reliability, governance, and performance at scale; * Work with Cloud Architectures and Advanced Platforms: Work with Databricks and GCP (BigQuery, Cloud Functions), integrating diverse data sources and formats into a unified analytics ecosystem; * Support BI and Analytics Teams: Build well-modeled data structures for business analytics, dashboards, and executive reporting; * Automate Processes and Drive Efficiency: Apply data engineering best practices and versioning using Databricks Bundles to deliver reproducible and scalable solutions; * Ensure Data Quality and Reliability: Implement tests, validations, and continuous monitoring to maintain data integrity throughout the data lifecycle. 2511290202181894468