Senior Data Engineer

Description

Job Summary: We are seeking a Senior Data Engineer to design robust data architectures, optimize Spark workloads, and lead DevOps automation in Databricks and SAP environments, with a focus on mission-critical systems. Key Highlights: 1. Participation in strategic, mission-critical data projects 2. Leadership in DevOps automation and Spark optimization 3. Development of scalable architectures on Databricks and Azure We are looking for a Senior Data Engineer to work on strategic data projects in mission-critical environments. This professional will be responsible for designing robust data architectures, optimizing Apache Spark workloads, and leading DevOps automation initiatives using command-line tools and SDKs, with a focus on Databricks and SAP environments.\* Design and implement scalable and resilient data architectures on Databricks and Azure. * Optimize data pipelines in Apache Spark, ensuring performance and resource efficiency. * Automate data engineering processes using CLI and SDKs in DevOps environments. * Support and evolve production environments requiring high availability and criticality. * Integrate and orchestrate data from SAP systems and other corporate sources. * Ensure data governance, quality, and traceability across the entire data lifecycle. * Collaborate with cross-functional teams to deliver data solutions aligned with business needs. * Contribute to engineering best practices, code reviews, and DevOps culture.\* Databricks certification. * Strong experience with Apache Spark and large-scale job optimization. * Proficiency in Databricks (Delta Lake, Workflows, Unity Catalog) and Azure Data Services. * Experience automating pipelines and infrastructure via CLI and SDKs (Databricks CLI, Terraform, Azure CLI). * Hands-on experience in production environments with high-availability and fault-tolerance requirements. * Knowledge of data integration with SAP systems (SAP BW, SAP HANA, SAP Data Services). * Experience with version control (Git), CI/CD, and containers (Docker). * Ability to work autonomously, lead technical initiatives, and collaborate within agile teams. Refer to Job description section for qualification required