Senior Data Engineer

Description

Job Summary: We are seeking a Senior Data Engineer to design robust architectures, optimize Apache Spark workloads, and lead DevOps automation in Databricks and SAP environments. Key Highlights: 1. Participation in mission-critical strategic data projects 2. Leadership in DevOps automation with a focus on Databricks and SAP 3. Development of scalable architectures and pipeline optimization We are looking for a Senior Data Engineer to work on strategic data projects in mission-critical environments. This professional will be responsible for designing robust architectures, optimizing Apache Spark workloads, and leading DevOps automation initiatives using command-line tools and SDKs, with a focus on Databricks and SAP environments.\* Design and implement scalable and resilient data architectures in Databricks and Azure environments. * Optimize data pipelines in Apache Spark, ensuring performance and resource efficiency. * Automate data engineering processes using CLI and SDKs in DevOps environments. * Support and evolve highly available, mission-critical production environments. * Integrate and orchestrate data from SAP systems and other corporate sources. * Ensure data governance, quality, and traceability across the entire data lifecycle. * Collaborate with cross-functional teams to deliver data solutions aligned with business needs. * Contribute to engineering best practices, code reviews, and DevOps culture.\* Databricks certification. * Solid experience with Apache Spark and large-scale job optimization. * Proficiency in Databricks (Delta Lake, Workflows, Unity Catalog) and Azure Data Services. * Experience automating pipelines and infrastructure via CLI and SDKs (Databricks CLI, Terraform, Azure CLI). * Hands-on experience in production environments requiring high availability and fault tolerance. * Knowledge of data integration with SAP systems (SAP BW, SAP HANA, SAP Data Services). * Experience with version control (Git), CI/CD, and containers (Docker). * Ability to work autonomously, lead technical initiatives, and collaborate in agile teams. Refer to Job description section for qualification required