Data Engineer | Azure & Databricks

Description

Job Summary: We are seeking a Data Engineer to build and optimize PySpark/Databricks pipelines on Azure, with a focus on data performance, quality, and security. Key Highlights: 1. Building and optimizing PySpark/Databricks pipelines on Azure 2. Data ingestion, transformation, and governance projects 3. Focus on data performance, quality, and security We are seeking a Data Engineer to build and optimize PySpark/Databricks pipelines using Azure services. The role involves data ingestion, transformation, and governance projects, with emphasis on performance, quality, and security of data. Hybrid work location.\* Develop and maintain data pipelines in PySpark/Databricks * Write and optimize SQL for large-scale datasets * Work on ingestion, transformation, and governance projects in Azure Data Lake and Data Factory * Ensure data performance, quality, and security * Collaborate with business and analytics teams to deliver reliable datasets\* Practical experience with SQL, PySpark, Databricks, and Azure * Knowledge of ADF, Data Lake, and data modeling best practices * Experience with code versioning (Git) and agile methodologies * Preferred qualifications: Delta Lake, Unity Catalog, CI/CD, and data architecture (Lakehouse, Data Mesh) Refer to Job description section for qualification required