Data Architect - Databricks (Azure)

Description

Job Summary: We are seeking a Data Architect with strong production experience in Databricks to work on a strategic project focused on data architecture, scalability, performance, and governance within an Azure environment, serving as both a technical reference and hands-on contributor. Key Highlights: 1. Strategic and hands-on engagement with Databricks and Azure 2. Focus on data architecture, scalability, performance, and governance 3. Technical reference for the team and client We are looking for a Data Architect with strong production experience in Databricks to work on a strategic project focused on data architecture, scalability, performance, and governance within an Azure environment. We seek a candidate with a more strategic mindset, yet also hands\-on, capable of acting as a technical reference for both the client and the team. Responsibilities Define and evolve the data architecture in Databricks Design and implement scalable and high-performance data pipelines Ensure engineering best practices, governance, security, and observability Optimize pipelines, clusters, and cloud costs Support production troubleshooting and guide technical improvements Serve as a technical reference for the team and the client **Requirements:** **Solid production experience with Databricks:** Workflows, Unity Catalog, Delta Lake, and Asset Bundles Advanced proficiency in PySpark and Spark Experience with Medallion architecture and dimensional modeling Strong expertise in pipeline optimization, performance tuning, and data volume management Experience with Databricks clusters and cost vs. performance tuning **Azure experience:** Key Vault, ADLS Gen2, Service Principals, and permissions Experience with CI/CD for data, versioning, and deployment Knowledge of Data Quality, Data Contracts, logging, and observability Experience with secure integrations and multiple data sources Production troubleshooting capability Advanced SQL **Nice-to-have:** Experience with FinOps Knowledge of security and governance in Unity Catalog Experience with data engineering frameworks and automated testing Databricks and/or Azure certifications