Databricks Data Engineer | Senior (Remote)

Description

. **Responsibilities and Assignments** * Implement and evolve the Enterprise Data Platform (Enterprise Lakehouse); * Participate in modernizing the corporate analytics ecosystem, including migrating workloads from Azure to AWS and Databricks; * Develop, maintain, and enhance scalable, reliable, and high-performance data pipelines; * Build data ingestion, transformation, and delivery solutions using Lakehouse architecture; * Implement Data Engineering standards, DataOps, CI/CD, and code versioning; * Support data governance, quality, and cataloging initiatives; * Refactor legacy pipelines into modern solutions based on Apache Spark and Databricks; * Work on complex integrations between corporate systems, ensuring data scalability, governance, and reliability; * Collaborate on implementing Data Lake, Data Warehouse, and Lakehouse architectures in cloud environments. **Requirements and Qualifications** * Advanced SQL proficiency; * Experience building ETL/ELT pipelines; * Practical experience with Databricks; * Knowledge of Apache Spark; * Experience with Data Lake, Data Warehouse, and/or Lakehouse architectures; * Knowledge of analytical and dimensional data modeling; * Experience processing large volumes of data; * Hands-on experience with cloud environments, preferably AWS; * Familiarity with AWS Glue, Unity Catalog, and Lake Formation; * Experience with Git and code versioning; * Knowledge of CI/CD practices applied to Data Engineering; * Experience with tools such as Airflow, Kafka, and dbt; * Knowledge of SQL and NoSQL databases (PostgreSQL, MongoDB, and Cassandra); * Experience in corporate data platform modernization and migration projects; * Ability to handle complex integrations between corporate systems and distributed environments. Compass UOL is a global firm and part of the AI Revolution Company, together transforming organizations using Artificial Intelligence, Generative AI, and other of today’s most advanced technologies. We equip our team with proprietary and external AI\-driven tools to design and build digital\-native platforms, integrating cutting\-edge technologies and enabling companies to innovate, transform their businesses, and drive success in their markets. To achieve this, we attract and develop the best talent, creating opportunities that enhance people’s lives and highlight the positive impact of disruptive technologies. We empower borderless talent and promote knowledge and opportunities in the latest market trends, driving significant personal and professional growth. **Join us and be part of the AI\-driven revolution.**