Description
***Temporary Project \- 160 project hours. Freelancer in RPA (payment receipt for self-employed individuals).***
We are seeking a **Senior Data Engineer**, with a **hands-on** profile, to work on the development and evolution of modern data and Artificial Intelligence solutions, using **Databricks** as the primary platform. We seek a professional with strong technical expertise to design, develop, and optimize data pipelines, as well as build generative AI solutions—with emphasis on **Databricks Genie**.
### **Key Responsibilities**
* Develop and maintain scalable data pipelines using Databricks, Spark, and PySpark.
* Design ETL/ELT solutions for high-volume environments.
* Develop notebooks and workflows in Databricks, focusing on performance, quality, and governance.
* Implement modern data architectures using Unity Catalog, Delta Live Tables, and other Databricks platform features.
* Build and enhance generative AI-based solutions using **Databricks Genie**, Mosaic AI, and integration with LLMs.
* Develop APIs and automations in Python for integration between platforms and analytical solutions.
* Perform data modeling, cataloging, and governance.
* Participate in defining architecture, technical standards, and best practices for Data Engineering.
* Collaborate with architects, data scientists, and business units to build analytical solutions.
### **Requirements**
* Solid experience as a Data Engineer.
* **Hands-on** profile, directly involved in solution development.
* Advanced experience in:
+ Databricks;
+ Apache Spark / PySpark;
+ Python;
+ SQL;
+ ETL/ELT;
+ Data Lake and Data Lakehouse.
* Experience with cloud environments (AWS, Azure, GCP, or OCI).
* Experience with code versioning, CI/CD, and DevOps.
* Knowledge of data modeling, governance, and performance optimization.
### **Preferred Qualifications**
* **Practical experience with Databricks Genie (Genie Spaces), Mosaic AI, and developing solutions using Generative AI and LLMs.**
* Experience with Unity Catalog and Delta Live Tables.
* Experience developing APIs (FastAPI or similar).
* Experience in Advanced Analytics and BI.
* Databricks certifications.
* **Prior involvement in projects within the energy sector (Utilities/Oil & Gas/Energy Distribution/Gas).**
### **Competencies**
* Analytical mindset and problem-solving orientation.
* Ability to thrive in collaborative environments.
* Proactivity and autonomy.
* Strong communication skills and ability to interact effectively with multidisciplinary teams.
* Focus on quality, performance, and innovation.