




Description: What will be required for our perfect match: * Bachelor's degree in IT or in any field with a postgraduate degree in IT; * Minimum 4 years of experience in Software Development, Data Engineering and/or Data Administration; * Practical experience with Databricks (clusters, notebooks, Delta Lake, DLT); * Knowledge of Big Data and distributed processing; * Solid knowledge of PySpark; * Proficiency in SQL; * Experience with version control tools (Git); * Logical and physical modeling in a Lakehouse environment; * Documentation and maintenance of metadata; * Experience ingesting data from RDBMS, APIs, and Cloud Storage; * Application of security rules and governance policies; * Implementation of Data Quality Gates; * Access management in the Databricks Workspace. On a day-to-day basis, we expect you to handle the following processes: * Build, test, and maintain ingestion and transformation pipelines (ETL/ELT); * Use Databricks with a focus on PySpark and SQL; * Ensure scalability and robustness for the BI team; * Optimize pipeline execution; * Perform query tuning; * Adjust Delta clusters and tables to maximize performance and reduce costs; * Apply security and quality policies to pipelines; * Implement governance from Bronze to Silver to Gold; * Ensure quality and traceability of transformations; * Create connections and ingest data from diverse systems; * Ensure integrations are reliable and performant; * Implement automated quality rules; * Develop gates in Databricks / Delta Live Tables (DLT); * Ensure only qualified data reaches the consumption layer; * Model structures in the Lakehouse; * Document and update metadata repositories and models; * Support administration of the Databricks workspace; * Manage access and apply information security best practices. 2512290202491476014


