Senior Data Engineer - PCD

Description

Job Summary: Develop and maintain data pipelines, contributing to Lakehouse architectures and process automation, with a focus on Databricks and multiple data sources. Key Highlights: 1. Working on projects with Databricks, Delta Lake, and data integration 2. Building and maintaining data pipelines and data platforms 3. Collaborating within a diverse team and implementing governance practices Posting Description: **Senior Data Engineer (PCD)** *Are you interested in working on projects with Databricks, Delta Lake, and integrating multiple data sources? If you answered “YES!”, this could be your next career step as a Senior Data Engineer (PCD).* *This role requires on-site work at our office located in São Paulo/SP.* **Aon is in the business of better decisions** At Aon, we shape decisions to better protect and enrich people’s lives worldwide. As an organization, we are united by trust, as an inclusive and diverse team, and are passionate about helping our colleagues and clients succeed. **Your typical day will include:** *Developing and maintaining data ingestion and transformation pipelines under supervision.* *Contributing to the implementation of Lakehouse architecture in internal projects.* *Assisting in automating tests and DevOps-integrated deployment processes.* *Supporting data catalog management and version control.* *Monitoring Spark jobs and proposing performance improvements under guidance.* **What makes this opportunity different** *In this role, you can apply your full expertise in data and cloud environments (AWS or Azure), building and maintaining data pipelines and platforms. You’ll actively participate in projects involving Databricks, Delta Lake, and integration of multiple data sources. You’ll be committed to best practices in governance, accessibility, and collaboration within diverse teams.* **Skills and experience that will lead to success** *Completed or ongoing academic degree in Information Technology.* *Databricks \& Spark* *Developing and maintaining data pipelines using PySpark and Spark SQL on Databricks.* *Implementing projects with Delta Lake on Amazon S3\.* *Applying basic Spark job optimization practices (partitioning, caching).* *Using Databricks Workflows for task orchestration.* *AWS (Amazon Web Services)* *Data manipulation in Amazon S3:* *Basic understanding of IAM and data security in buckets.* *Data Architecture and Platform* *Supporting implementation of Lakehouse architecture (bronze, silver, gold):* *Integrating data from relational databases, APIs, and batch files.* *Basic modeling and versioning with Delta Lake.* *DataOps \& Governance* *Collaborating in CI/CD processes using Databricks Repos or Azure DevOps.* *Basic pipeline monitoring using logs and alerts.* *Supporting data quality controls (Great Expectations or similar solutions).* *Assisting in permission management and data organization via Unity Catalog.* *Languages and Tools* *Practical proficiency in Python and SQL.* *Experience with data formats such as JSON and Parquet.* *Using collaborative notebooks in Databricks.* *Streaming data knowledge (Structured Streaming or Kafka).* *Introductory knowledge of Snowflake.* *Basic understanding of Infrastructure as Code (Terraform).* *Interest in learning about Data Mesh and Data Contracts.* *Intermediate English is considered a plus.* **How we support our colleagues** Beyond our comprehensive benefits package, we foster an inclusive team and workplace atmosphere. Moreover, our agile environment empowers you to manage your well-being and work-life balance, ensuring you can perform at your best at Aon. Additionally, all colleagues enjoy two “Global Well-Being Days” per year, encouraging them to reserve time to focus on themselves. We also offer a variety of flexible work arrangements for our colleagues. Our culture of continuous learning inspires and prepares you to learn, share, and grow—helping you reach your full potential. As a result, at Aon you become more connected, more relevant, and more valued. Aon values an innovative and inclusive workplace where all colleagues feel empowered to be authentic. Aon proudly champions equal opportunity for all. Aon provides equal employment opportunities to all employees and job applicants, regardless of race, color, religion, creed, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, marital status, domestic partner status, or any other legally protected status. We welcome applications from everyone and provide accessibility tools to individuals with disabilities throughout the recruitment process—including application, interviews—and for performing essential job functions once hired. If you would like to learn more about our accessibility policies, please email ReasonableAccommodations@Aon.com **\#LI\-V1**