Senior Data Engineer - PCD

Description

Job Summary: This role involves developing and maintaining data pipelines, contributing to Lakehouse architecture, and supporting data automation and governance with a focus on Databricks and Delta Lake. Key Highlights: 1. Working on projects with Databricks, Delta Lake, and multiple data sources 2. Developing and maintaining data pipelines using PySpark and Spark SQL 3. Actively participating in the implementation of Lakehouse architecture Posting Description: **Senior Data Engineer (PCD)** *Are you interested in working on projects involving Databricks, Delta Lake, and integration of multiple data sources? If you answered “YES!”, this could be the next step in your career as a Senior Data Engineer (PCD).* *This position requires on-site work at our office located in São Paulo/SP.* **Aon is in the business of better decisions** At Aon, we shape decisions to better protect and enrich people’s lives worldwide. As an organization, we are united by trust as an inclusive and diverse team, and we are passionate about helping our colleagues and clients succeed. **Your typical day will include:** *Developing and maintaining data ingestion and transformation pipelines under supervision.* *Contributing to the implementation of Lakehouse architecture in internal projects.* *Assisting in automating tests and DevOps-integrated deployment processes.* *Supporting data catalog management and version control.* *Monitoring Spark jobs and proposing performance improvements under guidance.* **What makes this opportunity different** *In this role, you can apply your full expertise in data and cloud environments (AWS or Azure), building and maintaining data pipelines and platforms. You’ll actively participate in projects involving Databricks, Delta Lake, and integration of multiple data sources. Committed to best practices in governance, accessibility, and collaboration within diverse teams.* **Skills and experience that will lead to success** *Completed or ongoing academic degree in Information Technology.* *Databricks & Spark* *Developing and maintaining data pipelines using PySpark and Spark SQL on Databricks.* *Implementing projects with Delta Lake on Amazon S3.* *Applying basic Spark job optimization practices (partitioning, caching).* *Using Databricks Workflows for task orchestration.* *AWS (Amazon Web Services)* *Data manipulation in Amazon S3:* *Basic knowledge of IAM and data security in buckets.* *Data Architecture and Platform* *Supporting implementation of Lakehouse architecture (bronze, silver, gold):* *Integrating data from relational databases, APIs, and batch files.* *Basic modeling and versioning with Delta Lake.* *DataOps & Governance* *Collaborating in CI/CD processes using Databricks Repos or Azure DevOps.* *Basic pipeline monitoring using logs and alerts.* *Supporting data quality controls (Great Expectations or similar solutions).* *Assisting in permission management and data organization via Unity Catalog.* *Languages and Tools* *Practical proficiency in Python and SQL.* *Experience with data formats such as JSON and Parquet.* *Using collaborative notebooks in Databricks.* *Knowledge of streaming data (Structured Streaming or Kafka).* *Introductory knowledge of Snowflake.* *Basic understanding of Infrastructure as Code (Terraform).* *Interest in learning about Data Mesh and Data Contracts.* *Intermediate English is considered a plus.* **How we support our colleagues** Beyond our comprehensive benefits package, we foster an inclusive team and workplace atmosphere. Moreover, our agile environment enables you to manage your well-being and achieve a healthy work-life balance, ensuring you can perform at your best at Aon. Additionally, all colleagues enjoy two “Global Well-Being Days” per year, encouraging them to set aside time to focus on themselves. We also offer a variety of flexible work arrangements for our colleagues. Our culture of continuous learning inspires and prepares you to learn, share, and grow—helping you reach your full potential. As a result, at Aon you become more connected, more relevant, and more valued. Aon values an innovative and inclusive workplace where all colleagues feel empowered to be authentic. Aon proudly stands as an equal-opportunity workplace for everyone. Aon provides equal employment opportunities to all employees and job applicants regardless of race, color, religion, creed, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, marital status, domestic partner status, or any other legally protected status. We welcome applications from all individuals and provide accessibility tools for people with disabilities to participate in recruitment processes—including application, interviews—and to perform essential job functions upon hiring. If you would like to learn more about our accessibility policies, please email ReasonableAccommodations@Aon.com **#LI-V1**