Job Summary: Join us in building a modern data foundation by migrating and modernizing a massive legacy system to a new architecture on Google Cloud Platform (GCP). Key Highlights: 1. Building a modern data foundation on GCP 2. Working with cutting-edge technologies on GCP and Data Mesh 3. A technically challenging environment with high-volume data We are specialists in **technological transformation**, combining human expertise with AI to build scalable tech solutions. With over 8,000 CI&Ters worldwide, we have partnered with more than 1,000 clients throughout our 30-year history. Artificial Intelligence is our reality. **Important**: If you reside in the Campinas Metropolitan Region, your physical presence at our city offices is mandatory, per our current attendance policy. About the Challenge: We are at a critical stage in evolving our data platform. The project involves migrating and modernizing a massive legacy system (based on Azure/Databricks) to a new architecture on Google Cloud Platform (GCP). You will help build a modern data foundation, applying Data Mesh principles, the medallion architecture (Raw/Silver/Gold), and strong governance—ensuring deactivation of the legacy system and enabling new AI and Analytics capabilities. Key Responsibilities: * Migration Execution (Refactoring & Modernization): Analyze and migrate legacy notebooks and pipelines (Spark/Databricks). This includes both refactoring logic to align with the new architecture and fully rewriting processes in SQL/Dataform or Dataflow. * ELT/ETL Pipeline Development: Develop and maintain data transformations using BigQuery and Dataform (SQL) to build Trusted/Silver and Gold layers, ensuring data quality, deduplication, and standardization. * Data Ingestion (Batch & Streaming): Implement ingestion patterns using Dataflow (Apache Beam) for event consumption (Kafka/Event Hubs) and Datastream for CDC from transactional databases. Manage Raw-layer persistence using Iceberg tables governed by BigLake. * Automation and IaC: Use Terraform to provision data resources (datasets, tables, views) and manage pipelines via CI/CD (GitHub Actions), following the Ingestion Factory model and domain-segregated repositories. * Data Quality and Governance: Implement data quality tests (Dataform assertions) and ensure data cataloging and lineage via Dataplex and Analytics Hub for secure cross-domain sharing. Required Qualifications: * Strong SQL experience: Ability to write complex, high-performance queries, preferably in Google BigQuery dialect. * Google Cloud Platform (GCP) knowledge: Hands-on experience with services such as BigQuery, Cloud Storage (GCS), Dataflow, and Cloud Composer (Airflow). * Data Engineering (Python/Spark): Experience processing data using Python and Apache Spark (to understand Databricks legacy and operate Dataproc when needed). * Data Architecture concepts: Understanding of Data Lakehouse, data modeling, partitioning, and file formats (Parquet, Avro, Iceberg). * Version Control and CI/CD: Experience with Git and automated deployment pipelines. Preferred Qualifications (Nice to Have): * Prior experience with Dataform or DBT for orchestrating SQL transformations. * Knowledge of Terraform for Infrastructure as Code (IaC). * Familiarity with event-driven architecture (Kafka or Event Hubs) and streaming processing. * Understanding of Databricks (to facilitate reading and migration of legacy code). * Knowledge of data governance (Dataplex, IAM) and security (VPC Service Controls). What You’ll Get: * A technically challenging environment handling petabytes of data and migrating thousands of objects. * Opportunity to work with cutting-edge GCP technologies (BigLake, Analytics Hub, Gemini for data enrichment). * Engagement in a Data Mesh model, with well-defined ingestion and processing domains. Mid-Level LI-RW1 **Our Benefits:** * Health and dental insurance; * Meal and food allowance; * Childcare assistance; * Extended parental leave; * Gym and health/wellness professional partnerships via Wellhub (Gympass) TotalPass; * Profit and Results Sharing (PLR); * Life insurance; * Continuous learning platform (CI&T University); * Discount club; * Free online platform dedicated to promoting physical, mental health and well-being; * Pregnancy and responsible parenting course; * Partnerships with online course platforms; * Language learning platform; * And many more. More details about our benefits here: https://ciandt.com/br/pt-br/carreiras At CI&T, inclusion starts from the first contact. If you are a person with a disability, it is important to **submit your medical report during the selection process.** *Click here to see which information must be included in the report.* This allows us to provide the support and accommodations you deserve. **If you do not yet have the qualifying medical report, don’t worry—we can support you in obtaining it.** We have a dedicated Health and Well-being team, inclusion specialists, and affinity groups ready to support you at every step. Count on us to walk this journey together.