At CI\&T, we help major enterprises transform the potential of AI into real business impact through AI Deployment, AI-native execution, and tech-integrated business solutions. With 30 years of experience in technological transformation, we accelerate innovation with expertise in agentic SDLC, application modernization, Data \& AI, martech, and business strategy. We are 8,000 CI\&Ters across more than 25 countries, collaborating to build solutions with real impact. AI is already part of how we work, evolve, and innovate every day. **Important**: If you reside in the Campinas Metropolitan Region, your presence at our city offices is mandatory, per our current attendance policy. **About the Challenge:** We are at a critical moment in the evolution of our data platform. The project involves migrating and modernizing a massive legacy system (based on Azure/Databricks) to a new architecture on Google Cloud Platform (GCP). You will build a modern data foundation, following Data Mesh principles, the medallion architecture (Raw/Silver/Gold), and strong governance—ensuring deprecation of the legacy system and enabling new AI and Analytics capabilities. Key Responsibilities: * Migration Execution (Refactoring \& Modernization): Analyze and migrate legacy notebooks and pipelines (Spark/Databricks). This includes both refactoring logic to align with the new architecture and fully rewriting processes using SQL/Dataform or Dataflow. * Building ELT/ETL Pipelines: Develop and maintain data transformations using BigQuery and Dataform (SQL) to build Trusted/Silver and Gold layers, ensuring data quality, deduplication, and standardization. * Data Ingestion (Batch \& Streaming): Implement ingestion patterns using Dataflow (Apache Beam) for event consumption (Kafka/Event Hubs) and Datastream for CDC from transactional databases. Work with persisting raw data using Iceberg tables managed by BigLake. * Automation and IaC: Use Terraform to provision data resources (datasets, tables, views) and manage pipelines via CI/CD (GitHub Actions), following the Ingestion Factory model and domain-segregated repositories. * Data Quality and Governance: Implement data quality tests (assertions in Dataform) and ensure data cataloging and lineage via Dataplex and Analytics Hub for secure cross-domain sharing. **Required Qualifications:** * Solid SQL experience: Ability to write complex and performant queries, preferably in the Google BigQuery dialect. * Knowledge of Google Cloud Platform (GCP): Hands-on experience with services such as BigQuery, Cloud Storage (GCS), Dataflow, and Cloud Composer (Airflow). * Data Engineering (Python/Spark): Experience processing data using Python and Apache Spark (to understand the Databricks legacy and operate Dataproc when needed). * Data Architecture Concepts: Understanding of Data Lakehouse, data modeling, partitioning, and file formats (Parquet, Avro, Iceberg). * Version Control and CI/CD: Experience with Git and automated deployment pipelines. **Nice-to-Have Qualifications:** * Prior experience with Dataform or DBT for orchestrating SQL transformations. * Knowledge of Terraform for Infrastructure as Code (IaC). * Familiarity with event-driven architecture (Kafka or Event Hubs) and streaming processing. * Understanding of Databricks (to facilitate reading and migrating legacy code). * Knowledge of data governance (Dataplex, IAM) and security (VPC Service Controls). **What You’ll Find Here:** * A technically challenging environment handling massive data volumes (petabytes) and migrating thousands of objects. * Opportunity to work with cutting-edge GCP technologies (BigLake, Analytics Hub, Gemini for data enrichment). * Engagement in a Data Mesh model, with well-defined ingestion and processing domains. \#LI\-LL1 **Our Benefits:** * Health and dental insurance; * Meal and food allowance; * Childcare assistance; * Extended parental leave; * Partnerships with gyms and health/wellness professionals via Wellhub (Gympass) TotalPass; * Profit and Results Sharing (PLR); * Life insurance; * Continuous learning platform (CI\&T University); * Discount club; * Free online platform dedicated to physical, mental, and overall well-being; * Pregnancy and responsible parenting course; * Partnerships with online learning platforms; * Language learning platform; * And many more More details about our benefits here: https://ciandt.com/br/pt\-br/carreiras At CI\&T, inclusion starts at first contact. If you are a person with a disability, it is important to **submit your medical report during the selection process.** *Check which information must be included in the report* *by clicking here.* This allows us to guarantee the support and accommodations you deserve. **If you don’t yet have an official medical report, don’t worry—we can support you in obtaining one.** We have a dedicated Health and Well-being team, specialists in inclusion, and affinity groups that will accompany you at every stage. Count on us to walk this journey alongside you.
João Silva
Indeed · HR
João Silva
Indeed · HR
