We are specialists in **technological transformation**, combining human expertise with AI to build scalable tech solutions. With over 8,000 CI&Ters worldwide, we have partnered with more than 1,000 clients throughout our 30-year history. Artificial Intelligence is our reality. **Important**: If you reside in the Campinas Metropolitan Region, your physical presence at our city offices is mandatory, per our current attendance policy. We are seeking a Data Engineer to join our team on a project to structure the data platform for a client in the financial services and asset management sector. You will be responsible for bringing architectural definitions to life—transforming technical guidelines into robust pipelines, consistent data models, and scalable solutions that enable the client to organize and extract value from their data—currently scattered across spreadsheets and unstructured repositories. In this role, you will operate collaboratively and operationally, implementing ELT/ETL pipelines, building Data Lake/Lakehouse layers, and ensuring data quality and traceability—all under the guidance of the Data Architect. The environment involves small data, with an emphasis on integration and structuring, which values deep technical expertise, sound engineering judgment, and the ability to deliver autonomously within the defined scope. **Responsibilities:** * Pipeline Development: Implement ELT/ETL pipelines for ingestion, transformation, and delivery of data from diverse sources—including spreadsheets, SharePoint, and relational databases—following architecture-defined standards; * Data Platform Construction: Build and organize Data Lake/Lakehouse layers (Bronze, Silver, Gold / Medallion Architecture) using optimized formats such as Delta Lake and Parquet; * Transformation and Modeling: Develop data transformations using PySpark and advanced SQL, implementing architecture-defined models in the platform’s analytical layers; * Ingestion Strategies: Implement appropriate loading strategies (full or incremental) based on volume and criticality per data domain—with emphasis on efficiency in small-data contexts; * Data Quality: Implement automated data quality tests and validations within pipelines, ensuring consistency, integrity, and traceability across layers; * Cataloging and Governance: Support cataloging and documentation of data assets with focus on lineage and classification, following governance guidelines defined by the Data Architect, establishing a unified data catalog as the central element of platform management; * Observability: Ensure pipeline observability through logs, alerts, and proactive monitoring; * Documentation: Document models, transformations, and business rules applied in pipelines, ensuring traceability and maintainability of the solution. **Requirements for this challenge:** * Proven experience in Data Engineering within cloud environments; * Solid experience with PySpark for developing pipelines and data transformations on distributed platforms; * Experience with advanced SQL for querying, transforming, and analytical modeling; * Knowledge of Microsoft Azure (Azure Data Factory, ADLS Gen2 and/or Microsoft Fabric); * Experience with data formats such as Delta Lake, Parquet, or ORC; * Experience with Data Lake, Lakehouse, and Medallion Architecture; Experience with data cataloging and lineage, including Unity Catalog (Databricks). * **Differentiators:** * Experience with Databricks and/or Snowflake as data processing platforms; * Familiarity with Microsoft Fabric (Lakehouses, Notebooks, Pipelines, or Dataflows); * Experience with Microsoft Purview for data cataloging and lineage; * Knowledge of dbt for data transformations and quality testing; * Experience orchestrating pipelines via Apache Airflow or Prefect; * DP-700 certification or equivalent; * Best development practices: Git versioning, automated testing, documentation, and CI/CD; * Familiarity with Gen AI and autonomous agents applied to data. #LI-BS3 #senior **Our benefits:*** Health and dental insurance; * Meal and food allowance; * Daycare assistance; * Extended parental leave; * Partnerships with gyms and health & wellness professionals via Wellhub (Gympass, TotalPass); * Profit and Results Sharing Program (PLR); * Life insurance; * Continuous learning platform (CI&T University); * Discount club; * Free online platform dedicated to promoting physical, mental, and overall well-being; * Pregnancy and responsible parenting course; * Partnerships with online course platforms; * Language learning platform; * And many more More details about our benefits here: https://ciandt.com/br/pt-br/carreiras At CI&T, inclusion begins at first contact. If you are a person with disability, it is important to **submit your medical report during the selection process.** *Check which information must be included in the report* *by clicking here.* This way, we can ensure the support and accommodations you deserve. **If you do not yet have the qualifying medical report, don’t worry—we can support you in obtaining it.** We have a dedicated Health & Wellness team, inclusion specialists, and affinity groups who will accompany you at every stage. Count on us to walk this journey alongside you.
João Silva
Indeed · HR
João Silva
Indeed · HR
