Data Engineer

Description

About the position: We are seeking a Data Engineer to build, optimize, and maintain scalable data pipelines, ensuring data quality, governance, and efficiency across the entire data processing and storage lifecycle. This person will play a strategic role in integrating, transforming, and delivering reliable data to support Analytics, Business Intelligence, Data Science, GenAI, and Data Product initiatives—directly contributing to data-driven decision-making and business evolution. Responsibilities: ✔ Develop and optimize scalable, high-performance batch data pipelines for ingesting structured, semi-structured, and unstructured data into the Data Lake (GCS). ✔ Develop and manage ETL/ELT processes using Spark and SQL to populate the Trusted (Silver) and Refined (Gold) layers in the Data Warehouse (BigQuery). ✔ Consolidate, integrate, and model data from internal and external sources to support analytical initiatives and data products. ✔ Develop data integrations and streaming pipelines for real-time ingestion and processing on GCP. ✔ Design and implement efficient data architectures, balancing performance, scalability, cost, and development speed. ✔ Translate and implement business rules into data transformation and dataset delivery processes. ✔ Establish and maintain processes related to data quality, governance, versioning, lineage, and reliability. ✔ Serve as the technical reference for the Data & Analytics team, supporting strategic and innovative initiatives. Requirements: ✔ Solid experience in Data Engineering, including building and optimizing ETL/ELT pipelines. ✔ Proficiency in Python and SQL for data development and processing. ✔ Hands-on experience with Google Cloud Platform (GCP), especially BigQuery, Dataflow, Pub/Sub, Cloud Storage, and Cloud Composer. ✔ Knowledge of integrating and processing structured, semi-structured, and unstructured data. ✔ Experience with data modeling and query optimization in BigQuery, PostgreSQL, Snowflake, or Redshift. ✔ Familiarity with workflow orchestration and automation tools such as Apache Airflow, Cloud Composer, or Prefect. ✔ Experience designing and implementing Data Lakes and Data Warehouses. ✔ Knowledge of data governance, including lineage, access control, versioning, and compliance best practices. ✔ Familiarity with code versioning and automation practices using Git and CI/CD pipelines. Nice-to-have: ➕ Experience with streaming data processing and ingestion using Kafka or Pub/Sub. ➕ Knowledge of infrastructure-as-code using Terraform. ➕ Experience in Data Science, GenAI, or Data Product initiatives. ➕ Knowledge of data privacy and security regulations and practices, such as LGPD and GDPR. ➕ Experience in product-oriented environments and data-driven cultures. Work model: Hybrid What you’ll find at Maitha: Individual Development Experience for soft skills, with guidance throughout your journey; Well-being management with tracking of metrics and indicators on satisfaction progress; Training sessions, workshops, and meetups to enhance both soft and hard skills; Rewards program with points and incentives; TotalPass offering multiple plan options and over 250 modalities—including weight training, cross-training, yoga, dance, and more; And much more!