**Description:** ---------------- We are seeking a Data Engineer to build and optimize our data pipelines. You will be responsible for ensuring that the information flow is reliable, scalable, and that data is ready for consumption by BI and Artificial Intelligence. The focus will be entirely on the Google Cloud ecosystem, transforming raw data into business assets. **Key Responsibilities** * Develop and maintain ETL/ELT pipelines (Batch and Streaming) using the GCP ecosystem. * Data modeling: Design and optimize tables in BigQuery, ensuring performance and cost control (partitioning and clustering). * Workflow orchestration: Create and monitor complex workflows using Cloud Composer (Airflow). * Source integration: Ingest data from APIs, relational databases, and legacy systems to feed the Data Lake/Lakehouse. * Ensure data quality and governance: Implement data tests and guarantee data lineage. * Cost optimization: Monitor cloud resource consumption and propose improvements to avoid unnecessary expenses. **Requirements:** ----------------- * Solid Experience: Proven experience in data engineering projects (minimum 3 years). * BigQuery Proficiency: Deep knowledge of advanced SQL, query optimization, and dataset management. * GCP Experience: Practical use of services such as Cloud Dataflow (Apache Beam), Cloud Storage, Pub/Sub, and Cloud Functions. * Orchestration: Experience with Apache Airflow (Cloud Composer). * Programming Language: Proficiency in Python for data processing. * Infrastructure: Knowledge of Docker and CI/CD concepts applied to data. **Nice-to-Haves** * Certification: Google Professional Data Engineer. * Infrastructure as Code: Knowledge of Terraform for provisioning GCP resources. * Software Engineering: Experience with unit testing and continuous integration for pipelines. * FinOps: Experience with cloud cost monitoring and reduction. Do you identify with this opportunity? Then apply now and \#VemPraSauter
João Silva
Indeed · HR
João Silva
Indeed · HR
