Job Summary: We are seeking a Data Engineer to map and document data sources, and to design, build, and operate pipelines with a focus on security, performance, reliability, and compliance. Key Highlights: 1. Mapping and documentation of internal and external data sources 2. Responsible for designing, building, and operating pipelines with a focus on security 3. Collaborating with Product/Business teams to define rules **Job Description:** We are looking for a **Data Engineer** to perform **mapping and documentation of data sources** alongside internal teams and **external vendors** (e.g., **SGA, PROSIS, ServiceNow**), defining **ingestion specifications**, **connector standardization**, and **ETL/ELT mechanisms**. You will be responsible for **designing, building, and operating pipelines** focused on **security, performance, reliability, and compliance** (governance/LGPD), ensuring **data availability** for consumption layers and the corporate data catalog. **Responsibilities** * Gather technical requirements from internal departments and partners (e.g., SGA, PROSIS, ServiceNow) to **map, classify, and document** data sources (structured, semi-structured, and APIs). * Define **ingestion specifications** (batch/streaming), **connector standards**, and **data contracts** (schemas, SLAs/SLOs, versioning). * Design and implement resilient and observable **ETL/ELT pipelines** (reprocessing, idempotency, alerts), with **data quality monitoring** (DQ checks) and **end-to-end lineage**. * Optimize **performance and cost** (partitioning, clustering, compression, parallelism), applying **FinOps** where applicable. * Publish datasets to **Bronze/Silver/Gold layers** and the **corporate data catalog** (metadata, access policies, classification). * Ensure **security and compliance**: IAM, encryption, masking, anonymization/pseudonymization, retention, and auditing—aligned with **LGPD** and governance policies. * Operate and evolve **orchestration** (job execution/retries, SLAs), perform **tuning/troubleshooting**, and support analytics/BI teams in data consumption. * Collaborate with Product/Business teams to define **business rules** and facilitate **handoff to consumption layers** (APIs, views, marts). **Requirements** * Proven experience in **data engineering**, including building and operating **ETL/ELT pipelines** (batch and/or streaming). * Strong **SQL** and **Python** skills; hands-on experience with **Spark** and/or **dbt** is a plus. * Practical experience with **cloud data platforms** (preferably **GCP**: **BigQuery, Dataflow/Dataproc, Pub/Sub, Cloud Storage, Cloud Composer, Dataplex**; or equivalent AWS/Azure services). * Knowledge of **data catalog and lineage tools** (e.g., **Dataplex/Data Catalog/Atlas**), **data quality**, and **analytics-oriented modeling** (Medallion architecture, marts). * Experience with **orchestration** (Airflow/Composer), **Git/CI\-CD**, and **observability** (logs, metrics, alerts). * Familiarity with **security and privacy** (IAM, encryption, LGPD), **API-based integration**, and connectors (REST, JDBC/ODBC). * Ability to produce technical documentation and **communicate effectively** with business units and vendors. * **Bachelor's degree** in IT, Engineering, Computer Science, Information Systems, or related fields; certifications (e.g., **GCP Data Engineer**, **dbt**, **ITIL/DAMA**) are advantageous. ### **Employment Type:** CLT ### **Benefits:** Meal Voucher, Transportation Voucher, Culture Voucher, Health Insurance, Life Insurance ### **Department:** Corporate
João Silva
Indeed · HR
João Silva
Indeed · HR
