···
Log in / Register
Senior Data Engineer
Indeed
Full-time
Onsite
No experience limit
No degree limit
Praça do Patriarca, 62 - Centro Histórico de São Paulo, São Paulo - SP, 01002-010, Brazil
Favourites
Share
Some content was automatically translatedView Original
Description

Job Summary: Design, build, and scale the organization’s data architecture, ensuring resilient pipelines, trustworthy data, high performance, observability, and applied governance. Key Highlights: 1. Lead the technical design of Data Lakes, Data Warehouses, and Data Mesh. 2. Serve as the technical reference in cross-functional squads. 3. Mentor and develop team members, fostering technical growth. ? Purpose of the Role Design, build, and scale the organization’s data architecture, ensuring **resilient pipelines**, **trustworthy data**, **high performance**, **observability**, and **applied governance**. This role is critical to enabling **advanced analytics, AI, and data-driven strategic decisions**, acting as the technical reference and accelerating the maturity of the data ecosystem. ? Key Responsibilities * Design and develop **scalable data pipelines** (batch and streaming), with a focus on **resilience, performance, and cost-efficiency**. * Lead the **technical design of Data Lakes, Data Warehouses, Lakehouse, and Data Mesh**, ensuring alignment with corporate standards and governance best practices. * Implement **ingestion, transformation, and orchestration** processes for large volumes of structured and unstructured data. * Ensure **data lineage, cataloging, metadata management, and technical documentation**, collaborating closely with Data Owners and Data Stewards. * Build and maintain **automated data quality tests**, ensuring validity, completeness, consistency, integrity, and compliance. * Optimize **data modeling, queries, and costs** in analytical engines such as BigQuery, Snowflake, Redshift, Databricks, Synapse, among others. * Structure **trusted data layers (curated/gold layers)** for analytical, BI, and AI consumption. * Act as the **technical reference in cross-functional squads**, supporting product, data science, and business teams. * Implement **data observability**, including monitoring, logging, metrics, alerts, and definition of data SLAs/SLOs. * Evaluate, recommend, and standardize **tools, frameworks, and architectures**, ensuring scalability, security, and compliance. * Mentor and develop team members, promoting **best practices, standardization, and continuous technical evolution**. ?️ Technical Stack (Hard Skills) Data Architecture & Strategy * Data Lake, Data Warehouse, Lakehouse * Data Mesh, Federated Data Mesh, Data Fabric * Domain-Driven Data Pipelines Ingestion & Pipelines * ETL / ELT, CDC, Streaming * API-based integration, Webhooks, events, and queues Orchestration * Airflow, Prefect, Dagster, DBT Cloud * Dataplex, Data Factory, Step Functions Transformation & Processing * Advanced SQL * Python, Spark (PySpark / Scala) * DBT (Core / Cloud), Dataform * Databricks, Dataproc, analytical notebooks Large-Scale Processing * BigQuery, Databricks, Hadoop * Dataflow, Dataproc, Glue, EMR * Delta Lake, Lakehouse architecture Streaming & Messaging * Kafka, Pub/Sub, Kinesis * Flink, Spark Structured Streaming Governance & Data Quality * Data cataloging, lineage, and classification * Data Quality and Data Contracts * Tools such as: Great Expectations, Soda, DBT Tests, OpenLineage, Dataplex, Purview, Collibra DevOps & Infrastructure * Terraform and infrastructure-as-code * CI/CD (GitHub Actions, GitLab CI, Jenkins, Cloud Build) * Docker, Kubernetes, Helm * Monitoring: Prometheus, Grafana, Cloud Monitoring Modeling & Performance * Data Modeling (Kimball, Star/Snowflake) * Partitioning, clustering * Query tuning, caching, and cost optimization * Senior-Level Differentiators * Technical leadership in **distributed architectures and highly complex projects**. * Experience with **high-volume data**, mission-critical pipelines, and sensitive data (financial, healthcare, benefits, services). * Experience in **regulated environments** (LGPD, BACEN, SOX, ISO 27001, GDPR). * Ability to design **end-to-end solutions**, balancing cost, performance, and security. * Experience with **real-time data, AI applied to pipelines, feature stores, and AI governance**. * Definition and management of **SLAs, SLOs, and Data Contracts across domains**. * Technical and executive communication, with strong business alignment. * Consistent track record as a **mentor and builder of high-performance data teams**. ? Soft Skills * Clear and assertive communication with technical teams and executive leadership. * Systems thinking and focus on scalability, quality, and business impact. * Ability to handle **critical incidents**, including root cause analysis. * Collaborative, proactive, and results-oriented profile. * Organization, discipline, and excellence in technical documentation. * Data-, metric-, and evidence-driven decision making. ? Expected Deliverables * **Documented, versioned, and standardized** data architecture and pipelines. * Automated pipelines with **observability, clear alerts, and metrics**. * Applied **data quality tests and data contracts**. * Data available in **trusted layers for BI, Analytics, and AI**. * Dashboards for monitoring pipeline health, data quality, and performance. * **Cost reduction** and measurable performance gains. * Solutions delivered with **resilience, security, and governance**.

Source:  indeed View original post
João Silva
Indeed · HR

Company

Indeed
Cookie
Cookie Settings
Our Apps
Download
Download on the
APP Store
Download
Get it on
Google Play
© 2025 Servanan International Pte. Ltd.