Data Engineer

Description

Job Summary: We are seeking a Data Engineer to structure, organize, and provide data for strategic analysis by developing and maintaining robust data pipelines in the insurance sector. Key Highlights: 1. Develop and maintain robust and scalable data pipelines 2. Integrate data from multiple sources for strategic analysis 3. Ensure data governance, quality, and security We are looking for a Data Engineer with strong capabilities to structure, organize, and provide data for strategic analysis. The professional will be responsible for developing and maintaining robust, highly available, and scalable data pipelines, ensuring data quality and governance in an environment focused on automobile insurance. Responsibilities * Develop, optimize, and maintain data ingestion, transformation, and processing pipelines (batch and streaming). * Design and manage data lakes and data warehouses to support business analytics, pricing, claims frequency, and policy performance. * Integrate data from multiple sources (ERP, inspection systems, telematics, CRM, partners, and external APIs). * Establish and ensure best practices for data governance, quality, traceability, and security. * Collaborate with dashboard development teams for decision-making; * Implement monitoring, observability, and automation solutions to ensure performance and reliability. Technical Requirements * SQL (advanced) * ETL/ELT and orchestration tools (Airflow, Data Factory, Prefect, etc.) * Data modeling (Kimball, Data Vault, analytical models) * Python or Scala for data processing * Knowledge of cloud services (Azure, AWS, Oracle, or GCP) * Streaming tools (Kafka, Event Hub, etc.) * Git * CI/CD * Best practices for security and LGPD compliance * Data Lakehouse, Delta Lake, Iceberg, or similar technologies Nice-to-Have * Containers and orchestration: Docker, Kubernetes, AKS/EKS/GKE * Big Data * Orchestration: Apache Airflow, Prefect, Dagster * Messaging and streaming: Apache Kafka, Redpanda * Data governance and cataloging: Amundsen, DataHub, OpenMetadata * Azure Synapse / Fabric * Power BI for supporting the analytics team ### **Employment Type:** Contractor (PJ) ### **Department:** Technology