Data Engineer

Description

Job Summary: A professional to develop and maintain scalable data pipelines, working on data modeling and ensuring resilience and performance of data routines. Key Highlights: 1. Develop and evolve scalable data pipelines with Airflow 2. Work on modeling and structuring analytical and operational data layers 3. Contribute to best practices and continuous improvement of the data platform **Area of Expertise:** Airflow Azure Cloud Non-relational Databases Relational Databases CI/CD ETL Python**City:** Curitiba Hybrid Remote São Paulo**State (UF):** PR SP**Country:** Brazil**2 openings** **About the Company** Scansource: Technology Solutions We are a market reference in IT distribution, delivering technology worldwide and connecting companies and people with the best solutions — all backed by a fully specialized and dedicated team. **Responsibilities** * Develop, maintain, and evolve scalable data pipelines using Airflow as the primary orchestration tool * Build robust integrations across diverse data sources, ensuring consistency, auditability, and data quality * Work on data modeling and structuring of analytical or operational data layers * Monitor and support data routines, ensuring resilience, performance, and rapid recovery in case of failures * Contribute to best practices such as versioning, automation, observability, and architectural standards * Collaborate with product and business teams to define requirements and prioritize demands * Actively participate in technical reviews, architectural discussions, and initiatives for continuous improvement of the data platform **Requirements** * Practical experience in **Data Engineering**, preferably in high-volume, highly complex environments * Proficiency in **Airflow** for creating and maintaining **DAGs**, including orchestration and monitoring best practices * Strong **Python** skills for developing tasks, connectors, and transformations, and for data manipulation, ingestion, and transformation * Solid experience with **relational and non-relational databases** (e.g., PostgreSQL, MySQL, MongoDB, BigQuery, Redshift, etc.) * Experience with **ETL/ELT**, data modeling, partitioning, query optimization, and structuring of data layers * Familiarity with **cloud environments** (Azure), including storage, processing, and security services * Hands-on experience with version control tools (Git/GitHub), CI/CD, containers, and automation * Bachelor’s degree in Engineering, Computer Science, Information Systems, or related fields **Preferred Qualifications:** * Experience with microservices, Kafka, Spark, Infrastructure as Code, or cloud certifications **Profile** * Analytical ability * Planning and organization * Communication * Flexibility * Initiative **Benefits** **For residents of SP, DF, or RJ** * Flexible meal card (VR) – BRL 45.50 per workday * Transportation allowance or company parking – available only for employees in São Paulo capital * Porto Seguro health insurance, extendable to spouse and children * Dental insurance * Totalpass * Life insurance * PPR/PLR (Profit Sharing Program) * Employee referral program * Pet care benefits **Residents of other regions** * Flexible meal card (VR) – BRL 39.00 per workday * Transportation allowance or company parking – available only for employees in Curitiba and surrounding areas * Unimed Nacional health insurance, extendable to spouse and children * Dental insurance * Totalpass * Life insurance * PPR/PLR (Profit Sharing Program) * Employee referral program * Pet care benefits **Employment Model:** CLT – Hybrid – 3 days per week at the office for São Paulo and Curitiba. São Paulo/SP residents – Av. Paulista, 2300 – 16th Floor – Consolação Curitiba/PR residents – Avenida Sete de Setembro, 2451, 5th Floor – Centro Residents of other regions: CLT – 100% remote.