Senior Data Engineer

Description

Job Summary: Build robust data pipelines, ensure data quality, and optimize processing environments to enhance customer experience. Key Highlights: 1. Building robust and scalable data pipelines 2. Ensuring data quality from collection through processing 3. Continuously improving customer experience Responsibilities and Duties: Build robust and scalable data ingestion and transformation pipelines that meet the information needs of business and data teams; Contribute to a resilient data platform covering all aspects of data quality. This quality assurance must be applied from data collection from systems, vendors, and customers through processing and delivery to the curated layer for analytical use. Identify stakeholders across all processes and establish data modeling and governance contracts. Analyze processing environments for stability improvements; Understand usage of available tools and customer pain points to improve their experience. Requirements and Qualifications: Prior experience in data engineering and Big Data; Prior experience with Python and/or Scala, SQL, and the Spark framework; Data architecture and mapping/modeling (relational and/or analytical); ETL/Orchestration tools such as Apache Airflow, Databricks, etc.; Cloud experience, preferably AWS; DevOps experience, including CI/CD and GitHub; Metrics for analytical data quality; Unit testing. Nice-to-have: Relevant academic background in Computer Science, Software Engineering, or related fields; Experience handling financial data: Knowledge and experience working with financial data, including transaction processing, data security, and privacy; Participation in open-source projects. Position: Senior Data Engineer Employment Type: CLT (Permanent) Professional Area: Engineering \- Others / Marketing Working Hours: 40 Number of Openings: 1 Benefits: What We Offer: Health and Well-being Medical and Dental Insurance; Life Insurance; "Lado a Lado" Program, offering legal, social, psychological, and financial support; TotalPass. Meals/Mobility or Remote Work: Credit card-based benefits, widely accepted, where you receive your meal/food, mobility, or remote work allowance (as per your employment contract). Requirements: Prior experience in data engineering and Big Data; * Prior experience with Python and/or Scala, SQL, and the Spark framework; * Data architecture and mapping/modeling (relational and/or analytical); * ETL/Orchestration tools such as Apache Airflow, Databricks, etc.; * Cloud experience, preferably AWS; * DevOps experience, including CI/CD and GitHub; * Metrics for analytical data quality; * Unit testing; Nice-to-have: * Relevant academic background in Computer Science, Software Engineering, or related fields; * Experience handling financial data: Knowledge and experience working with financial data, including transaction processing, data security, and privacy; * Participation in open-source projects. Prior experience in data engineering and Big Data; ● Prior experience with Python and/or Scala, SQL, and the Spark framework; ● Data architecture and mapping/modeling (relational and/or analytical); ● ETL/Orchestration tools such as Apache Airflow, Databricks, etc.; ● Cloud experience, preferably AWS; ● DevOps experience, including CI/CD and GitHub; ● Metrics for analytical data quality; ● Unit testing; Nice-to-have: ● Relevant academic background in Computer Science, Software Engineering, or related fields; ● Experience handling financial data: Knowledge and experience working with financial data, including transaction processing, data security, and privacy; ● Participation in open-source projects.