Mid-Level Data Engineer

Description

Job Summary: We are seeking a professional to develop data ingestion and transformation pipelines, automating them and implementing re-execution controls. Key Highlights: 1. Develop data ingestion and transformation pipelines 2. Work with transactional and analytical databases 3. Use GCP technologies such as Dataflow, Composer, and Cloud Storage **We are looking for someone who dares to co-lead with us.** **Responsibilities and Duties:** Develop data ingestion pipelines from source systems into the raw/trusted layers of the data lake; Develop data transformation pipelines to meet business requirements; Develop batch and streaming (NRT) pipelines; Automate pipelines for automatic execution according to business requirements; Create controls for pipeline re-execution/resumption in case of failure. **Requirements and Qualifications:** Transactional databases (Oracle, Sybase, SQL Server, MySQL); Analytical databases (GCP BigQuery); DBT -> Data/system integration; GCP Dataflow (Apache Beam); GCP Composer (Apache Airflow); Knowledge of GCP Cloud;-> Data storage (Google Cloud Storage). **Desirable:** **Additional Information:** * For your health, we offer **medical and dental assistance** with no co-payment for exams or consultations. * Want more well-being? We offer **Gympass** and **Zenklub** (with two free sessions per month) to support your physical and mental health. * When hunger strikes, use our meal benefit **Caju** to shop at any supermarket or restaurant. * Love traveling? Here, we have **Férias & Co**, a benefit that helps make your dream vacation possible. * Have children? Count on our **childcare allowance**. * Growing family? We offer **MomCare**, a flexible monetary incentive post-maternity leave for new Datunian mothers. * To personalize your workspace, count on our **home office allowance**. * Want to invest in your professional development? We partner with several renowned educational institutions to offer you discounts of up to 70% on **undergraduate**, **postgraduate**, **MBA**, **language**, and many other courses! **To learn about all our benefits, visit the homepage of our careers page.**