AWS Data Engineer (Mid-Level)

Description

Job Summary: A professional for software development, creating and maintaining data pipelines, modeling databases, and collaborating with teams and users. Key Highlights: 1. Software Development 2. Creation and Maintenance of Data Pipelines 3. Collaboration with Teams and Data Requirements Elicitation Description: * Software Development; * Cloud Computing on the AWS platform; * ETL Concepts; * SQL; * Spark; * Programming in Python and/or PySpark; * Open Source Projects; * SRE (Concept); * RedShift; * S3; * Crawler; * Glue; * Kubernetes; * Airflow * Creation/Maintenance of data pipelines across layers in the data lake, including optimization. * Database preparation/modeling; * Ingestion of databases into the data lake; * Data requirements analysis and elicitation from users (business area). * Support the development team with daily and new demands; * Create/maintain data pipelines across layers in the data lake; * Develop/model databases for dashboards, reports, algorithms, and/or refined datasets; * Ingest data into the data lake; * Document daily activity updates in JIRA; * Document metadata/business rules using the standard internal document; * Understand/elicit database/variable requirements from the business; * Perform database ingestion into the data lake; * Conduct data quality analysis and ensure analytics standards are applied; * Version developed code. 251127020240255870