AWS Data Engineer (Mid-level)

Description

Job Summary: A professional to develop software, create and maintain data pipelines, model databases, and collaborate with business teams to meet requirements. Key Highlights: 1. Software Development and Cloud Computing (AWS) 2. Creation and maintenance of data pipelines in the data lake 3. Database modeling and requirements analysis Description: * Software development; * Cloud Computing on the AWS platform; * ETL concepts; * SQL; * Spark; * Python and/or PySpark programming; * Open Source projects; * SRE (Concept); * RedShift; * S3; * Crawler; * Glue; * Kubernetes; * Airflow; * Creation/maintenance of data pipelines between layers in the data lake, including optimization; * Database preparation/modeling; * Ingestion of databases into the data lake; * Data requirements analysis and elicitation together with users (business area); * Support the development team with daily and new requests; * Create/maintain data pipelines between layers in the data lake; * Develop/model databases for dashboards, reports, algorithms, and/or refined datasets; * Ingest data into the data lake; * Document daily activity updates in JIRA; * Document metadata/business rules using the internal standard document; * Understand/analyze database/variable requirements with the business; * Perform database ingestion into the data lake; * Conduct data quality analysis and ensure analytics standards are applied; * Version control of developed code. 251127020240255870