Senior Data Engineer – Python, Scala & Spark | Data Engineering & Data Modeling

Description

Job Summary: We are seeking a data engineering specialist with strong expertise in Python, Scala, and Apache Spark, capable of serving as a technical reference in building scalable data pipelines and high-performance solutions. Key Highlights: 1. Autonomy and architectural vision 2. Dynamic and challenging environment 3. Opportunity to work with diverse technologies **What We Are Looking For?** A data engineering specialist with strong proficiency in Python, Scala, and Apache Spark. We seek someone who can serve as a technical reference—autonomous, architecturally minded, and experienced in building scalable pipelines, data modeling, and high-performance solutions. **Key Responsibilities:** Build and maintain high-performance data pipelines (ETL/ELT); Develop solutions in Python, Scala, and Spark for large-scale data; Data modeling (dimensional/Kimball, Data Vault, relational/analytical); Orchestrate workflows (Airflow, ADF, Luigi; Control-M is a plus); Work with data layers (Landing, Cleaning, Processing, Curation, Data Marts); Integrate sources/services using SQL, NoSQL, and Data Lakes (Parquet/Delta/ORC); Implement best practices: Git, data testing, observability; Collaborate with architecture, product, and analytics teams. **Requirements:** Python and Scala for pipelines and automation; Apache Spark (Spark SQL, DataFrames, optimization/partitioning); Data modeling: Dimensional (Kimball), Data Vault, relational/analytical; Orchestration: Airflow, ADF, Luigi (Control-M preferred); Big Data: Hadoop, Hive, Kafka (integration/streaming); Databases and formats: Postgres, SQL Server, Oracle; MongoDB, Cassandra, Redis; Parquet, Delta Lake, ORC; Cloud platforms: AWS, Azure, or GCP; Data layers: Landing, Curation; ETL chaining; Data Marts; Best practices: Git, testing, observability, query/load performance; Advanced Spanish. **Nice-to-Haves:** MapReduce, data governance, advanced monitoring/tuning; Self-service tools (Looker/Metabase) and applied descriptive statistics; Intermediate/advanced English. **Work Location: Remote** **Why Build Your Career at Meta?** We offer autonomy, clear goals, and a dynamic and challenging environment where professionals have the opportunity to interact with diverse technologies, participate in all types of projects, bring new ideas, and work from anywhere in Brazil—and why not, the world. Additionally, we are one of the best companies to work for in Brazil according to Great Place to Work, and one of the 10 fastest-growing companies in the country for three consecutive years, per the Informática Hoje Annual Report. **What Are Our Values?** * We are people serving people * We think and act like owners * We strive for performance * We grow and learn together * We pursue excellence and simplicity * Innovation and creativity are in our DNA All individuals are welcome regardless of background, disability, ethnicity, religious belief, sexual orientation, appearance, age, or related factors. We want you to grow with us in an inclusive, welcoming environment full of opportunities. If this resonates with you, \#VemSerMeta!