Senior Data Engineer

Description

As a **Data Engineer**, you will work closely with Product teams to translate business needs into efficient, scalable, and highly valuable data products. You will be responsible for designing the data architecture, modeling data, building ultra-low-latency pipelines, and constructing integrations (APIs) for direct consumption of data from modern analytical repositories. Additionally, you will play both a consultative and execution role in aligning with Artificial Intelligence (AI) processes by creating skills (competencies/tools) for intelligent agents. **ABOUT CONTA AZUL** Conta Azul is driven by the belief that every entrepreneur deserves to succeed. Despite their busy day-to-day responsibilities, we want every small-business owner to have time to focus on what they’ve always dreamed of since deciding to start their company. That’s why we use technology to build a cloud-based platform where entrepreneurs — together with their accountants — can easily and simply access everything they need, in real time. We seek people motivated by this purpose. If you’re one of them, join us! **ABOUT THE TEAM** The **Data Engineering** team unites software engineering and modern analytics to ensure an ultra-fast and resilient data ecosystem at Conta Azul. Our mission is to sustain the foundation that powers the company’s next-generation products and intelligent agents. We operate with high technical autonomy and a product mindset — the ideal place for those seeking ownership and wanting to see the real-world impact of their code at the end user. **Responsibilities and Assignments** * **Architecture & Modeling**: Design and evolve the analytical data architecture, focusing on scalable, resilient, and Near Real-Time (NRT)-oriented process solutions. * **Partnership with Product**: Collaborate synergistically and consultatively with Product teams to understand business rules, optimize data modeling, and ensure data accurately reflects the product’s and end customer’s actual needs. * **Consumption via APIs**: Develop and implement efficient integration layers to expose analytical data through robust APIs, extracting maximum performance directly from columnar databases. * **Data Transformation**: Build, test, and document data transformation pipelines applying best practices of software engineering (CI/CD, data testing, versioning). * **AI Enablement**: Create skills, connectors, and integration components enabling AI systems and agents to securely and swiftly consume, interpret, and interact with the analytical data layer. * **Data-Driven Culture**: Support governance of data modeling and structuring of datasets ready for consumption by data visualization tools (Data Vis). **Requirements and Qualifications** **REQUIRED** * **Cloud Computing**: Strong hands-on experience with GCP (Google Cloud Platform) or AWS stacks (Azure knowledge is acceptable but GCP/AWS are preferred). * **Programming Language**: Proficiency in Python (preferred) or another modern language suited for pipeline and integration development. * **Analytical (Columnar) Databases**: Advanced experience with ClickHouse or StarRocks and BigQuery (with strong emphasis on the first two to ensure ultra-low-latency processing). * **Transformation Pipelines**: Solid knowledge and practical application of best practices using modern transformation tools such as Google Dataform or dbt (data build tool). * **Integration & APIs**: Experience with tools, frameworks, or architectures for exposing analytical data via high-performance APIs. * **Relational Databases**: Strong expertise in relational databases, especially PostgreSQL. * **Modeling and Visualization**: Solid background in analytical data modeling techniques (dimensional, Data Vault, etc.) and familiarity with Data Vis tools (e.g., Looker, Looker Studio, Power BI, Metabase, etc.). * **Product Mindset**: Ability to engage on equal footing with Product Managers and Product Designers, focusing on value delivery and end-user experience. * **Architectural Vision**: Ability to view the data ecosystem holistically, anticipating performance bottlenecks and unnecessary cloud costs. * **Autonomy and Technical Leadership**: Capacity to make architectural decisions for the Near Real-Time layer and guide the team toward industry best practices. **DESIRED** * **CDC (Change Data Capture) Processes**: Experience with CDC tools and workflows for continuous ingestion from transactional databases. * **Messaging and Streaming**: Practical knowledge of high-throughput messaging tools, especially Apache Kafka (Confluent / Amazon MSK). * **AI Engineering**: Prior experience developing tools or pipelines feeding LLMs, using vector databases (Vector DBs), or frameworks for autonomous agents. **Additional Information:** **BENEFITS** * **Health Insurance**: Nationwide coverage with no monthly fee or co-payment. Can be extended to dependents (spouse and children), per internal policy. * **Dental Insurance**: Nationwide coverage with no monthly fee. Can be extended to dependents (children and spouse), per internal policy. * **Flexible Benefit**: R$ 1,000.00 per month to invest in education (70% reimbursement on tuition), languages (70%), physical activities, and sessions with psychologists, nutritionists, massage therapists, and physiotherapists (50%). * **Childcare Assistance** for parents with children aged 0–24 months. * **Monthly Home Office Allowance** — Monthly support starting at R$ 150.00 (based on location). * **Home Office Setup Allowance** — One-time payment to cover workstation setup. * **Meal and Food Voucher**: Flash card with no payroll deduction. * **Zenklub**: Unlimited access to psychologists, therapists, coaches, and psychoanalysts. The first two sessions each month are free; subsequent sessions cost only R$ 20.00. * **Bonus Program**, based on goal achievement. * **Pharmacy Benefit**: Vidalink app offering 75% subsidy for prescription medications. Can be extended to dependents (children and spouse), per internal policy. * **Extended Maternity and Paternity Leave**. * **Free Sesc Membership**: Benefits extendable to dependents. Membership grants access to numerous activities and services at special prices — including education, sports, leisure, excursions, hotels, cultural programming, dental care, and nutrition. You pay only for the activities you use. * **Transportation and Parking**: Transportation voucher (up to 6% payroll deduction) and free parking at headquarters (Joinville). * **GoGood**: Access to various gyms across Brazil with a 50% subsidy for Azuis. Dependents may enroll in the full plan. * **Relocation Assistance**: For all Azuis wishing to relocate closer to headquarters, we offer financial support for the move. Experience Package: * **Baby Blue Time**: Reduced working hours with extra schedule flexibility upon returning from Maternity and Paternity Leave; * **Days Off Experience**: All Azuis with over one year of tenure receive four days off annually; * **Partially Paid Sabbatical**: Eligible Azuis may take time off while receiving an advance stipend; * **Life Insurance**: No payroll deduction; * **Free Conta Azul Platform License**. And more: * Innovative environment fostering growth, with exceptional colleagues and continuous learning; * Accessible office designed to stimulate creativity and productivity, with food and beverages available at no cost; * Flexible working hours (Monday to Friday). **Diversity & Inclusion at Conta Azul** At Conta Azul, we value an inclusive environment where everyone feels respected and where diverse life experiences and individual characteristics of each Azul are appreciated. Therefore, we promote diversity throughout our selection processes — reflecting our belief in integration and appreciation of every person. Feel free to declare relevant information in the registration form. These data are essential to foster an inclusive selection process and will be used exclusively for this purpose, with no impact during or after the process. **Inside Conta Azul** Our teams love sharing knowledge. See what we’re up to here. Visit: https://medium.com/cultura-contaazul https://medium.com/design-contaazul https://engineering.contaazul.com