Browse
···
Log in / Register
We are looking for a candidate focused on Generative AI Engineering to work directly with our technology team, driving the evolution of our Generative AI products. Your role will involve instrumenting and collecting system operational data, transforming it into actionable metrics and analyses, and guiding product and business decisions to improve quality, safety, efficiency, and impact. **Your main responsibilities** ------------------------------------- * End-to-end product instrumentation: define events, logs, and telemetry for conversations/queries, ensuring privacy and compliance (LGPD). * Build and maintain data pipelines and semantic layers that consolidate usage, cost, latency, satisfaction, and response quality. * Define and monitor key LLM and product metrics: resolution rate, groundedness/factual accuracy, hallucination rate, RAG coverage, retention, NPS, cost per session, and latency SLA. * Develop offline/online evaluation frameworks: benchmark datasets, evaluation rubrics, human-in-the-loop (HITL) feedback, LLM-based self-evaluation, and regression testing. * Conduct A/B experiments and causal analyses; translate results into clear prioritization recommendations and estimated gains. * Create dashboards and alerting systems for product, engineering, and business stakeholders. * Collaborate with PMs to improve prompts, agent architectures, RAG, and knowledge bases. * Support RAG governance: source curation, chunking/embedding, deduplication, and versioning. * Monitor and optimize efficiency: model selection/versioning, context size, caching, cost, and latency. Map root causes of errors and structure taxonomies to guide roadmap planning and risk mitigation (security, toxicity...) * **Essential Requirements:** -------------------------- * Bachelor’s degree in Engineering, Computer Science, Mathematics, Statistics, or related fields. * 5+ years of experience in ML/Data/Analytics applied to digital products. * Experience with Generative AI products (LLMs) and their metrics; experience with OpenAI/Azure/Open Source or equivalent providers. * Strong proficiency in Python and SQL. * Experience with event instrumentation and tracking (e.g.: Segment, Amplitude, Mixpanel, GA4) and pipelines/orchestration (e.g.: Airflow, Prefect, dbt). * Practical experience in experimentation and evaluation: A/B testing, experimental design, and statistical analysis. * Experience creating dashboards (e.g.: Looker, Metabase, Power BI, Databricks SQL) and executive reports. * Knowledge of RAG, embeddings, and prompt/dataset versioning. * Clear communication, product mindset, and impact orientation. **How you can impress us** ---------------------------------- * LLM observability (e.g.: Langfuse, Helicone, Arize, WhyLabs, Evidently, MLflow, Weights & Biases). * Tools/foundations of LLM apps (LangChain, LlamaIndex), vectors (Pinecone, Weaviate, FAISS, Milvus). * Tracing and monitoring (OpenTelemetry, Datadog) and data governance/security (LGPD, PII redaction). * LLM cost/performance optimization and guardrail/moderation design. * Knowledge of financial markets/investments. * English/Spanish for collaboration and technical reading. **Benefits** **Benefits that make a difference in your journey with us!** Here, employee experience is taken seriously—from well-being to career development, covering important life moments. See what awaits you: **Meal and Food Voucher (Pluxee)** Flexible use so you can manage it your way: supermarket, restaurant, delivery… you choose! **Health Insurance** High-quality plans with Bradesco or Unimed Nacional, with no co-payment. Because good health should be accessible. **Dental Plan (Odontoprev)** Take care of your smile with peace of mind and full coverage! **Pet Health (Doglife)** Your pet is part of the family too! We offer partnerships for exclusive pet health plans. **Mental Health & Well-being** Free therapy sessions through Wellz and access to gyms and online classes via Wellhub. Body and mind in balance! **Continuous Development** Free access to LinkedIn Learning and partnerships with schools such as Open English and The English Academy. As a fintech, we encourage innovation through Rockeseat Santander Open Academy. **Childcare Assistance** Supporting those who care for those who need it most. **Inclusion & Diversity** * 6-month maternity leave and 20-day paternity leave * Lifetime childcare assistance for children with disabilities * Inclusive onboarding for people with disabilities * Reimbursement for materials and peripherals ensuring accessibility **Birthday Day Off** A full day just for you, to enjoy however you wish! **Referral Program (TalenToro)** Referred someone with our DNA? You earn from it! **Gamification Platform** On our #Valorize culture platform, you can accumulate **Torocoins to exchange for education vouchers or redeem amazing rewards in our store.** Achievements + engagement = prizes and unique experiences **Certification Reimbursement** We cover the cost of mandatory certifications for your role. We invest in your growth! **Home Office Allowance** Reimbursement for chair, desk, and peripherals to ensure comfort and productivity during your remote work. **Profit Sharing (PLR)** Your effort is recognized—and valued! **THE BEST COMBINATION OF PEOPLE AND TECHNOLOGY ❤️** -------------------------------------------------- **We are one of the largest investment fintechs in the world!** We are part of **Santander**, the largest bank in the Eurozone, and we double in size every year. Our purpose is to **generate financial freedom for people**, serving as the bridge between them and their goals! That's why we dream big and pursue continuous innovation to help them find the best investment opportunities in the market and invest easily, safely, and with true Zero Brokerage. To achieve this, we combine people and technology, valuing and preserving a collaborative, diverse, and innovative spirit. Every team member is essential to our development and success. **❤️ Our culture:** We divide and materialize our daily lived culture into three core values: **Ownership Mindset:** We are protagonists who strive to make a difference. We take responsibility and get hands-on, thinking about the company as a whole. We deliver results by doing more with less and valuing meritocracy. **Team Vision:** We are a collaborative company, spread across Brazil. We work together transparently, humbly, and respectfully. We practice autonomy with alignment and always act in favor of the company’s and customer’s greatest objectives. **Customer Focus:** Our goal is to provide our customers with the best opportunities, happiness, and achievement of their goals. We deliver value through simple solutions and a genuine passion for service. We innovate by learning quickly and without fear of failure. **Our commitment to diversity:** Toro respects the plurality of identities and strives to promote an inclusive culture. We know there’s still much to do, which is why we seek people who want to dream this dream with us. Do you relate to this? Then join us in this constant evolution and **#JoinToro** ❤️
