Job Summary: We are seeking a professional to evaluate, monitor, and enhance conversational agents on WhatsApp, ensuring quality, accuracy, and alignment with the customer experience. Key Highlights: 1. Work with data analysis, qualitative reading, and critical thinking. 2. Evaluate and monitor conversational agents to ensure outstanding interactions. 3. Collaborate closely with data, technology, product, and design teams. **About the Role** We are the WhatsApp team at the bank, responsible for one of the most strategically important customer engagement channels. Our mission is to simplify people’s financial lives with simple, fast, secure, and increasingly intelligent solutions—exactly when they matter most. In an era of accelerated digital transformation, Itaú reaffirms its commitment to being where our customers are. With millions of monthly accesses and presence on over 98% of Brazilian mobile phones, WhatsApp has become a critical platform for customer experience, relationship building, and operational efficiency at the bank. Within this context, we seek a professional to evaluate, monitor, and evolve conversational agents on the WhatsApp channel, ensuring quality, accuracy, and alignment with the Itaú customer experience. This role combines data analysis, qualitative reading of conversations, critical thinking about consumer behavior, and evaluation of conversational models—with strong emphasis on real-life interactions between customers and AI agents. The role is fundamental to reducing errors, hallucinations, conversational friction, and ensuring outstanding, secure, and brand-aligned interactions. You will work closely with data, technology, product, and design teams, supporting evidence-based decisions and continuous improvements. **Main Responsibilities** * Evaluate customer-to-conversational-agent interactions on the WhatsApp channel, analyzing response quality, correctness, clarity, and appropriateness. * Curate conversational models by identifying flaws such as incorrect responses, hallucinations, contextual inconsistencies, and comprehension issues. * Process and analyze large volumes of conversational data using SQL (primary tool) and Python as a plus. * Define and apply analytical segmentation, statistical sampling, confidence levels, and conversation evaluation criteria. * Conduct quantitative and qualitative analyses linking metrics, conversational context, and customer behavior. * Identify problematic conversations at scale using data and indicators to guide curation efforts. * Build and track conversational agent health metrics, identifying error patterns, pain points, friction areas, and improvement opportunities. * Translate analyses and findings into actionable insights to improve models, content, business rules, and customer experience. * Partner with technical and business teams—including data scientists and data analysts—to support data-driven prioritization and decision-making. * Document analyses, hypotheses, learnings, and recommendations to ensure traceability and continuous curation evolution. **Profile We Seek** * Strong analytical and data-driven mindset, capable of structuring analyses from large datasets. * Inquisitive, questioning, and critically thinking profile. * Strong qualitative reading ability and critical judgment to interpret real conversations—going beyond metrics. * Ability to connect data, context, and consumer behavior. * Organized, autonomous, and clear in communicating results. * Genuine interest in customer experience, language, and applied AI. **Requirements** * Completed undergraduate degree. * Prior experience with data analysis applied to digital products, customer service, CX, or conversational channels. * Solid SQL knowledge for data extraction, processing, and analysis. * Foundational knowledge of applied statistics (sampling, confidence levels, metric interpretation). * Knowledge of consumer behavior, customer experience, or digital journeys. * Experience with marketing analytics, consumer behavior analysis, social listening, or digital interactions. * Experience with conversational models, chatbots, or virtual assistants. **Preferred Qualifications** * Experience identifying model errors such as hallucinations, off-context responses, or comprehension failures. * Knowledge or experience with generative AI, LLMs, or model curation. * Python usage for analysis, automation, or data exploration. * Prior experience with messaging channels (WhatsApp, chat, virtual assistants). **Benefits** * Meal Allowance; * Transportation Allowance; * Food Allowance; * Daycare Assistance; * Medical Assistance; * Dental Assistance; * Life Insurance; * Private Pension; * Wellhub and Totalpass partnerships; * Univers partnership (Drogasil and Droga Raia); * Visits, reservations, and accommodation at Itaú Unibanco Club (Guarapiranga, Itanhaém, and São Sebastião); * IUPP Program (benefits and discounts with various partners across Brazil and Latin America); * Fique OK Program; * Corporate University; * Discounts on financial products and services; * And much more! We are undergoing constant transformation—and we look forward to having you contribute to this journey! \#JoinItaú Itaú will use the information provided for recruitment processes, including job databases hosted on third-party platforms. If you wish to object to this use or exercise any of your rights, please contact us via the channels available in Itaú Unibanco’s Candidate Privacy Policy (https://www.itau.com.br/privacidade/candidatos/). No consent is required—it is merely informational. Candidate Privacy Policy \| Itaú If you are applying for a position within the Itaú Group, learn which information is being used and review our privacy policy.
João Silva
Indeed · HR
João Silva
Indeed · HR
