AI Quality Engineer New

Description

Job Summary: AI Quality Analyst/Engineer to ensure high reliability, security, and accuracy across all the company's Artificial Intelligence solutions. Key Highlights: 1. Create and execute AI-specific tests and validate LLMs. 2. Collaborate with technical teams to identify and resolve failures and risks. 3. Define quality metrics and dashboards for continuous AI monitoring. NEORIS, now part of EPAM, is a Digital Accelerator helping enterprises take their next step toward the future. With over 20 years of experience as digital partners to some of the world's largest companies, we are more than 4,000 professionals across 11 countries, fostering a multicultural, startup-style culture that cultivates innovation and continuous learning to deliver high-value solutions for our clients. We believe diversity drives innovation. Therefore, all our opportunities are open to everyone, regardless of gender, race, age, identity, orientation, or condition. **Location:** Rio de Janeiro **Work Model:** On-site **Employment Type:** CLT + Benefits **Job Description** We are seeking an AI Quality Analyst/Engineer to serve as the guardian of quality for our models, agents, and intelligent systems, ensuring high reliability, security, and accuracy across all the company's Artificial Intelligence solutions. **Responsibilities:** * Design and execute AI-specific tests, evaluating models for accuracy, regression, and behavior. * Validate security aspects, biases, hallucinations, and consistency in LLMs (Large Language Models). * Build automated testing pipelines for autonomous agents, AI APIs, and models. * Develop realistic usage scenarios and benchmarks to validate solutions. * Collaborate closely with technical teams to identify and resolve failures, errors, and risks. * Define quality metrics and dashboards for continuous monitoring of AI solutions. * Ensure compliance with security policies, privacy regulations, and internal standards. **Requirements:** * Experience with automated testing and system validation. * Basic knowledge of Machine Learning/Artificial Intelligence for model evaluation. * Proficiency in Python, testing tools, and data analysis. * Ability to document and communicate results clearly and objectively. **We Offer:** * Meal Allowance and Food Allowance: Monthly benefits for food-related expenses, both at home and outside. * Private Pension Plan: Future planning with supplementary retirement contributions. * Health Insurance: High-quality medical coverage for you and your legal dependents. * Dental Insurance: Comprehensive dental care with an extensive provider network. * Life Insurance: Protection and security for you and your loved ones. * English/Spanish Classes: Language learning incentives. * Wellhub: Benefits and discounts at gyms and wellness programs. * Childcare Assistance: Financial support for early childhood education expenses. * Neoris Global Campus: A development platform offering continuous learning and professional growth opportunities. We invite you to learn more about us at http://www.neoris.com, Facebook, LinkedIn, Twitter, and Instagram: @NEORIS. #LI-TL1