AI Quality Engineer New

Description

Job Summary: An AI Quality Analyst/Engineer ensures high reliability, security, and accuracy across all Artificial Intelligence solutions by evaluating models and building testing pipelines. Key Highlights: 1. Act as a guardian of the quality of AI models and intelligent systems. 2. Create and execute AI-specific tests to evaluate accuracy and behavior. 3. Build automated testing pipelines for autonomous agents and AI APIs. NEORIS, now part of EPAM, is a Digital Accelerator helping enterprises take their next step toward the future. With over 20 years of experience as digital partners to some of the world’s largest companies, we are more than 4,000 professionals across 11 countries, fostering a multicultural, startup-like culture that cultivates innovation and continuous learning to deliver high-value solutions for our clients. We believe diversity drives innovation. Therefore, all our opportunities are open to everyone, regardless of gender, race, age, identity, orientation, or condition. **Location:** Rio de Janeiro **Work Model:** On-site **Employment Type:** CLT + Benefits **Job Description** We are seeking an AI Quality Analyst/Engineer to serve as a guardian of the quality of our AI models, agents, and intelligent systems—ensuring high reliability, security, and accuracy across all company AI solutions. **Responsibilities:** * Create and execute AI-specific tests to evaluate model accuracy, regression, and behavior. * Validate security aspects, biases, hallucinations, and consistency in Large Language Models (LLMs). * Build automated testing pipelines for autonomous agents, AI APIs, and models. * Design realistic usage scenarios and benchmarks for solution validation. * Collaborate with technical teams to identify and resolve defects, errors, and risks. * Define quality metrics and dashboards for continuous monitoring of AI solutions. * Ensure compliance with security policies, privacy regulations, and internal standards. **Requirements:** * Experience with automated testing and system validation. * Basic knowledge of Machine Learning / Artificial Intelligence for model evaluation. * Proficiency in Python, testing tools, and data analysis. * Ability to clearly and objectively document and communicate results. **We Offer:** * Meal Voucher and Food Allowance: Monthly benefits for food-related expenses, both at home and outside. * Private Pension Plan: Future planning with supplementary retirement contributions. * Health Insurance: High-quality medical coverage for you and your legal dependents. * Dental Insurance: Comprehensive dental care with an extensive provider network. * Life Insurance: Protection and security for you and your loved ones. * English/Spanish Classes: Language learning incentive. * Wellhub: Benefits and discounts for gyms and wellness programs. * Daycare Assistance: Financial support for early childhood education expenses. * Neoris Global Campus: A development platform offering continuous learning and professional growth opportunities. We invite you to learn more about us at http://www.neoris.com, Facebook, LinkedIn, Twitter, or Instagram: @NEORIS. #LI-TL1