Machine Learning QA

We are seeking a skilled Machine Learning QA Engineer to bridge the gap between traditional software testing and artificial intelligence. In this role, you will ensure the quality, reliability, and fairness of our ML models and cloud-based applications. You will take ownership of the end-to-end testing lifecycle—from validating data pipelines to tracking production performance—while working within an Agile development framework.

‍

JOB DESCRIPTION

Data & Model Validation

• Verify input data quality to ensure it is clean, relevant, and representative of real-world scenarios.
• Define evaluation metrics tailored to specific problem domains, aligning them directly with project objectives and stakeholder requirements.
• Implement cross-validation techniques to robustly assess model performance and compare outputs against expected benchmarks.
• Evaluate models for bias to detect and mitigate unfair outcomes for protected groups or specific demographics.

Test Planning & Execution

• Translate product requirements (functional, non-functional, and project schedules) into comprehensive test plans, designs, and test cases.
• Create and execute tests across both manual and automated testing frameworks.
• Perform regression testing to guarantee that new builds or model retraining do not introduce defects.
• Estimate, plan, and prioritize all QA activities to match sprint cycles.

Systems & Cloud Operations

• Debug and troubleshoot live, cloud-based applications to improve system reliability.
• Track quality assurance metrics (e.g., defect density, test coverage, model drift) to maintain high performance.
• Write detailed test reports to communicate system readiness and limitations clearly to project stakeholders.

Collaboration & Process Ownership

• Establish QA best practices and set up robust processes in close alignment with project stakeholders.
• Foster cross-functional collaboration between data scientists, engineers, domain experts, and business teams.
• Participate as an integral member of our Agile software development lifecycle (SDLC).

‍

REQUIREMENTS

Technical Skills

• Strong experience testing Machine Learning/AI pipelines and data-driven systems.
• Proficiency in data validation, statistical analysis, and evaluating model metrics (e.g., F1-score, ROC-AUC, MAE).
• Experience with automation tools and scripting languages (e.g., Python, SQL).
• Familiarity with cloud platforms (AWS, GCP, or Azure) and debugging live cloud applications.
• Solid understanding of traditional software QA methodologies, test case design, and regression testing.

Soft Skills & Methodologies

• Experience working in an Agile/Scrum environment.
• Strong communication skills to explain complex ML limitations or test metrics to non-technical stakeholders.
• Sharp analytical mindset with an eye for detecting bias or anomalies in data.

Experience & Education

• 8 - 14 years of experience in Quality Assurance, Machine Learning, or related fields.
• Bachelor’s degree in computer science, Engineering, or a related discipline (or equivalent practical experience).

‍

BeyondAI is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. We do not discriminate based on race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, or any other legally protected status.