AI Test Lead
Pay Rate Range: $140.00 - $166.66/hr.
JOB DESCRIPTION
Role Overview
We are seeking a forward-thinking AI Platform Test Lead to design, build, and operationalize the QE testing leveraging AI tools and utilities to accelerate innovation in IT's Software Development Lifecycle. This role is W2 hourly with Roth Staffing, with all work taking place for our client in Alameda, CA.
This role will lead the design, implementation and operations of AI QE testing platforms, tools and utilities to support a multi-platform AI ecosystem, spanning AWS (Bedrock, SageMaker), Databricks (Mosaic AI), Claude for Enterprise, and emerging AI-native / agentic engineering tools and platforms. The AI QE test platforms will also be used with non-AI software products. This role will ensure scalable, secure, and compliant deployment of AI capabilities while enabling rapid experimentation and adoption.
A critical success factor is the ability to stay ahead of industry trends, quickly validate new technologies, and operationalize high-value capabilities in a regulated life sciences environment.
Key Responsibilities
AI QE Test Platform Strategy & Architecture
o Help define and mature Exelixis' enterprise AI QE Test platform architecture across cloud and data ecosystems
o Design interoperable AI driven QE test solutions to support software solutions to support
o AWS AI stack (Bedrock, SageMaker, model hosting, orchestration)
o Databricks / Mosaic AI (ML lifecycle, feature engineering, LLM ops)
o Claude for Enterprise (secure conversational AI and enterprise workflows)
o SaaS and in-house developed software products
AI Capability Engineering & Operations
o Operationalize reusable AI capabilities:
Prompt, tool, and agent orchestration frameworks
Evaluation, monitoring, and observability pipelines
Enable secure, compliant AI usage (GxP, HIPAA where applicable)
o Implement AI platform guardrails
o Auditability and traceability
Design and operationalize Defect Statistics
Drive adoption of agentic software development lifecycle (SDLC) practices
Define frameworks for:
o Spec-driven agentic development (Claude Code, Github Copilot, code agents)
o Autonomosemiautonomous agents across workflows
Integrate AI-native platforms into enterprise engineering workflows (CI/CD, DevSecOps)
Required Qualifications
5+ years in software quality engineering and testing, AI/ML engineering
3+ years hands-on experience with AI Test platforms (AWS preferred)
Proven experience with:
o Experience with one or more AI QE Testing Platforms: Tricentis Testim/Tosca, ACCELQ, Mabl, LambdaTest, Katalon
o Enterprise LLM platforms (e.g., Claude, OpenAI, or similar)
o Strong understanding of LLM architectures (RAG, fine-tuning, embeddings, Vector DBs, Graph DBs, Multi agent orchestration)
Preferred Qualifications
Familiarity with:
o GxP validation processes for AI/ML systems
Exposure to:
o Agent frameworks (LangChain, Semantic Kernel, etc.)
o AI testing and evaluation tooling
o Multi-cloud / hybrid architectures
Key Competencies
Strategic + hands-on balance (thinks like an architect, executes like an engineer)
Ability to translate emerging AI trends into enterprise value
Strong systems thinking across platforms, data, and workflows
Excellent stakeholder communication-able to influence senior leadership and engineering teams alike
Bias for action-rapid experimentation and iterative delivery
TECHNICAL SKILLS
Must Have
All qualified applicants will receive consideration for employment without regard to race, color, national origin, age, ancestry, religion, sex, sexual orientation, gender identity, gender expression, marital status, disability, medical condition, genetic information, pregnancy, or military or veteran status. We consider all qualified applicants, including those with criminal histories, in a manner consistent with state and local laws, including the California Fair Chance Act, City of Los Angeles' Fair Chance Initiative for Hiring Ordinance, Los Angeles County Fair Chance Ordinance, and San Francisco Fair Chance Ordinance.
