AI Operations Engineer

ClickHouse is a globally distributed company whose IT Operations team manages internal systems, tooling, and infrastructure securely and efficiently. The AI Operations Engineer will help establish a new AI Ops function by designing, deploying, integrating, evaluating, and maintaining AI-driven solutions and agents across business and engineering teams, while supporting cost visibility, governance, and organization-wide AI training.

Responsibilities:

• Develop and deploy solutions using LLMs, automation frameworks, and internal tooling, for both business and engineering use cases
• Integrate AI into existing systems (HRIS, ATS, ERP, procurement orchestration, ticketing, CI/CD, developer tooling) and legal infrastructure like CLM or Matter Management
• Build workflows such as document generation, data extraction, knowledge retrieval, and decision support across systems and knowledge bases
• Use modern integration standards (e.g., MCP) to give AI models and agents direct, secure access to internal knowledge and context
• Help test and benchmark AI models against accuracy, latency, cost, and safety criteria
• Support model risk assessments and due diligence alongside Security and GRC
• Help implement and maintain the internal model routing solution
• Help build dashboards and reporting for AI/LLM spend across teams and tools
• Implement tagging, monitoring, and alerting for cost anomalies
• Identify and execute on cost-optimization opportunities (prompt efficiency, caching, model right-sizing)
• Help provision, credential, and de-provision AI agent access
• Support deployment, versioning, monitoring, and retirement of agents
• Maintain and patch agent infrastructure and dependencies
• Help maintain audit trails and logging for agent actions
• Design and deliver company-wide AI training programs tailored to different audiences — non-technical business users through engineers
• Create lightweight playbooks, guides, templates, and reusable components for safe, effective AI adoption
• Run onboarding sessions, office hours, and workshops to build AI fluency and drive adoption of approved tools and agents
• Establish a feedback loop with users to continuously improve training content, tooling, and documentation