Staff Site Reliability Engineer

Sight Machine is a company that strengthens manufacturers by providing a standard data model and system-level visualization capabilities. The Staff Site Reliability Engineer will join the Cloud Infrastructure Team to drive reliability, automation, and scalability, while mentoring engineers and addressing complex systems problems.

Responsibilities:

• Champion an agentic-AI-first engineering mindset: identify where AI-driven automation and agent-based tooling can replace manual toil, and hold that work to the same quality, testing, and reliability bar as any other production system
• Evolve reliability practices for meeting reliability SLO’s, error budgets, drive incident postmortems to systemic (not just symptomatic) fixes, and lead reliability reviews for new services before they hit production
• Troubleshoot and resolve the org's most complex, cross-layer systems problems CI/CD, container orchestration, networking, OS, cloud resources, databases, and increasingly, agentic AI/LLM orchestration layers
• Design, build, and operate the infrastructure supporting agentic AI workloads, LLM gateway routing, agent orchestration frameworks, monitoring of non-deterministic/AI-driven services, and the operational tooling needed to run them reliably at scale
• Architect and instrument monitoring, alerting, and observability infrastructure for critical services, with an eye toward what 'critical' means for AI-driven systems specifically
• Author and continuously improve operational runbooks and automation, increasingly incorporating agentic/AI-assisted tooling (e.g., automated triage, AI-assisted incident response) where it measurably reduces toil
• Design and build internal platforms and developer tooling that other engineers build on top of
• Participate in on-call coverage and help evolve the program as we scale including escalation paths and reducing avoidable pages through better automation
• Bring a startup mindset of daily engagement: staying close to what's breaking, what customers are hitting, and where the team needs help, even outside a formal ticket or rotation
• Mentor senior and mid-level engineers; act as a technical sounding board across teams
• Proactively identify and drive cross-team initiatives that improve stability, reliability, and availability, this is expected to be self-directed, not assigned

Requirements:

• Demonstrated experience designing, building, or operating agentic AI/LLM-based systems in production, held to the same quality-first, test-driven rigor as traditional infrastructure code, not just prototype-grade work
• Embody a quality-first and security-first culture in all that you do
• 10+ years of experience with Kubernetes/Docker in at least one top-tier cloud provider (Azure, GCP, AWS), including production-scale multi-tenant or multi-cluster environments
• 10+ years coding experience (Python, Go, Java, or similar) with a track record of building tools/platforms used by other engineers, not just scripts
• 10+ years with IaC and CI/CD tooling (Terraform/OpenTofu, FluxCD or similar GitOps tooling, Jenkins/GitHub Actions)
• Strong, provable Linux and networking fundamentals (TCP/IP and application-layer)
• Practical experience integrating or operating LLM/agentic AI systems in a production context this can be API-based orchestration, LLM gateways, or agent frameworks
• A track record of authoring technical documentation (design docs, ADRs, runbooks) that other engineers actually use
• Demonstrated mentorship of other engineers, without needing formal management authority to do it
• Strong bias for action over endless planning, hands-on, has made mistakes, learned from them, and can weigh risk vs. customer impact under pressure
• Clear, empathetic communicator, comfortable pushing back on architecture decisions across teams
• Operational experience with monitoring/alerting systems (Prometheus, Grafana, Loki, Sentry, Signoz or equivalents)
• Deep understanding of cloud performance, able to diagnose and resolve bottlenecks others can't
• Experience with elements of our current tech stack are a plus: Kubernetes, FluxCD, Terraform, Helm Charts, Prometheus, Elasticsearch, Python, Java, Kafka, Postgres, and Jenkins
• Previous experience or a keen interest in industrial IoT, analytics, or manufacturing a plus