Senior Site Reliability Engineer, Infrastructure Foundations

The Wikimedia Foundation is looking for a Senior Site Reliability Engineer to support and develop the platform serving Wikipedia to millions globally. The role involves operational tasks on public-facing infrastructure, leading continuous improvement through automation, and collaborating with product teams to enhance service scalability.

Responsibilities:

• Performing day-to-day operational/DevOps tasks on Wikimedia’s public facing infrastructure (deployment, maintenance, configuration, troubleshooting)
• Implementing and utilizing configuration management and deployment tools (Puppet, Kubernetes)
• Leading continuous improvement, by automating the installation, configuration and maintenance of services on our platform
• Work closely with product teams helping them bring scalable functionality to our users by assisting in the architectural design of new services and making them operate at scale
• Participating in a 24/7 on-call rotation shared across the broader SRE team. This includes taking part in incident response, diagnosis and follow-up on system outages or alerts across Wikimedia’s production infrastructure
• Collaborating with a global, cross-functional team in an asynchronous communication environment
• Mentoring peers in your areas of technical and operational strength
• Ability and willingness to travel 1-2 times a year for in-person events and team meetings

Requirements:

• 6+ years of experience in an SRE/Operations/DevOps role as part of a team
• Experience with shell and any scripting languages used in an SRE context (Python, Go, Bash, Ruby; we primarily use Python) and configuration management tools (Puppet, Ansible; we use Puppet)
• Experience designing and managing infrastructure security for large fleets of diverse services
• Experience with technical response during security incidents
• Experience with package management on Linux systems (we use Debian)
• Strong Linux system-level troubleshooting skills
• History of automating tasks and processes, identifying process gaps, and finding automation opportunities
• Strong English language skills (verbal and written) and ability to work independently, as an effective part of a globally distributed team working across multiple time zones
• Experience leading and participating in incident response and post-incident review rituals, with the goal of conducting root cause analysis and implementing preventive measures
• Performing day-to-day operational/DevOps tasks on Wikimedia's public facing infrastructure (deployment, maintenance, configuration, troubleshooting)
• Implementing and utilizing configuration management and deployment tools (Puppet, Kubernetes)
• Leading continuous improvement, by automating the installation, configuration and maintenance of services on our platform
• Work closely with product teams helping them bring scalable functionality to our users by assisting in the architectural design of new services and making them operate at scale
• Participating in a 24/7 on-call rotation shared across the broader SRE team. This includes taking part in incident response, diagnosis and follow-up on system outages or alerts across Wikimedia's production infrastructure
• Collaborating with a global, cross-functional team in an asynchronous communication environment
• Mentoring peers in your areas of technical and operational strength
• Ability and willingness to travel 1-2 times a year for in-person events and team meetings
• Experience setting and implementing fleet-wide security policies
• Experience with software supply chain security
• Awareness of the current open source infrastructure security landscape
• Experience working together with software security teams
• Experience with credential management systems
• Experience implementing immutable logging and auditing
• Experience with the use, maintenance and configuration of monitoring, metrics and logging infrastructure (Prometheus, Grafana, etc.)
• Developing/contributing to Free and Open Source software, or being part of an open-source community (share your favourite pull requests!)
• Experience with LAMP stack technologies (PHP/HHVM, memcached/Redis) -- MediaWiki experience is a definite plus
• Experience with defining cross-team SLOs and their implementation