Principal Cloud Engineer - AI Platform

Buildertrend is hiring a Principal Cloud Engineer to shape and own the secure, reliable cloud foundation for production AI agents. The role leads platform architecture and standards, enabling engineering teams to deploy and scale AI capabilities safely while protecting data and managing performance and cost.

Responsibilities:

• Design and maintain the infrastructure behind production AI agents, including workload identity, ingress and egress gateways, network isolation and platform-level sensitive data redaction
• Lead decisions on Google Cloud Platform (GCP) networking, private connectivity, zero-trust egress and service perimeters to protect workloads without limiting the platform’s ability to grow
• Develop reusable Terraform modules and CI/CD pipelines that help product teams deploy safely and independently, including temporary environments for validating changes before merge
• Establish a centralized, approved path for large language models (LLM) access, including routing, virtual keys, provider onboarding, observability and cost attribution
• Document platform and vendor decisions with total cost of ownership, barriers to entry, trade-offs and open questions clearly defined so teams can make informed decisions and build on a consistent foundation
• Serve as a technical counterpart to major cloud and AI providers, escalate complex issues and validate roadmap commitments before they influence platform architecture
• Take incident command during critical production issues, make informed go/no-go decisions under pressure and communicate risks and status clearly to technical leaders and executives
• Implement safeguards for credentials, secrets and AI usage, including automated secrets rotation and policy-as-code controls that reduce the risk of unauthorized AI deployments
• Lead blameless postmortems, identify systemic weaknesses and turn incident findings into measurable improvements in platform security, resilience and operational readiness
• Embrace using AI as a tool to enhance your work, while remaining open to learning, applying critical thinking, and owning the quality of your final work

Requirements:

• 10+ years of experience architecting enterprise-grade GCP environments, with advanced knowledge of GCP-native networking, including VPC Service Controls and Private Service Connect, as well as IAM perimeters and Workload Identity
• Previous experience at a B2B SaaS company
• Strong infrastructure as code practice
• Hands-on experience running AI or ML workloads in production, such as model serving, agent runtimes or inference gateways, and understand how their needs differ from traditional web applications
• Practical depth in workload identity and machine-to-machine authorization, including service accounts, short-lived credentials, certificate-based identity, and token delegation, and the challenges of on-behalf-of authorization flows
• Track record-owning production incidents, including calling deployments under pressure and updating executives during an outage
• Experience with agent development frameworks, agent runtime platforms and the emerging tooling around them
• Approach incidents without blame, communicate directly, take ownership of platform outcomes and turn lessons learned into practical improvements that help other engineers deliver reliable software
• Experience with automated secrets rotation, policy as code and governance that limits unauthorized “Shadow AI” deployments, with an eye on the new attack surfaces AI creates