Staff Software Engineer, Artificial Intelligence/LLM

Beacon AI is an AI platform company building systems to make flying safer, more efficient, and more capable. The Staff Software Engineer will ship LLM-powered product features end-to-end, lead technical direction across services and teams, and build reliable retrieval, tool-calling, evaluation, safety, and production operations capabilities for aviation systems.

Responsibilities:

• Design and implement retrieval-augmented generation and tool-calling flows using frameworks like LangChain or equivalent primitives, where simpler is better
• Deliver robust JSON and schema-bound outputs with validation, retries, and fallbacks
• Add function calling to integrate with internal tools, search, routing, and data services
• Ship APIs and workers in Python or TypeScript with clear contracts, streaming, and backoff
• Add caching, request shaping, prompt templates, and context packing to control latency and cost
• Integrate with AWS Bedrock, OpenAI, Anthropic, or self-hosted endpoints as needed
• Collaborate with infrastructure teammates to develop chunking, embeddings, and indexing capabilities for documents, time series, and multimedia
• Choose and tune vector backends such as OpenSearch, pgvector, or Pinecone
• Keep knowledge bases fresh with data syncs from S3, Aurora, DynamoDB, and external sources
• Create offline evals and golden sets for prompts, retrievers, and tools
• Stand up online metrics for task success, hallucination rate, retrieval precision/recall, p95 latency, and cost per request
• Run A/B tests and prompt/version rollouts with guardrails and canaries
• Implement content and policy checks, PII detection and redaction, access controls, and auditing
• Design human-in-the-loop paths for sensitive actions
• Handle aviation data with care and follow internal security standards
• Add tracing, logs, and dashboards for model calls, token usage, errors, and saturation
• Debug tricky failures across retrieval, prompts, tools, and providers