Senior Software Engineer, Internal Tools

Syllo is hiring a Senior Software Engineer, Internal Tools, to build and scale internal software that improves productivity across the company. The role focuses on AI-first development, automated testing and evaluation of LLM workflows, and creating tools in partnership with internal teams.

Responsibilities:

• Drive internal tool evolution: Build, maintain, and scale internal software tools, including document/contract parsing engines, interactive Slack applications, and custom operational utilities
• Build with AI-first workflows: Utilize AI coding agents and assistants to supercharge your execution, writing clean, well-architected code while applying strong engineering discernment to agent output
• Pioneer AI-assisted testing: Implement automated testing strategies and evaluation pipelines using AI tooling to ensure test coverage keeps pace with product releases
• Evaluate AI systems: Build scoring frameworks and evaluation harnesses to assess and improve the reliability and accuracy of LLM-backed workflows over time
• Design smart team workflows: Partner with teams across Finance, Legal, Customer Success, and Revenue Operations to engineer tools tailored to their evolving processes
• Help set AI engineering practices: Collaborate on best practices for integrating AI-generated code into the codebase, sharing methodologies that elevate velocity across the wider engineering team

Requirements:

• Daily experience using AI coding agents to accelerate development, paired with a sharp eye for code quality, security, and architecture
• Hands-on experience creating robust automated tests and developing benchmarking tools for non-deterministic model outputs
• Practical knowledge building on LLM APIs, including prompt design, output parsing, performance tuning, and cost management
• Ability to take open-ended business needs and turn them into fully functional, user-friendly internal tools
• Dedication to writing readable, modular, and well-instrumented code that remains easy for others to collaborate on
• Excitement to try new tooling, adapt to changing requirements, and continuously improve how internal tools are delivered
• 5+ years of professional software engineering experience, including ownership of production software systems
• Consistent experience using AI coding agents or assistants as part of a modern software development workflow
• Experience building automated testing workflows or evaluation systems for AI/LLM applications
• Hands-on experience integrating large language model APIs into user-facing applications or internal platforms
• Experience building internal applications, workflow automation, or platform services used daily by non-technical teams
• Proficiency in Python and modern frontend frameworks (e.g., React), alongside practical experience with web APIs and cloud integrations
• Bachelor's degree in computer science or a technical field, or equivalent practical experience
• Experience with document processing, contract workflows, or legal technology is a strong plus