Key skills
PythonAIMLLLMRAGAgenticA/B Testing
About this role
Role Overview
- Build and maintain the agent harness
- Own the evaluation stack
- Do prompt and behaviour engineering and retrieval/RAG at scale
- Host and improve our own small models
- Work closely with Product teams and domain experts
Requirements
- 5-8 years of experience
- Very strong in Python
- Hands-on experience building LLM and agentic systems at scale in production
- Evaluation as a first-class discipline: golden datasets, LLM-as-judge, human eval, A/B testing
- Good grasp of applied LLM/ML and AI infrastructure
- Exposure to model fine-tuning and post-training (SFT, DPO, RL)
- Balanced blend of technical, business, and product skills
- Fluent in English (French is not mandatory)
Tech Stack
- Python
Benefits
- 25 vacation days paid by Pennylane
- Competitive compensation package
- Company shares to enjoy a piece of the success story
- Budget to turn your home into a more comfortable workspace
- Monthly allowance to work from a coworking space
- Access to 8,000 fitness spaces in Europe and more than 300 wellness activities
- Access to Busuu to perfect your English or French
- Latest Apple equipment
- Possibility to work remotely from your country of residence in Europe
- Company events such as Tech Days and annual company seminar