Own and extend our offline eval suite across products — datasets (capability + regression), judges, metrics
Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, latency
Close the production feedback loop: mine failure patterns from real traffic → turn them into regression cases → propose fixes to Product and domain experts
Harden methodology: judge stability, non-determinism handling
Translate numbers into decisions – weekly syncs, clear trade-offs, no dashboards for their own sake

3+ years in analyst / data scientist roles, at least one in a product context
Python and SQL — you can build an analysis end-to-end
Solid foundation in statistics — sampling, hypothesis testing, variance
Analytical mindset — you start from the business question, not from the tool
Experience in quality analytics for ML systems — ranking, recommendations, classification, etc. (Nice-to-Haves)
Hands-on experience evaluating LLM applications (RAG, agents, tool use, judges) (Nice-to-Haves)
Experience building LLM agents — side projects, toy builds, personal experiments all count (Nice-to-Haves)
We're looking for analysts who are already curious and fluent with AI coding — or genuinely excited to become fluent fast

Make a genuine impact on the product
Join our upward trajectory, and grow with us
We provide the resources and opportunities for continuous personal and professional development
Enjoy the flexibility of traveling and working remotely or in a hybrid model across Europe
Become a stock options holder
Unlock your inner entrepreneur through our Stock Options Program
Receive unwavering support and care
We offer constant support and care to ensure your Finom experience is successful and fulfilling
Immerse yourself in our exclusive Work & Swim Program
Spend one month in a comfortable corporate apartment in Cyprus
It's the ideal opportunity to strike the perfect work-life balance

Product Data Scientist – AI Evaluation, Quality

Key skills