Описание:
Finom is a European tech startup headquartered in Amsterdam that develops an all-in-one financial B2B solution for entrepreneurs. Its mobile-first platform integrates banking functions, accounting, financial management, invoicing, and AI-enabled accounting, and operates across key EU markets.
Задачи:
- Own and extend the offline evaluation suite across products, including datasets, judges, and metrics;
- Build and maintain online quality dashboards covering resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, and latency;
- Mine failure patterns from real traffic, convert them into regression cases, and propose fixes to Product and domain experts;
- Harden evaluation methodology through judge stability and non-determinism handling;
- Translate metrics into decisions through weekly syncs and clear trade-offs;
- Collaborate closely with AI engineers, Product, and domain experts across the company.
Требования:
- Build end-to-end analyses with Python and SQL;
- Have a solid foundation in statistics, including sampling, hypothesis testing, variance, and noisy metrics;
- Start from the business question rather than the tool;
- Have 3+ years of experience in analyst or data scientist roles, including at least one year in a product context;
- Be curious about and fluent with AI-assisted coding, or genuinely excited to become fluent quickly;
- Nice to have: experience in quality analytics for ML systems such as ranking, recommendations, or classification;
- hands-on experience evaluating LLM applications including RAG, agents, tool use, or judges;
- experience building LLM agents through side projects, toy builds, or personal experiments.
Условия:
- Remote or hybrid work and travel across Europe are available;
- Stock Options Program available to every team member;
- Resources and opportunities for continuous personal and professional development;
- Work & Swim Program with one month in a corporate apartment in Cyprus;
- Equal opportunity employer with support for employee well-being.
