Product Data Scientist — AI Evaluation & Quality (Remote)

Das ist der Job

We embrace diversity and invite applications from all walks of life.

Darum lohnt es sich

Hands‑on experience evaluating LLM applications (RAG, agents, tool use, judges) Experience building LLM agents – side projects, toy builds, personal experiments all count How We Work AI‑assisted coding is our default authoring environment, not a bonus Claude Code is our main tool – you’ll reach for it for SQL, Python, analyses, dashboards, and internal scripts We’re looking for analysts who are already curious and fluent with AI coding — or genuinely excited to become fluent fast We care about what you ship and how clearly you think If this idea excites you rather than worries you, you’ll feel at home Benefits Make a genuine impact on the product and grow with us Work in the EU – flexibility of traveling, working remotely, or hybrid across Europe Become a stock options holder – program available to all team members Support and care – modern, friendly, eco‑conscious corporate culture, continuous support Work & Swim program – one month in a corporate apartment in Cyprus with Mediterranean views Equal Opportunity Statement At Finom, we’re an equal opportunity employer and value diversity at our company.

These tools assist our recruitment team but do not replace human judgment. Company Overview Finom is a European tech startup headquartered in Amsterdam, on a mission to develop an all‑in‑one financial B2B solution that integrates banking, accounting, financial management, and invoicing into a mobile‑first platform.

What You Will Be Doing Own and extend our offline evaluation suite across products — datasets (capability + regression), judges, metrics Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM‑as‑judge signals, error rate, latency Close the production feedback loop: mine failure patterns from real traffic → turn them into regression cases → propose fixes to Product and domain experts Harden methodology: judge stability, non‑determinism handling Translate numbers into decisions – weekly syncs, clear trade‑offs, no dashboards for their own sake Must‑Haves Python and SQL ability to build analysis end‑to‑end Solid foundation in statistics – sampling, hypothesis testing, variance, understanding noisy metrics Analytical mindset – starting from the business question, not the tool 3+ years in analyst / data‑scientist roles, at least one in a product context Nice‑to‑Haves Experience in quality analytics for ML systems – ranking, recommendations, classification, etc.

We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, disability status, or other applicable legally protected characteristics.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. Final hiring decisions are ultimately made by humans.

If you would like more information about how your data is processed, please contact us. #J-18808-Ljbffr