Data Scientist

What you'll work on

• Build and improve LLM-based evaluation models (LLM-as-a-Judge) to assess the quality, safety, and effectiveness of AI products (e.g., AI assistant outputs, agent copilots, support automation).
• Help design and implement automated QA and evaluation pipelines that enable continuous monitoring and benchmarking of AI product performance at scale.
• Contribute to end-to-end model development, from problem framing and data exploration to deployment and monitoring, with guidance and support from the team.
• Own the forecasting models and work with WFM to improve the accuracy.
• Develop and maintain data pipelines that combine product signals, customer support interactions, and feedback loops to support reliable evaluations and insights.
• Support the creation of qualitative and quantitative observability frameworks, including rubrics, evaluation criteria, labeling strategies, and error categories to understand model performance and failure modes.
• Produce actionable insights from evaluation outputs to identify gaps and improvement opportunities, and communicate recommendations to product and operations stakeholders.
• Build and iterate on ML models (AI or traditional) that improve support efficiency, such as prediction models for routing, deflection, resolution likelihood, or quality scoring.
• Collaborate closely with data scientists, analysts, ML engineers, and product managers to deliver measurable improvements to customer support experiences.
• Learn and share best practices across the data and AI community at SumUp, contributing to a culture of continuous improvement.

You'll be great for this position if you have:

• 3+ years of experience applying data science to real-world products, ideally in AI systems, customer support, quality measurement, or evaluation frameworks.
• Solid understanding of machine learning and statistics, including experimentation and metric design.
• Experience working with LLMs or NLP systems, and interest in evaluation methods (e.g., rubric-based scoring, prompt-based judges, calibration approaches, or human-in-the-loop evaluation).
• Strong ability to write clean, reliable code in Python (pandas, numpy, scikit-learn) and strong working knowledge of SQL.
• Experience building or contributing to data workflows and pipelines, and familiarity with taking models into production environments.
• Confidence communicating results and insights to both technical and non-technical stakeholders, with a strong focus on practical business impact.
• A proactive mindset and willingness to take ownership of tasks while collaborating closely with the team in a fast-paced setting.

Why you should join SumUp:

• Opportunity to work with SumUppers globally on large-scale fintech products used by millions of businesses worldwide, from our Berlin office. This involves an office-first setup.
• Commitment to Diversity and Inclusion: Be part of a workplace that values and promotes diversity, fostering an inclusive environment where everyone's perspectives are respected and embraced
• A dedicated annual L&D budget for attending conferences and/or advancing your career through further education.
• A corporate pension scheme where we match up to 20% of your contributions
• Enjoy the unique opportunity to take a well-deserved break with our 30-day sabbatical benefit after completing 3 years of employment with SumUp.
• Earn additional rewards by referring talented individuals to join the SumUp team