Role overview
**3 REQUIRED SKILLSETS:**
• Software Engineering - Python
• Demonstratable experience with implementing LLM agents or working with evaluation frameworks
• Strong collaboration and communication skills (collaborate with Data Science, product owners, PMO)
ProSearch is hiring a Machine Learning Infrastructure Engineer to join our global client’s MLOps team and design scalable, robust ML infrastructure that powers AI solutions and large language models (LLMs). In this role, you will leverage your expertise in Python and ML infrastructure to develop evaluation frameworks, feedback tools, and monitoring pipelines that streamline the ML lifecycle from data annotation to model training, deployment, and performance evaluation.
As part of the MLOps team, you will collaborate with data scientists, ML engineers, and product teams to deliver production-ready AI systems that enable high-quality, reproducible ML workflows. You’ll build scalable tools and pipelines for LLM evaluation, feedback collection, and observability, ensuring that ML systems are reliable, efficient, and optimized for real-world performance.
**What you will do:**
• Design and implement LLM evaluation frameworks to support automated and human-in-the-loop assessment of model performance.
• Build custom feedback tools to collect structured and unstructured user feedback on model predictions.
• Develop systematic analysis tools for logged predictions, enabling deep dives into model behavior, error patterns, and performance trends.
• Create and maintain tooling and infrastructure that supports the end-to-end ML lifecycle, including data preparation, annotation, training, evaluation, and monitoring.
• Collaborate with cross-functional teams to integrate evaluation and feedback tools into production ML pipelines.
• Ensure scalability, reliability, and usability of Machine Learning infrastructure across teams and projects.
**What you need:**
• 3+ years of experience in Machine Learning infrastructure, MLOps, or backend engineering for Machine Learning systems.
• Strong programming skills in Python and experience with ML/DS libraries (e.g., PyTorch, TensorFlow, scikit-learn).
• Deep understanding of Machine Learning evaluation methodologies, especially for LLMs and generative models.
• Experience with Databricks for data engineering, model training, and collaborative workflows.
• Hands-on experience with SuperAnnotate or similar data annotation platforms.
• Proficiency with AWS services (e.g., S3, Lambda, SageMaker, ECS) for scalable ML infrastructure.
• Familiarity with logging, monitoring, and observability tools for Machine Learning systems.
If you are a Machine Learning Infrastructure Engineer who is passionate about generative AI, building scalable ML infrastructure, and advancing AI systems, this role is for you. Join the team and help deliver high-performance ML solutions, LLM pipelines, and MLOps tools that accelerate the ML lifecycle and bring AI innovations into production. Apply today to be part of the MLOps team designing next-generation AI systems and LLM infrastructure.