Senior AI Engineer

Our requirements

• 6–8 years of hands‑on software engineering experience, with at least 3 years focused on production AI and LLM applications.
• Extensive experience with Python as the primary language for building AI services, RAG pipelines, agents and evaluation tooling.
• Proven track record integrating LLM APIs and SDKs in production environments.
• Hands‑on expertise in prompt engineering, RAG architectures and LLM orchestration frameworks.
• Demonstrable experience designing and operating LLM evaluation frameworks in production: accuracy metrics, hallucination detection, regression testing, ground truth dataset design and output monitoring.
• Familiarity working with .NET and C# at the integration level; ability to design clear Python‑to‑.NET API contracts and collaborate on a .NET codebase when required.
• Strong software engineering fundamentals: API design, automated testing, version control and CI/CD pipelines.
• Effective communication skills, able to engage clinicians and non‑technical stakeholders on practical implications of model evaluation.
• Fluency in English (working language of the team).

Optional

• Experience in healthtech, medtech or regulated industries; background in MLOps or production AI monitoring; familiarity with Medical Device Regulation (MDR), the EU AI Act or quality management for AI systems; experience with clinical safety frameworks or human‑in‑the‑loop systems.

Your responsibilities

• Develop and maintain production code for RAG pipelines, LLM orchestration, prompt engineering and agent workflows on the healthcare LLM platform.
• Design, implement and own the evaluation framework for model outputs, including accuracy metrics, hallucination detection, clinical safety checks and regression testing across releases.
• Create and maintain ground truth datasets for evaluation in collaboration with clinicians and content partners, ensuring data quality and traceability.
• Implement production monitoring and alerting for model outputs: drift detection, quality regressions and anomaly reporting.
• Produce evaluation evidence and traceability required for regulatory and quality processes, including planned AI and Quality Control certifications alongside existing ISO 27001 and ISO 27701 controls.
• Design robust API contracts between Python‑based AI services and the .NET / Angular platform deployed on Azure; collaborate on integration and handoffs.
• Pair with the architecture‑focused Senior AI Engineer on day‑to‑day implementation, share knowledge with the in‑house engineering team and document evaluation practices.
• Engage with clinicians and non‑technical stakeholders to translate clinical safety requirements into measurable evaluation criteria and test plans.
• Contribute to software engineering best practices: code review, automated testing, version control and CI/CD for AI services.

About the project

This is how we work on a project

• Continuous Deployment
• Continuous Integration
• testing environments

What we offer

• Opportunity to shape the AI architecture of a Medical Device Class IIa module used by millions of European users.
• Work alongside a strong in‑house engineering team and a dedicated second Senior AI Engineer focused on quality and evaluation.
• Significant technical ownership and influence over integration, orchestration and certification activities (MDR, AI Act alignment).
• Hands‑on exposure to state‑of‑the‑art LLM platforms, agent frameworks and Azure‑based production environments.
• Remote work
• Unique TEAL culture, relationship- and respect-driven community, non-corporate atmosphere.
• Agile approach and no bureaucracy.
• Outstanding integration trips to various places in Europe for all employees.
• Activities to support your well-being and health.
• Luxmed Gold Extended medical care and Multisport Plus benefit.

Benefits

• sharing the costs of sports activities
• private medical care
• sharing the costs of professional training & courses
• remote work opportunities
• flexible working time
• integration events

YOUR ITEAMS sp. z o.o.