EVENTS

Alex Derr from the Software Engineering Institute at Carnegie Mellon University will join us at the HiRSE Seminar Series to talk about “MLTE: A process and tool for test and evaluation of machine learning models”

Abstract:
Test and Evaluation (T&E) of ML models largely focuses on model performance (e.g., accuracy) and often does not consider system aspects, which leads to models that fail in production. We will present MLTE, a semi-automated process and tool that enables negotiation, specification, and testing of ML model functional and non-functional requirements. A Negotiation Card records results of stakeholder discussions, which drive model development decisions and relevant test cases. MLTE automates test case execution and stores results that can be shared with stakeholders to provide evidence of testing that guides future iterations and system-level decisions.

The announcement and connection details are posted here:

• HiRSE-internal HiRSE Mattermorst channel,
• HIFIS Mattermost channel,
• Helmholtz Open Science Mattermost channel,
• FZJ-internal #rse Rocket.Chat channel,
• de-RSE mailing list and de-RSE Mattermost channel,
• Helmholtz Open Science Pro Mailinglist,
• RSE mailing lists of the HiRSE partners (if any).

If you are unable to join via one of these channels please email hirse@fz-juelich.de.

Institutions

• Helmholtz Platform for Research Software Engineering - Preparatory Study