A simple way to validate and monitor the performance of your ML Applications.
Part of the All Things Data! specialist track
With all the hype about applications that uses machine learning, I think there is one key aspect that developers tend to forget: "Performance check and monitoring".
ML and AI services have become very accessible and can be integrated into any application you can think of. But what do you do after you integrated your ML models to your application? <u>How do you know that the output of the ML models are correct and up to standard? What are the signs that the model's performance is changing and how do you take action on such changes? </u>
Typically, these problems are just discussed on a theoretical and research level. But how can we carry over these techniques and apply it to our application? Not just that, how can we make it so that monitoring and performance check is as simple as writing a unit test (or not).
In this session, we will learn some simple but effective ways on model performance monitoring as well as look at some python implementation and architecture consideration.
See this talk and many more by getting your ticket to PyCon AU now!
I want a ticket!The inspiration for this talk came from a couple of months ago when my partner (software engineer) started looking into ChatGPT for her work. Almost every time, our conversation always ends with her asking <u>"how do I know that the output of our application using chatgpt is correct?"</u>
Model validation and performance checking is a very common topic on research and in the academe. However, due to the rise of AI as a service, more and more people started encountering this problem.
Having implemented machine learning applications for more than 5 years, model validation can be quite cumbersome, especially when dealing with high traffic and variable type of datasets. But through the years, we've picked up some simple but practical approach to add these features in our python applications.
The talk will focus on the following key ideas:
- Why do you need model validation?
- How do you choose the right validation metric for your ML application?
- Implementing a simple model validation module in python.
- Understanding the model validation result and how this affects your ML application.
This session will require some basic understanding of machine learning but all the resource will be beginner friendly and code examples will be available for participants to try out in their free time.
Hi, I'm Ninz!
tldr; Software Engineer to Machine Learning Engineer. Finished masters with research focus on agriculture and image processing. Very active in the community, helps local school with research and programming, and occasionally talks at events. Usually out in the trails or just casually looking up in the night sky.