In the last training of the series, we expand our knowledge of how to score machine learning models, discuss common pitfalls and show how to deal with them. We will do this by first examining the concepts of bias, variance, overfitting and underfitting, followed by diving into important performance metrics such as accuracy, precision, recall, F1 scores, ROC curves, etc. for classification problems and elaborating on commonly used metrics for regression. This last part in our basic toolkit allows us to properly assess a prediction model that we train to recognize images of handwritten digits during the hands-on lab session.
The training includes theory, demos, and hands-on exercises.
After this training you will have gained knowledge about:
- Overfitting, underfitting. bias-variance tradeoff
- Model evaluation in practice using sci-kit learn
- Evaluation metrics for classification, such as accuracy, precision, recall, F1, area under curve
- Interpreting confusion matrices, classification reports and ROC curves
- Decision function and classification probabilities
- Dealing with unbalanced datasets
- Evaluation metrics for regression, such as MAE, RMSE, R^2