Home Data Science Assessing machine learning algorithm performance

Assessing machine learning algorithm performance

by Jack Simpson

I wrote a few quick bullet points down from the article “How To Implement Machine Learning Algorithm Performance Metrics From Scratch With Python“.

Metrics

  • Classification accuracy
    • Test how well predictions of a model do overall
    • accuracy = correct predictions / total predictions
  • Confusion matrix
    • Use to identify how well your predictions did with different classes
    • Very useful if you have an imbalanced dataset
    • I wrote an extremely hacked together confusion matrix for my tag identification software. I had 4 classes (U, C, R, Q) and the confusion matrix shows you what your model predicted against what the real category was.

U

C

R

Q

U

175

17

67

1

C

11

335

14

0

R

26

8

298

0

Q

6

0

3

93

  • Mean absolute error for regression
    • Positive values – the average of how much your predicted value differ from the real value
  • Root mean squared error for regression
    • Square root of the mean of squared differences between the actual and predicted value
    • Squaring the values gives you positive numbers and finding the root lets you compare the values to the original units.

You may also like