Model Evaluation
- 02:02
The process of evaluating trained machine learning models using R-squared and mean absolute error metrics on testing data to select the most accurate model for predicting values.
Downloads
No associated resources to download.
Transcript
Now that your models are trained and tuned, it's time to evaluate them with your testing data and choose a winner. We'll evaluate using two metrics R squared and mean absolute error, which calculates the average error in the same units as the target variable, which in this case is millions of dollars. Go ahead and import both of these metrics into your notebook by following along with the code that you see here.
To evaluate each model, you'll use each model to make predictions based on your test inputs, and then compare those predictions against the actual target variable values. You make predictions with your model using Scikit-learns predict function in the format that you see right here. You score predictions using the format that you see in the lines below first the metric, and then in the first argument of that metric, the actual values, and then in the second argument your predicted values. So first, I'm creating this new object called lasso pres for lasso predictions, and then I'm going into my trained models and I'm picking lasso, and I'm using the Scikit-learn predict function based on my inputs in my testing data. In the second two lines I'm evaluating, so I'm printing out the string R squared, and then I am, I'm rounding this just so that it's a little bit cleaner, but I'm using the R square score function that we just imported. The first argument is my target values from my testing data, and then I'm comparing that to my lasso predictions object that I just created. Then finally, I'm using the mean absolute error function, and the first argument again is my target values from my test data, and then I'm comparing that again to my lasso predictions object that I just created, and also rounding that to three decimal places. When I execute that cell, it tells me that my R squared for my lasso model is 0.498, and my mean absolute error is 1,710. Now we're going to repeat this process with the other models that we trained and tuned and see, which gives us the most accurate predictions.