Model Evaluation Workout
- 03:00
Practice model evaluation in Python, including writing a for loop in a Jupiter notebook to make predictions with trained models, evaluate them using R-squared and mean absolute error metrics.
Downloads
No associated resources to download.
Glossary
For Loop Machine Learning Python trained modelTranscript
In your Jupyter notebook, write a for loop that achieves all of the following for each train model in the model's dictionary, use the model to make predictions using the test inputs and save those predictions in a temporary variable inside your for loop called preds. Then print the model name and then print the R squared and mean absolute error of preds compared to the actual target variable values and round that to three decimal places.
During the lesson, you should have imported the R squared metric and the mean absolute error metric. Now we're going to write a for loop that uses our trained models to make predictions and then evaluates them. So first I'm gonna say for every key in my model's dictionary, I wanna create a new temporary variable to store my predictions. And here I've called it preds for short. I'm gonna go into my model's dictionary and select my trained model because all of them are trained now. And then I'm going to use the predict function and I'm going to apply it to our test input data frame. And now remember what this actually is. This is our data frame containing all of the inputs from those 600 and however many observations, and it's giving our trained model all of those inputs without the actual available liquidity. Remember, we've separated those, so when the model makes predictions, it's making predictions on those inputs as if it didn't know what the available liquidity would be. Then we're printing the key or the name of our model. And then in the next line we're printing out our R squared. So we're taking our r2 score function. And the way that this function works is first we give it the right answer. It's that target testing data that we separate it out. This is the available liquidity that we already know as the second argument, we're giving it the predictions that this model made. And then the R squared score function compares the target testing data and the predictions of our model, and it gives our model a score. In the next line, we're doing the mean absolute error and it works in a very similar way. So we're calling the mean absolute error function. And as the first argument, we're giving it the correct answers, our target testing data, which is just that one serious containing available liquidity. And then as the second argument we're giving it are machine learning models, predictions, and then the function is comparing those values for each of these, we're rounding it to three decimal places. Finally, I'm printing this little spacer just for readability. And when you execute that cell, you'll see that we have scores for all of our 5 models.
The values that you get when you execute this code cell should match the values here. And if they don't, you need to go back from the very top and make sure that your code matches what you see in the exercise and in these solution videos. When you execute this code cell, you might still get a data convergence warning. And if you do, don't worry, everything is still working correctly.