深度学习—训练集、验证集和测试集概念.docxVIP

Training, Validation and Test Data Example: (A)We have data on 16 data items , their attributes and class labels. TrainingItem No.1.2.3.4.–dAttributesKNOWN FOR ALLClass0011RANDOMLY divide them into Training Item No. 1. 2. 3. 4. –dAttributes KNOWN FOR ALL Class 0 0 1 1 5. DATA ITEMS 1 6. 1 7. 0 8. 0 Validation 9. 0 10. 0 11. 1 12. 0 Test 13. 0 14. 0 15. 1 16. 1 (B). Next, suppose we develop, three classification models A, B, C from the training data. Let the training errors on these models be as shown below (recall that the models do not necessarily provide perfect results on tra—innienigthdeartathey are ClassificationError2/83/8 Classification Error 2/8 3/8 3/8 Item No. d- Attributes True Class Classification Model A Model B results from Model C 1. 0 0 1 1 2. ALL KNOWN 0 0 0 0 3. 1 0 1 0 4. 1 1 0 1 5. 1 0 0 0 6. 1 1 1 1 7. 0 0 0 0 8. 0 0 0 0 (C). Next, use the three models A, B, C to classify each item in the validation set based on its attribute vales. Recall that we do know their true labels as well. Suppose we get the following results: Classification results from Item No. d- Attributes True Class Model A Model B Model C 9. 0 1 0 0 10. 0 0 1 0 11. 1 0 1 0 12. 0 0 1 0 Classification 2/4 2/4 1/4 Error If we use minimum validation error as model selection criterion, we would select model C. Now use model C to determine class values for each data point in the test set. We do so by substituting the (known) attribute value into the classification model C. Again, recall that we know the true label of each of these data items so that we can compare the values obtained from the classification model with the true labels to determine classification error on the test set. Suppose we get the following results. Item No. d- Attributes True Class from Model C 13. 0 0 14. ALL KNOWN 0 0 15. 1 0 16. 1 1 Classification 1/4 Classification resultsErrorBased on the above, an estimate of generalization error Classification results Error What this means is that if we use Model C to clas