training set,validation set, test set的区别

lhdgriver

书上说的是在training set里训练参数，在validation set里挑选超参数，然后用test set里的数据作为最后的结果。有人能举个详细点的例子吗谢谢

bigknife

回复第1楼的 lhdgriver：检测集里的数据，学习的最终目的。通过学习，得到模型，然后对检测集中的数据做预测或者分类。这是最终目的。而为了防止对训练数据的过拟合，将训练数据再分为training和validation，training做模型参数的估计，而validation则是用来验证你的模型是否对训练数据过拟合，通过验证了，就可以在test set上做预测了。应该是这么个意思。

micro@

The names are a little bit misleading, because both the training set and the validation set participate in training, whereas the test set does not. In this sense, both training set and the validation set can be called the training set. The reason why they are separated is only because it provides a surprisingly simply way to limit the effect of overfitting during training. If you have other methods to achieve this, there is no strict requirement for using a validation set.

lhdgriver

回复第3楼的 micro@：So if I add the regularization term, there is no need using validation set?

e.g In Knn, can validation set be used to determine para k?

lhdgriver

回复第2楼的 bigknife：看伯克利的机器学习教程，上面说validation set是用来确定超参数的，(比如kNN的k)，然后我问了好几个人，每人都有自己的理解。。

twinsken

回复第5楼的 lhdgriver：hyperparameter 你觉得是什么 k是么？

nan.xiao

不必纠结于表达。在老师的指导下做个实验写一篇应用的文章，就自然知道是怎么回事儿了。以至于各式表达的含糊和多义性都显得无足轻重。

micro@

回复第7楼的肖楠：Not necessarily unimportant, because so many people have (mis)used validation set as the test set when reporting error rates, which are usual overly optimistic.

Literally validation set and test set seem to be closer to each other; however, validation set is actually used in training and is closer to the training set than the test set.

lhdgriver

回复第7楼的肖楠：平时做实验就没用到过validation set。所以很好奇，是不是只有实际应用的时候validation才有意义，写论文的时候，只需要 training set 和 testing set 就行了