Support Vector Selection for Regression Machines

Lee Wan-Jui National Sun Yat-Sen University
Yang Chih-Cheng National Sun Yat-Sen University
Lee Shie-Jue National Sun Yat-Sen University
In this paper, we propose a method to select support vectors to improve the performance of support vector regression machines. First, the orthogonal least-squares method is adopted to evaluate the support vectors based on their error reduction ratios. By selecting the representative support vectors, we can obtain a simpler model which helps avoid the over-fitting problem. Second, the simplified model is further refined by applying the gradient descent method to tune the parameters of the kernel functions. Learning rules for minimizing the regularized risk functional are derived. Experimental results have shown that our approach can improve effectively the generalization capability of support vector regressors.
Orthogonal least-squares
gradient descent
learning rules
error reduction ratio
mean square error