This paper describes an improved ridge-adding approach to handling singularity problem that is frequently encountered among the entire regularization path of SVM. Different from the existing ridge-adding method which directly modifies each data point, our new approach adds a small random ridge to the Karush-Kuhn-Tucker (KKT) condition instead. Such random ridge can ensure that only one index in each iteration enters or leaves the active set, and guarantee a simpler implementation and lower computational complexity. Compared with the existing ridge-adding method, our improved approach can effectively reduce the accumulated influence of the added ridges on the solution path. Experimental results are performed to verify both the efficiency and computational advantages of the proposed method.
