Principal Components Analysis (PCA)
How do you choose how many and which eigenvalues/eigenvectors to use?
Kaiser Criterion
This says to retain only factors with eigenvalues greater than 1. In other words, if a factor does not extract at least as much as the equivalent of one original variable then it is discarded. This criterion is named after Kaiser as he proposed it in 1960. It seems used quite frequently.
The Scree Test
This is a graphical test used to decide how many factors to keep. To perform this test, first, plot the eigenvalues in decreasing order. Next, Cattell suggests to find the place where the smooth decrease of eigenvalues appears to level off (to the right) similar geological scree (loose rock debris at the bottom of a rocky slope).
Here are some other useful terms and definitions from the dictionary:
Multicollinearity refers to linear inter-correlation among variables. Simply put, if nominally "different" measures actually quantify the same phenomenon to a significant degree -- i.e., wherein the variables are accorded different names and perhaps employ different numeric measurement scales but correlate highly with each other -- they are redundant.
1 comment:
Questions of "How many?" or "Which?" principal components to retain can only be answered within the context of the problem. For visualization problems, for example, it is usually the case that only two or three principal components can be used. In modeling, however, it might be the case that all components could be used, though a principled attribute selection process should really be making this decision.
-Will Dwinnell
Data Mining in MATLAB
Post a Comment