apparently lda from the MASS package can be used in situations with
collinear variables. It only produces a warning then but at least it
defines a classification rule and produces results.
However, I can't find on the help page how exactly it does this. I have a
suspicion (it may look at the hyperplane containing the class means,
using some kind of default/trivial within-group covariance matrix) but I'd
like to know in detail if possible.
I find particularly puzzling that it produces different
results whether I choose CV=TRUE or I run a manual LOO cross-validation.
Constructing an example, I realised that I'm puzzled about
CV=TRUE not only in the collinear case. The example is below. Actually it
also produces different (though rather similar) results for p=10 (no
n <- 50
p <- 200 # or p<- 10
testdata <- matrix(ncol=p,nrow=n)
for (i in 1:p)
testdata[,i] <- rnorm(n)
class <- as.factor(c(rep(1,25),rep(2,25)))