How to perform test for significance difference?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

How to perform test for significance difference?

NehaBologna
Hello to all.

I have a small confusion, kindly if you could suggest something?

I need to compare two algorithms, CART and NNET in R. The results show that
NNET has better MAE value, but I want to find if there is any significance
difference between the results of both algorithms? The dataset I used is
about regression problem. Hence, I used wilcoxon test as:

//For CART

search <- train(Results ~ ., data = training,
method = "rpart",

metric = "MAE",
preProc = c("center", "scale", "zv" ),

getTrainPerf(search)
x=resid(search)

Then
<https://www.facebook.com/thenmozhi.ramagounder?__tn__=%2CdK-R-R&eid=ARBBMjI1AS5FYMfsAWEjxzbMRUd3GZnEPAAuE2ci8UORtsysXGV9i9TGqCy_e6m3hE_e-t8V7TXcVo-F&fref=mentions>
for
NNET

search2 <- train(Results ~ ., data = training,
method = "nnet",

metric = "MAE",
preProc = c("center", "scale", "zv" ),

getTrainPerf(search2)
x2=resid(search2)
wilcox.test(x,x2)

I want to ask is it the right way to find the significance test? And , if
not what is the better way. Kindly suggest me a solution as I have searched
a lot related to it on the web but found nothing.

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.