I've not tried an automated system like you describe but I have tried to make
test scripts available that compare the output of their programs to correct
output. That way the students can check their progress as they work. In
general, I find this approach doesn't work well for grading because in order to
specify a homework to the point where the students' output matches your output,
the homework ends up being 10 pages long (not unlike a legal document).
For example, in a number of instances I'd say "the function should return a
number..." and then I'd compare the returned result to the actual answer. But a
student would 'print' the number instead, not knowing the difference between
'return+autoprint' and 'print'. I found it difficult to write a test script
that gives partial credit for getting the answer right but the concept wrong.
Perhaps that's a problem with the way I grade things, but I found it difficult
to write an acceptable test script.
-roger
Barry Rowlingson wrote:
> Someone recently suggested building a system for automatically testing
> student's R programs. They would upload them to our Virtual Learning
> Environment, which would then run the code on some inputs and see if
> it got the right output. If it does, the student scores points for
> that course.
>
> My first thought was "you want to run unchecked, student-submitted
> code on a server that has access to students' grades?".
>
> Can this be done securely? The idea might be to run R in a
> chroot-jail, freshly generated for each run. The jail would not be
> able to access anything outside of it, and once the R session has
> finished the calling process can pick up the output from within the
> jail.
>
> Maybe that's overkill. Perhaps if you run the user's code as an
> ordinary user and store the answers/results in a directory only root
> can read that would work (given no local root exploits). Other
> precautions could include limiting the runtime or cputime for the R
> session. It might be necessary to limit network access too.
>
> Anyone done anything like this? Personally I think there are too many
> other problems with automated systems like this, particularly that
> just because a program produces the correct output that makes it a
> good one. Sure, at the production stage that's a requirement, but I'd
> rather students learnt to program well than to program correctly -
> since correctness follows goodness but goodness does not follow
> correctness. But that's an argument for another day!
>
> Barry
>
> ______________________________________________
>
[hidden email] mailing list
>
https://stat.ethz.ch/mailman/listinfo/r-devel> .
>
--
Roger D. Peng |
http://www.biostat.jhsph.edu/~rpeng/______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Locked
