GiveSpec: Autotesting
Examples
Refer to the give manual for detail documentation
&runspecs and &testmode
Directive | Purpose |
---|---|
| Autotesting specification follows after this directive until end of file |
|
|
Pre-test Section
Directive | Purpose |
---|---|
| compile submission and print code listings |
| run before each test |
| print the first m output lines and the last n output lines |
| cpu time limit in seconds |
| $exec anywhere in the test suite will be substituted by the command |
Example:
&compile
rm -f *.c
rm -f a.out
f=grid.c
classify -v $1
dos2unix $f
list $f
options="-O -Wall -Werror"
echo "Compiling with $options ..."
echo
gcc $options $f
compile_status=$?
if [ $compile_status != 0 ]
then
echo
echo "** No good. Exiting..."
exit 1
fi
cp ~/bin/trim.pl .
cp $A/gridTest[1-6] .
&end
&timelim 3
&linelim 120 20
&exec ./a.out
Test Section
Directive | Purpose |
---|---|
| Test number |
| Test mark and description |
| Command to execute |
| Directive to match student output and sample output. Add the match command/script on the next line. |
| Print given file |
| Print test input data |
| Print differences between student output and sample output
|
Example:
&test 1
&mark 1.5 gridTest1 no valid row col input
&invoke $exec < gridTest1 | ./trim.pl
&match < Student output and > Sample output
diff -iw $stdout $stdres
&showdiff -120w -iWt
&test 2
&mark 1.5 gridTest2 middle cell
&invoke $exec < gridTest2 | ./trim.pl
&match < Student output and > Sample output
diff -iw $stdout $stdres
&showdiff -120w -iWt
Post-test Section
Directive | Purpose |
---|---|
&special <mark> <description>
MARK=
&end | Test that can't be defined using the standard testing directives above. The mark comes from MARK environment variable. |
| can use
|
| You must insert a <TAB> character between description and mark in order that Xmark parses the subjective assessment correctly. |
| Total &special, &performance, &subjective |
Example: