Submission and Autotesting

Owned by Derrick Lin
May 18, 2024
3 min read
[ 1 Submission ] [ 2 Autotesting ]

Ā 

SMS Field Permissions

give.spec: Submission

give.spec: Autotesting

Tutor Marking: Xmark or Imark

Plagiarism

Submission

  1. Install give and sms

  2. Create the assignment field in sms:

    1. Interactively: Student ā†’ Maintain Fields ā†’ Insert

      • To insert before the Comment field, click on the Comment item in the listbox, then click on the Insert button.

    2. Commandline:

      • e.g. echoĀ "assign1Ā markĀ 0Ā 10Ā 0.1Ā hardĀ (closed)Ā @7"Ā |Ā smsaddfield

      • open, closed, private, private closed are SMS field permissions

      • The input format of smsaddfield is based on the output format of smsfieldĀ -v

        smsfield -v Running Tcl8.6... ClassKey patstring \d\d\L\d\4{\A}\4{\d} [0-9][0-9][a-zA-Z][0-9](([A-Z])([A-Z])([A-Z])([A-Z]))(([0-9])([0-9])([0-9])([0-9])) (secondary closed) @0 StudentID patstring \7{\d} (([0-9])([0-9])([0-9])([0-9])([0-9])([0-9])([0-9])) (primary closed) @1 Name name (secondary closed) @2 Program patstring \4{\d}/\d (([0-9])([0-9])([0-9])([0-9]))/[0-9] (secondary closed) @3 Plans string (secondary closed) @4 Login string (secondary closed) @5 SpecialFlags set audited_subject;z satisfactory;s unsatisfactory;u enrolment_continues;e withhold;w query;q further_assessment;f absent;a newname;n consideration_received;c consideration_acknowledged;y z->!(s|u|e|w|q|f) s->!(z|u|e|w|q|f) u->!(z|s|e|w|q|f) e->!(z|s|u|w|q|f) w->!(z|s|u|e|q|f) q->!(z|s|u|e|w|f) f->!(z|s|u|e|w|q) c->!y y->!c (privateclosed) @6 Comment text (private) @7

  3. Install give for assignment: give-install assignment. The installation does the following:

    1. creates assignment directory under the current work directory ($WORK)

    2. creates give.spec in the assignment directory ($WORK/<assignment>)

    3. creates submission directories for each class (tuts or labs or tut-labs e.g. $WORK/<assignment>/tue15a, fri10-kora)

    4. creates all directory instead if course donā€™t run classes

  4. Enter the submission specification into file: $WORK/<assignment>/give.spec.

  5. Simple testing during submission:

    1. Create $WORK/<assignment>/dryrun - dryrun example

  6. Submit command: give cs9999 assignment files..

    1. Under the assignment directory are the ā€œclassā€ directories and under them are the student/group directories:

      • if <class> exists: <class>/<studentID> or <class>/<groupID>

      • if course donā€™t run classes: all/<studentID> or all/<groupID>

      • if donā€™t belong to any class in SMS: orphans/<studentID>

    2. Under the directory above are the files:

      • submission.tar

      • sub0.tar, sub1.tar, ... sub{N-1}.tar - Previous submissions, &backups = N

      • log - Submission and Autotesting history. Submission line contains date and number of days and hours late

Ā 

Autotesting

  1. Enter the autotesting specification into file: $WORK/<assignment>/give.spec

  2. Run the autotesting on the submissions. Commands you need to know below:

Ā 

Ā 

Ā 

Ā 

genrun assignment

Compile give.spec into shell scripts

  • GROUPS

  • SUBKEY (created only if &subkey is not the default stuID)

  • compile

  • execute

  • compare

  • cleanup

genresults assignment submission.tar

Generates the standard output of a reference submission

  • tests directory

  • results directory

run assignment group studentID

Runs specified assignment, output to screen

run1 assignment group studentID

Runs specified assignment, output to file: $OUT/group/assignment.studentID

rungrp [options] assignment [group]

Runs a group of assignments

  • options:

    -vverbose

    -cremove the temp directory after each run, default is to use the same temp directory for all submissions

    -ttest phase, will not timestamp, will not update logs

    -mmark phase, will not run a submission if it has been marked

Must clear the temp directory before each run at the start of &compile directive.

Example:

&compile rm -f datecalctest datecalctest_dcc datelib.o datelib.c fetch $A datecalctest.c datelib.h classify -v $1 list datelib.c <compile file here> &end

rerun assignment

rerun assignment group

rerun assignment group/assignment.studentID

Reset time stamp so that you can run autotesting again.

To avoid a rerun, you can use the -t option of rungrp

torun assignment

List assignments (e.g. late submissions) that have not been tested

  1. Check the autotesting results

    1. Testing results or summary: $OUT/summary.assignment

    2. Check late penalties are applied correctly - grepĀ latepenĀ $OUT/*/assignment.*

  2. Subjective marking - Xmark or the newer Imark (Imark uses a different suite of autotesting commands)

  3. To stop students collecting assignments before all tutors have finished marking, add the assignment to the withhold directive in $WORK/give.spec

    1. &withhold exam_q1 exam_q2 exam_q3 &end