8 Grading Utilities

This module provides utilities that can serve as the basis for an automated grading system. In particular, it provides:

• A mechanism to establish deadlines with or without allowing late submissions. The idea is to avoid having someone stop the server at midnight in order to stop accepting submissions.

• A mechanism to establish a maximum number of submissions. The idea is that students can get limited feedback before doing the definitive submission.

• A mechanism to keep track of the user’s score, and where failed tests add a penalty to that score.

• A mechanism to create a report (a text file) of the evaluation of the submission, instead of reporting errors immediately when they are found. Moreover, it is possible to delay the publication of this report, in order to avoid potential abuses from the students.

The following example illustrates a checker module using this infrastructure:

(module checker handin-server/checker

(require handin-server/grading-utils)

; Checks that submission is on time and that the user has submissions left

(pre:

(check-deadline)

(check-max-submissions))

; Ends the report by adding the score and writes it in the user directory

; This way, students can see their reports from the web interface.

(post:

(add-score-to-report!)

(write-report))

(check:

; Get timestamp of the submission and add it to header and report

(update-submission-timestamp!)

(add-header-line! (get-submission-timestamp))

(add-report-line! (get-submission-timestamp))

; Acceptance tests: reject the submission if any of these tests fail

(!test (foo 1) 3)

; ...

; Grading

; Initialize max score

(set-test-max-score! 100)

; Failure discounts 25 points

(@test "Sample case 1"

"Error using even? predicate"

(bar '(1 2 3 4) even?)

'((2 4)(1 3))

25)))

Deadlines are configured in the "config.rktd" file (see Server Setup), in a per-assignment basis, through the "deadline" key. This is an optional keyword that holds an association list where each sublist has three elements:

• the assignment name, which must coincide with the name used in the "active-dir" key

• the deadline date, specified as a list "(year month day hour minutes seconds)"

• a non-negative integer indicating the amount of days allowed for late submissions. If this value is 0 then the deadline is strict. Late submissions are marked as such in the report and in the textual version of the submission (if created).

Consider an example configuration:

By default users have an unlimited number of submissions. Restrictions on the number of submissions must be defined in the "config.rktd" file, through the optional max-submissions key. This key must hold an association where each sublist has two elements: the assignment name, and a non-negative integer specifying the allowed submissions. For example:

check-max-submissions only counts the number of existing successful submissions, more specifically, it counts the number of "\"SUCCESS-n\"" directories specific to each user. Therefore, submissions that rejected by the pre-checker or checker do not count against the submission limit.

(define (add-score-to-report!)

(add-report-line! (format "Final Score: ~a out of ~a"

(get-student-score)

(get-test-max-score))))

Internally, the macro calls the start-timer procedure in the following way:

(define-syntax (write-report stx)

...

(define report-delay (get-report-delay-in-minutes))

(start-timer (start-timer-manager) (* 60 report-delay) ...))

The default behavior is obtained because get-report-delay-in-minutes returns 0 if the key is not found in "config.rktd".