Parameterized Assessment Templates

edeitrick · January 23, 2023, 1:40pm

A thread to collect templates for quickly authoring auto-graded, parameterized assessments on Codio.

edeitrick · January 23, 2023, 1:52pm

Create multiple choice/select all assessments where presented answer choices are pulled from lists.

Useful when checking for understanding about…

A newly introduced concept (“Select all that is true about New Concept”)
A piece of code/code tracing (“In which situations does the code below become an infinite loop?”)
Syntax (“Which of the examples below correctly Concept?”)

Parameters tab template:

import random

codio_parameters = dict()

correct_options = [ ] # fill in this list
random.shuffle(correct_options)

incorrect_options = [ ] # fill in this list
random.shuffle(incorrect_options)

codio_parameters["correct_1"] = correct_options[0]
codio_parameters["correct_2"] = correct_options[1]
codio_parameters["correct_3"] = correct_options[2]

codio_parameters["incorrect_1"] = incorrect_options[0]
codio_parameters["incorrect_2"] = incorrect_options[1]

Execution tab example:

Example in Codio Global assessment library:

What is Code Style?

edeitrick · January 24, 2023, 11:55am

Create a fill in the blank assessment where students match answers that were pulled randomly from a list

Useful when checking for understanding about…

Vocabulary definitions
Identifying concept usage in an example

Parameters tab template:

import random

codio_parameters = dict()

words = [
  ('term', 'definition') # fill in this list with tuples
]

random.shuffle(words)

codio_parameters["first_term"] = words[0][0]
codio_parameters["first_definition"] = words[0][1]

codio_parameters["second_term"] = words[1][0]
codio_parameters["second_definition"] = words[1][1]

codio_parameters["third_term"] = words[2][0]
codio_parameters["third_definition"] = words[2][1]

codio_parameters["fourth_term"] = words[3][0]
codio_parameters["fourth_definition"] = words[3][1]

codio_parameters["first_distractor"] = words[4][0]
codio_parameters["second_distractor"] = words[5][0]
codio_parameters["third_distractor"] = words[6][0]

Execution tab example:

Example in Codio Global assessment library:

Matching OOP Vocabulary

edeitrick · January 24, 2023, 12:14pm

Create a Parsons assessment where students build code

Useful when checking for understanding about…

Syntax practice
Common code patterns (e.g. swapping variable values, finding the maximum in a list/array)

In this case, it is generally easier to write the code in the Execution tab first and figure out what can be randomized within the code pattern.

Execution tab example:

Parameters tab template:

import random, string

codio_parameters = dict()

# generating variable names
letters = string.ascii_lowercase
chosen = ''.join(random.choice(letters) for i in range(3)) #replace 3 with number of variables
codio_parameters["var_1"] = chosen[0]
codio_parameters["var_2"] = chosen[1]
codio_parameters["var_3"] = chosen[2]

# generating starting values
codio_parameters["start_1"] = random.randint(1, 150)
codio_parameters["start_2"] = random.randint(150, 300)

Example in Codio Global assessment library:

Swapping Variable Values

edeitrick · January 24, 2023, 2:56pm

Create an Advanced Code Test that pulls random functionality and specifications from lists and runs the corresponding UnitTests

Useful when checking for…

Combining patterns to author code
Turning retired unit-tested projects into shorter-form parameterized practice

Parameters tab template:

import random

codio_parameters = dict()

functionality = [] # fill in set of functions students might write
codio_parameters["function"] = random.choice(functionality)

# input validation is a common specification detail to randomize
errors = [
  ("Type", "not a number"),
  ("Value", "negative")
]
error = random.choice(errors)
codio_parameters["error_type"] = error[0]
codio_parameters["error_type_lower"] = error[0].lower()
codio_parameters["error_description"] = error[1]

Execution tab and auto-grading script details:

Set to Custom type and for the Command field:

bash .guides/unit_test_runner.sh

Note that there are two unit tests running -

test_{{function}} checks the functionality
test_{{function}}_{{error_type_lower}} checks the input validation

.guides/unit_test_runner.sh

cd .guides/

# pulling parameters out of envt variable json string
echo $CODIO_PARAMETERS >> "parameters.json"
FUNCTION=`grep -o '"function": "[^"]*' parameters.json | grep -o '[^"]*$'`
ERROR=`grep -o '"error_type_lower": "[^"]*' parameters.json | grep -o '[^"]*$'`

# Running only two of the six unit tests based on parameters
python3 -m unittest test_circle.TestCircle.test_$FUNCTION test_circle.TestCircle.test_$FUNCTION"_"$ERROR

Python let’s you call specific cases by name.
With JUnit you can use either Categories (JUnit4) or Tags (JUnit5) to do test case filtering.

.guides/secure/test_file.py

import unittest
from math import pi
import sys
sys.path.append("/home/codio/workspace/")
import student_file_name  ## TO DO

class TestClass(unittest.TestCase): ## Rename class
  def test_function1(self):
    self.assertAlmostEqual(student_file_name.function1(1), 2)
    
  def test_function1_value(self):
    self.assertRaises(ValueError, student_file_name.function1, -2)
    
  def test_function1_type(self):
    self.assertRaises(TypeError, student_file_name.function1, "3")
    
  def test_function2(self):
    self.assertAlmostEqual(student_file_name.function2(1), 4)
    
  def test_function2_value(self):
    self.assertRaises(ValueError, student_file_name.function2, -2)
    
  def test_function2_type(self):
    self.assertRaises(TypeError, student_file_name.function2, "3")

if __name__ == '__main__':
    unittest.main()

sjason · January 24, 2023, 7:46pm

A parameterized Standard code test for SQL - the column is the value that is parameterized.

For this particular example there are four possible Select statements a student may be asked to write.

Parameters tab template:

import random as r


codio_parameters = dict()

choice = r.randint(1, 4)

if choice == 1:
  codio_parameters['column'] = 'id'
  codio_parameters['solution'] = '+----+\n| id |\n+----+\n|  1 |\n|  2 |\n|  3 |\n|  4 |\n|  5 |\n|  6 |\n|  7 |\n|  8 |\n|  9 |\n| 10 |\n+----+\n'
  codio_parameters['solution_code'] = 'SELECT id FROM trips;'
elif choice == 2:
  codio_parameters['column'] = 'trip_datetime_start'
  codio_parameters['solution'] = '+---------------------+\n| trip_datetime_start |\n+---------------------+\n| 2015-09-15 03:20:59 |\n| 2015-09-15 20:44:17 |\n| 2015-09-15 04:01:04 |\n| 2015-09-15 22:56:33 |\n|2015-09-15 21:12:33 |\n| 2015-09-15 07:14:59 |\n| 2015-09-15 15:48:41 |\n| 2015-09-15 09:40:55 |\n| 2015-09-15 10:00:08 |\n| 2015-09-15 14:57:24 |\n+---------------------+\n'
  codio_parameters['solution_code'] = 'SELECT trip_datetime_start FROM trips;'
elif choice == 3:
  codio_parameters['column'] = 'trip_datetime_end'
  codio_parameters['solution'] = '+---------------------+\n| trip_datetime_end   |\n+---------------------+\n| 2015-09-15 03:54:08 |\n| 2015-09-15 21:18:58 |\n| 2015-09-15 04:20:18 |\n| NULL                |\n| 2015-09-15 22:00:00 |\n| 2015-09-15 08:24:19 |\n| NULL                |\n| 2015-09-15 10:05:32 |\n| NULL                |\n| 2015-09-15 15:11:26 |\n+---------------------+\n'
  codio_parameters['solution_code'] = 'SELECT trip_datetime_end FROM trips;'
else:
  codio_parameters['column'] = 'trip_total_fare'
  codio_parameters['solution'] = '+-----------------+\n| trip_total_fare |\n+-----------------+\n|           79.40 |\n|           91.79 |\n|           17.67 |\n|           25.00 |\n|           92.33 |\n|           85.38 |\n|           25.00 |\n|           43.73 |\n|           25.00 |\n|            4.50 |\n+-----------------+\n'
  codio_parameters['solution_code'] = 'SELECT trip_total_fare FROM trips;'

General tab:

Execution tab example:

Grading tab:

edeitrick · January 25, 2023, 3:06pm

For more complex parameters such as lists, dictionaries, etc that grep might struggle to handle, jq can be used (link to jq online docs). First, it needs to be added to the assignment stack (i.e. sudo apt-get install jq, Project>Stack>Create New, and Publish with Stack Update).

unit_test_runner.sh then becomes:

cd .guides

# pulling parameters out of envt variable json string
FUNCTION=$(echo "$CODIO_PARAMETERS" | jq -r '.function')
ERROR=$(echo "$CODIO_PARAMETERS" | jq -r '.error_type_lower')

# Running only two of the six unit tests based on parameters
python3 -m unittest test_circle.TestCircle.test_$FUNCTION test_circle.TestCircle.test_$FUNCTION"_"$ERROR

Topic		Replies	Views
Creating "evergreen" course materials with parameterized and randomized assesments Assessments & Grading coding , auto-grading , curriculum	0	349	November 30, 2022
New feature - leave your solution in the code file using tags Assessments & Grading auto-grading , coding	0	315	March 30, 2022
How to set up auto-grading on a custom Java program Assessments & Grading	3	208	January 5, 2024
Using Python to grade Java code Assessments & Grading auto-grading , java , python	0	356	January 28, 2022
Teaching High School Level APCS A (Java) Assessments & Grading auto-grading , java	2	348	May 23, 2023

Parameterized Assessment Templates

Execution tab and auto-grading script details:

Related topics