Parameterized assessments allow instructors to author one question, with some randomized variables, which will then generate a large set of unique assessments.
Students are assigned one of the many instances of the question - often meaning they have a unique or individualized assessment.
Codio recently launched support for parameterized assessments within our existing assessment engine.
Learn more about using parameterized assessments as part of an "Evergreen Curriculum" in our webinar "Moving Beyond Detecting Cheating: Creating "Evergreen" Course Materials with Randomized & Parameterized Assessments."
Here are the basics of using parameters in Codio assessments:![Assign parameter in PARAMETERS tab: codio_parameters["variable_name"] = random.randint(0,100) codio_parameters = dictionary that holds parameters variable_name = Variable name is dictionary key Mustache templating to use variable on other assessment tabs: Some other text that stays the same](https://www.codio.com/hs-fs/hubfs/Parameterzied%20assessments%20-%20Page%201.png?width=1240&height=580&name=Parameterzied%20assessments%20-%20Page%201.png)
While the tooling is pretty simple, designing parameterized assessments can be challenging. To help get you started and understand their potential use, below are a few examples across assessment types.
Multiple Choice - What is the expected output?
One of the easiest ways to use parameters is to vary small details in questions - like loop conditions in this predict the output question:
Variables x and y allow us to control the range on the outer loop, while variables a and b allow us to control the variables on the inner loop.
We then go to the PARAMETERS tab to declare these four variables:![import random codio_parameters = dict() codio_parameters["x"] = x = random.randint(0, 1) codio_parameters["y"] = y = random.randint(2, 5) codio_parameters["a"] = a = random.randint(0, 1) codio_parameters["b"] = b = random.randint(6, 8)](https://www.codio.com/hs-fs/hubfs/mcq-parameters.png?width=1600&height=1214&name=mcq-parameters.png)
There is a handy “Add Code Example” button on the right-hand side of the screen which will import random, declare the codio_parameters dictionary and provide a sample of the syntax.
You can also use the “Generate Sample Parameters” button on the right-hand side to check your code and see an example set of parameters in the small box below the button.
The next part of this question is handling the answer choices. These need to be different depending on the variables which are being randomly selected. We can simply specify four options based on misconceptions we know students often have about nested loop execution (e.g. an off by one error, mixing up rows and columns):
Back in the PARAMETERS tab, we generate the four answers by copying the code from the question and swapping out prints for string concatenation:![import random codio_parameters = dict() codio_parameters["x"] = x = random.randint(0, 1) codio_parameters["y"] = y = random.randint(2, 5) codio_parameters["a"] = a = random.randint(0, 1) codio_parameters["b"] = b = random.randint(6, 8) correct = "" for i in range(x, y): for j in range(a, b): correct += "X" correct += "\n" codio_parameters["correct"] = correct more_by_one = "" for i in range(x, y+1): for j in range(a, b+1): more_by_one += "X" more_by_one += "\n" codio_parameters["more_by_one"] = more_by_one rotated = "" for i in range(a, b): for j in range(x, y): rotated += "X" rotated += "\n" codio_parameters["rotated"] = rotated rotated_more_by_one = "" for i in range(a, b+1): for j in range(x, y+1): rotated_more_by_one += "X" rotated_more_by_one += "\n" codio_parameters["rotated_more_by_one"] = rotated_more_by_one](https://www.codio.com/hs-fs/hubfs/unnamed-4.png?width=1600&height=1195&name=unnamed-4.png)
The PARAMETERS tab is just a python script - so the full power of the core python programming language is available to you! Need a library added? Contact support through our in-product chat or email help@codio.com.
Previewing the assessment within the assignment authoring interface makes it easy to see where variables are used in the assessment:
After you publish, you can see an example of the assessment by either logging in as a Test Students or using the Preview button on the course dashboard:
To make the counting easy on the students, this assessment was set up to make rows and columns which are less than 10. This feels limiting in terms of randomization, but with the 4 variables each with 2, 3, or 4 options, we have 48 different unique instances of this question.
Fill in the Blank - Vocabulary Matching
One of our favorite Codio tricks is using a markdown table in a Fill in the Blanks question to create a matching assessment. We set it up with four vocabulary words and definitions, and three distractors using mustache templating:
This creates a nice matching assessment:
In the PARAMETERS tab is a list of tuples storing vocabulary word and definition pairs. These are shuffled using random.shuffle and the first four terms and definitions are assigned to the table variables and last three terms are assigned to the distractor variables:![import random codio_parameters = dict() words = [ ('class', '"blueprint" used to create objects'), ('attribute', 'property of an object that is generally accessed through `get` and `set` methods'), ('method', 'a function of an object'), ('object', 'a particular instance of a class'), ('inheritance', 'basing an object on another object'), ('encapsulation', 'bundling together related data and functions'), ('polymorphism', 'code that works across different types') ] random.shuffle(words) codio_parameters["first_term"] = words[0][0] codio_parameters["first_definition"] = words[0][1] codio_parameters["second_term"] = words[1][0] codio_parameters["second_definition"] = words[1][1] codio_parameters["third_term"] = words[2][0] codio_parameters["third_definition"] = words[2][1] codio_parameters["fourth_term"] = words[3][0] codio_parameters["fourth_definition"] = words[3][1] codio_parameters["first_distractor"] = words[4][0] codio_parameters["second_distractor"] = words[5][0] codio_parameters["third_distractor"] = words[6][0]](https://www.codio.com/hs-fs/hubfs/fill-in-blank-parameters.png?width=1600&height=1133&name=fill-in-blank-parameters.png)
After you publish, you can see an example of the assessment by either logging in as a Test Students or using the Preview button on the course dashboard:
If you consider order important, 7 choose 4 creates 840 unique matching tables. Ignoring the order of the rows, 7 choose 4 means there are 35 unique subsets of vocabulary words.
Parsons - Creating Parent and Child Classes
Parameterized assessments even work well with Parsons problems. We can use variables to randomize the names and attributes in a simple parent and child class:
The lines of code that become blocks use the same mustache templating:
When previewing we can see that the variables are used many times within the assessment:
Similar to the vocabulary exercise above, we create some nested structures and use random.shuffle to select a specific example at random. The main list indexes parent classes, the first next list is potential child classes, and the second nested list has potential class attributes:![import random codio_parameters = dict() classes = [ ('vehicles', ['car', 'motocycle', 'scooter', 'moped', 'truck'], ['mileage', 'make', 'model', 'year', 'axels', 'wheels', 'seats', 'storage_capacity']), ('pets', ['cat', 'dog', 'bird', 'hamster', 'turtle'], ['birth_year', 'age', 'breed', 'name', 'identifying_markings', 'owner_name', 'owner_phone_number', 'vaccines']) ] # pick one of 2 parent classes random.shuffle(classes) codio_parameters["parent_class"] = classes[0][0] # pick one of 5 child classes random.shuffle(classes[0][1]) codio_parameters["child_class"] = classes[0][1][0] # pick 2 of 8 class attributes (56 permutations) random.shuffle(classes[0][2]) codio_parameters["parent_class_attribute"] = classes[0][2][0] codio_parameters["child_class_attribute"] = classes[0][2][1] # 2 * 5 * 56 = 560 unique assessments](https://www.codio.com/hs-fs/hubfs/parsons-3.png?width=1600&height=884&name=parsons-3.png)
Assuming the order of class attributes matters since it determines which class they belong to, 2 possible parent classes with 5 possible child classes, with 2 out of 8 possible class attributes results in 560 unique Parsons assessments.
Input/Output Code Tests - 2D List Manipulation
So far we have covered formative assessments, but coding exercises can be parameterized too (even using Codio’s scriptless Input/Output Code Test)!
As an example, the assessment below asks students to manipulate a 2D list:
The first two cases that should be tested are (1) can they match the provided 3x3 pattern and (2) can they correctly shift their pattern. We start with these two cases and can hard-code the inputs since those are the same regardless of the pattern:
In the PARAMETERS tab, we pick a random spotted animal just for fun, but the main randomization is creating the 3x3 pattern that is constructed in the `spot` 2D list and converted to a String called `spot_pattern`:![import random codio_parameters = dict() animals = ['leopard', 'giraffe', 'cheetah', 'deer', 'snowy owl', 'gecko', 'dalmation'] codio_parameters['animal_type'] = random.choice(animals) spot_places = [item for item in range(0, 9)] random.shuffle(spot_places) spot = [['_', '_', '_'],['_', '_', '_'],['_', '_', '_']] for i in range(5): spot[spot_places[i]//3][spot_places[i]%3] = '#' def list_to_string(list_name): string_version = '' for i in range(len(list_name)): string_version += ''.join(list_name[i]) string_version += '\n' return string_version codio_parameters['spot_pattern'] = list_to_string(spot) zoo_print = [] for i in range(25): zoo_print += [['_' for item in range(25)]] print(zoo_print) # top-left spot only zero_zero = [row[:] for row in zoo_print] for i in range(5): zero_zero[spot_places[i]//3][spot_places[i]%3] = '#' codio_parameters['zero_zero'] = list_to_string(zero_zero) # bottom right x = 22 y = 22 bottom_right = [row[:] for row in zoo_print] for i in range(5): bottom_right[spot_places[i]//3+y][spot_places[i]%3+x] = '#' codio_parameters['bottom_right'] = list_to_string(bottom_right)](https://www.codio.com/hs-fs/hubfs/code_test_spot-pattern.png?width=1600&height=1366&name=code_test_spot-pattern.png)
As mentioned above, core Python is fully available in the PARAMETERS tab - meaning I can create a handy `list_to_string` helper method.
The second part of the code is generating the `spot_pattern` in the top-left corner of the 25x25 swatch (stored in `zero_zero`) for the first test case, and then generating the `spot_pattern` in the bottom right corner for the second test case.
After you publish, you can see an example of the assessment by either logging in as a Test Students or using the Preview button on the course dashboard:
We provide some starter code to handle the user input for the students since the objective is 2D list manipulation, and the unique 3x3 spot pattern has been generated and is printed on the right as part of the question specification.
The randomness of the pattern requires students to figure out the unique row and column math for their specific pattern – one of 126 possible patterns.
Learn more about using parameterized assessments as part of an "Evergreen Curriculum" in our webinar "Moving Beyond Detecting Cheating: Creating "Evergreen" Course Materials with Randomized & Parameterized Assessments."
