Individualizing Student Assessments with Parameterized Assessments

11:35 AM on November 21, 2022 | computer science Assessments

Parameterized assessments allow instructors to author one question, with some randomized variables, which will then generate a large set of unique assessments.

Students are assigned one of the many instances of the question - often meaning they have a unique or individualized assessment.

Codio recently launched support for parameterized assessments within our existing assessment engine.

Learn more about using parameterized assessments as part of an "Evergreen Curriculum" in our webinar "Moving Beyond Detecting Cheating: Creating "Evergreen" Course Materials with Randomized & Parameterized Assessments."

Webinar: Creating Evergreen Course Content

Here are the basics of using parameters in Codio assessments:Assign parameter in PARAMETERS tab:  codio_parameters["variable_name"] = random.randint(0,100)  codio_parameters = dictionary that holds parameters variable_name = Variable name is dictionary key  Mustache templating to use variable on other assessment tabs:   Some other text  that stays the sameWhile the tooling is pretty simple, designing parameterized assessments can be challenging. To help get you started and understand their potential use, below are a few examples across assessment types.

Multiple Choice - What is the expected output?

One of the easiest ways to use parameters is to vary small details in questions - like loop conditions in this predict the output question:## What is the output of the following code?  ```python-hide-clipboard for i in range(,): for j in range(,): print("X", end="") print("") ```Variables x and y allow us to control the range on the outer loop, while variables a and b allow us to control the variables on the inner loop.

We then go to the PARAMETERS tab to declare these four variables:import random  codio_parameters = dict()  codio_parameters["x"] = x = random.randint(0, 1) codio_parameters["y"] = y = random.randint(2, 5) codio_parameters["a"] = a = random.randint(0, 1) codio_parameters["b"] = b = random.randint(6, 8)There is a handy “Add Code Example” button on the right-hand side of the screen which will import random, declare the codio_parameters dictionary and provide a sample of the syntax.

You can also use the “Generate Sample Parameters” button on the right-hand side to check your code and see an example set of parameters in the small box below the button.

The next part of this question is handling the answer choices. These need to be different depending on the variables which are being randomly selected. We can simply specify four options based on misconceptions we know students often have about nested loop execution (e.g. an off by one error, mixing up rows and columns):unnamed-3-1

Back in the PARAMETERS tab, we generate the four answers by copying the code from the question and swapping out prints for string concatenation:import random  codio_parameters = dict()  codio_parameters["x"] = x = random.randint(0, 1) codio_parameters["y"] = y = random.randint(2, 5) codio_parameters["a"] = a = random.randint(0, 1) codio_parameters["b"] = b = random.randint(6, 8)  correct = "" for i in range(x, y): for j in range(a, b): correct += "X" correct += "\n" codio_parameters["correct"] = correct   more_by_one = "" for i in range(x, y+1): for j in range(a, b+1): more_by_one += "X" more_by_one += "\n" codio_parameters["more_by_one"] = more_by_one  rotated = "" for i in range(a, b): for j in range(x, y): rotated += "X" rotated += "\n" codio_parameters["rotated"] = rotated  rotated_more_by_one = "" for i in range(a, b+1): for j in range(x, y+1): rotated_more_by_one += "X" rotated_more_by_one += "\n" codio_parameters["rotated_more_by_one"] = rotated_more_by_oneThe PARAMETERS tab is just a python script - so the full power of the core python programming language is available to you! Need a library added? Contact support through our in-product chat or email help@codio.com.

 Previewing the assessment within the assignment authoring interface makes it easy to see where variables are used in the assessment:parsons-output-codioAfter you publish, you can see an example of the assessment by either logging in as a Test Students or using the Preview button on the course dashboard:

parsons-output-previewTo make the counting easy on the students, this assessment was set up to make rows and columns which are less than 10. This feels limiting in terms of randomization, but with the 4 variables each with 2, 3, or 4 options, we have 48 different unique instances of this question.

Fill in the Blank - Vocabulary Matching

One of our favorite Codio tricks is using a markdown table in a Fill in the Blanks question to create a matching assessment. We set it up with four vocabulary words and definitions, and three distractors using mustache templating:

mustache templatingThis creates a nice matching assessment:matching-mustache-assessmentIn the PARAMETERS tab is a list of tuples storing vocabulary word and definition pairs. These are shuffled using random.shuffle and the first four terms and definitions are assigned to the table variables and last three terms are assigned to the distractor variables:
import random  codio_parameters = dict()  words = [ ('class', '"blueprint" used to create objects'), ('attribute', 'property of an object that is generally accessed through `get` and `set` methods'), ('method', 'a function of an object'), ('object', 'a particular instance of a class'), ('inheritance', 'basing an object on another object'), ('encapsulation', 'bundling together related data and functions'), ('polymorphism', 'code that works across different types') ]  random.shuffle(words)  codio_parameters["first_term"] = words[0][0] codio_parameters["first_definition"] = words[0][1]  codio_parameters["second_term"] = words[1][0] codio_parameters["second_definition"] = words[1][1]  codio_parameters["third_term"] = words[2][0] codio_parameters["third_definition"] = words[2][1]  codio_parameters["fourth_term"] = words[3][0] codio_parameters["fourth_definition"] = words[3][1]  codio_parameters["first_distractor"] = words[4][0] codio_parameters["second_distractor"] = words[5][0] codio_parameters["third_distractor"] = words[6][0]After you publish, you can see an example of the assessment by either logging in as a Test Students or using the Preview button on the course dashboard:example-parameterized-assessmentIf you consider order important, 7 choose 4 creates 840 unique matching tables. Ignoring the order of the rows, 7 choose 4 means there are 35 unique subsets of vocabulary words. 

Parsons - Creating Parent and Child Classes

Parameterized assessments even work well with Parsons problems. We can use variables to randomize the names and attributes in a simple parent and child class:

Drag the blocks below to create a super class or parent class called `` with the attribute ``. Then, declare a sub or child class called `` with the attribute ``.  *Remember* that indention matters.The lines of code that become blocks use the same mustache templating:Drag the blocks below to create a super class or parent class called `` with the attribute ``. Then, declare a sub or child class called `` with the attribute ``.  *Remember* that indention matters.When previewing we can see that the variables are used many times within the assessment:parsons-parameterized-outputSimilar to the vocabulary exercise above, we create some nested structures and use random.shuffle to select a specific example at random. The main list indexes parent classes, the first next list is potential child classes, and the second nested list has potential class attributes:import random  codio_parameters = dict()  classes = [ ('vehicles', ['car', 'motocycle', 'scooter', 'moped', 'truck'], ['mileage', 'make', 'model', 'year', 'axels', 'wheels', 'seats', 'storage_capacity']), ('pets', ['cat', 'dog', 'bird', 'hamster', 'turtle'], ['birth_year', 'age', 'breed', 'name', 'identifying_markings', 'owner_name', 'owner_phone_number', 'vaccines']) ]  # pick one of 2 parent classes random.shuffle(classes) codio_parameters["parent_class"] = classes[0][0]  # pick one of 5 child classes random.shuffle(classes[0][1]) codio_parameters["child_class"] = classes[0][1][0]  # pick 2 of 8 class attributes (56 permutations) random.shuffle(classes[0][2]) codio_parameters["parent_class_attribute"] = classes[0][2][0] codio_parameters["child_class_attribute"] = classes[0][2][1]  # 2 * 5 * 56 = 560 unique assessmentsAssuming the order of class attributes matters since it determines which class they belong to, 2 possible parent classes with 5 possible child classes, with 2 out of 8 possible class attributes results in 560 unique Parsons assessments.

Input/Output Code Tests - 2D List Manipulation

So far we have covered formative assessments, but coding exercises can be parameterized too (even using Codio’s scriptless Input/Output Code Test)!

As an example, the assessment below asks students to manipulate a 2D list:for alt text:   Zooologists and biologists log the markings on animals as coordinates, but when trying to identify the animal later, they want a visual representation of the markings.  You are programming a utility called Zoo Print - which takes in coordinates and creates a representation of the animal's markings visually.  The first animal for you to implement is a  where the scientists log their unique spots. You will print out a 25 x 25 2D list where an underscore (`_`) is the default and a octothorp or hashtag (`#`) indicates part of the 's spot.  For each spot, the coordinate represents the top-left corner of the spot pattern. Each spot has the following 3x3 pattern: ```markdown-hide-clipboard  ```The first two cases that should be tested are (1) can they match the provided 3x3 pattern and (2) can they correctly shift their pattern. We start with these two cases and can hard-code the inputs since those are the same regardless of the pattern:
parameterized-code-test-outputIn the PARAMETERS tab, we pick a random spotted animal just for fun, but the main randomization is creating the 3x3 pattern that is constructed in the `spot` 2D list and converted to a String called `spot_pattern`:import random  codio_parameters = dict()  animals = ['leopard', 'giraffe', 'cheetah', 'deer', 'snowy owl', 'gecko', 'dalmation'] codio_parameters['animal_type'] = random.choice(animals)  spot_places = [item for item in range(0, 9)] random.shuffle(spot_places) spot = [['_', '_', '_'],['_', '_', '_'],['_', '_', '_']] for i in range(5): spot[spot_places[i]//3][spot_places[i]%3] = '#'  def list_to_string(list_name): string_version = '' for i in range(len(list_name)): string_version += ''.join(list_name[i]) string_version += '\n' return string_version  codio_parameters['spot_pattern'] = list_to_string(spot)  zoo_print = [] for i in range(25): zoo_print += [['_' for item in range(25)]] print(zoo_print)  # top-left spot only zero_zero = [row[:] for row in zoo_print] for i in range(5): zero_zero[spot_places[i]//3][spot_places[i]%3] = '#' codio_parameters['zero_zero'] = list_to_string(zero_zero)  # bottom right x = 22 y = 22 bottom_right = [row[:] for row in zoo_print] for i in range(5): bottom_right[spot_places[i]//3+y][spot_places[i]%3+x] = '#' codio_parameters['bottom_right'] = list_to_string(bottom_right)As mentioned above, core python is fully available in the PARAMETERS tab - meaning I can create a handy `list_to_string` helper method.

The second part of the code is generating the `spot_pattern` in the top-left corner of the 25x25 swatch (stored in `zero_zero`) for the first test case, and then generating the `spot_pattern` in the bottom right corner for the second test case.

After you publish, you can see an example of the assessment by either logging in as a Test Students or using the Preview button on the course dashboard:preview-code-testWe provide some starter code to handle the user input for the students since the objective is 2D list manipulation, and the unique 3x3 spot pattern has been generated and is printed on the right as part of the question specification.

The randomness of the pattern requires students to figure out the unique row and column math for their specific pattern – one of 126 possible patterns.

Learn more about using parameterized assessments as part of an "Evergreen Curriculum" in our webinar "Moving Beyond Detecting Cheating: Creating "Evergreen" Course Materials with Randomized & Parameterized Assessments."

Webinar: Creating Evergreen Course Content

Elise Deitrick

Elise is Codio's VP of Product & Partnerships. She believes in making quality educational experiences available to everyone. With a BS in Computer Science and a PhD in STEM Education, she has spent the last several years teaching robotics, computer science and engineering. Elise now uses that experience and expertise to shape Codio's product and content.