Usage of Codio in 2023 expanded significantly, with learners in 183 countries and all U.S. states and territories utilizing our platform to complete millions of assignments.
Learners attempted over 2.6 million assignments and nearly 14 million assessments, spending almost 1.8 million hours on the platform (equivalent to over 200 years).
Course admin and delivery
Zooming out, there were over 6,500 courses delivered via Codio in 2023. Courses taught on Codio might use a curriculum written entirely by the instructor, Codio's customizable interactive course text and assessments, or a combination of the two.
45% of courses in Codio utilized Codio's LMS integration, enabling them to integrate Codio assignments with their LMS seamlessly.
In 2023, Codio released the LTI-based mentor filtering feature, which is particularly helpful for tech skill courses mentored by industry professionals or higher education courses with TAs managing subsets of students. This allows administrators or professors to have mentors or TAs join a course easily while ensuring they only have the capabilities they need for their group of learners.
Behavior Insights
In March, Codio released our patented Behavior Insights feature, quantifying student programming behaviors. Behavior Insights can be configured for whatever behavior a faculty member cares about. Some configurable dimensions are only helpful for identifying a single behavior. For example, the over half a million assignments on Codio with detected pastes are only relevant if your concern is plagiarism.
However, other configurable dimensions can help detect different behaviors. For example, an instructor might be interested in the percentage of time students are in an error state, probably debugging the error. If this percentage of time is very low, instructors might consider an academic integrity intervention. Conversely, an instructor may be concerned about student frustration if this percentage is very high. On average, in 2023, students spent about 25% of their coding in an error state.
Virtual Machines
Codio enables instructors to run Windows and Ubuntu virtual machines directly within the platform—each running its own OS and functioning separately from other VMs in the same course or organization.
This year, almost 75,000 VMs were started, and here at Codio, we worked to develop interactive capabilities between VMs and Codio instructional content (Guides) in August, and we have enabled interaction with virtual machines using terminal commands.
Prime Assignments for Timed Exams
With over 2.6 million assignments started by students, there are some use cases where instructors want to be sure students can seamlessly access content—like timed exams. Codio released an assignment "Prime" feature in July, which allows instructors to indicate the start time and number of students so Codio can pre-provision assignments for instantly accessible containers in time-sensitive, synchronous contexts.
Instructors can prime an assignment whenever they feel it will bring them peace of mind—several clients who have switched from other platforms or courseware use it for as few as 50 students due to the poor reliability of their prior solution.
We recommend priming for large assignments, like ones that contain node modules or extensive data sets, or for assignments that have 100+ students starting at the same time, like a large exam. Instructors can also prime smaller assignments with starter code files for thousands of students beginning simultaneously.
Parameterized Assessments
Whether learners attempted timed exams or weekly check-in quizzes, they had many opportunities to display their skills and knowledge; in fact, almost 14,000,000 assessment attempts were made in 2023.
With so many completed assessments, dishonesty and plagiarism can concern some instructors. In late 2022, Codio released parameterized assessment, enabling instructors to generate many unique assessments, ensuring students get different assessment questions even when taking the same exam.
To create parameterized assessments, instructors write parameters in Python code in the built-in code editor around one question with some randomized variables. Then, a large set of unique assessments will be generated, and each student will be assigned one instance of the question, often meaning they will have an individualized assessment. In 2023, almost 25,000 assessments were parameterized assessment attempts.
Pair Programming
Collaboration through group work is common in computing courses, as it enables practicing and building critical soft skills. Over 15,000 group assignments were accomplished using pair programming in Codio, allowing groups of students to work together.
With pair programming, one student is the "driver" and types code, while the others are navigators who can see the changes, and students can switch roles as needed. In the classroom, pair programming can enable more diverse solutions, designs, and processes through collaboration. Additionally, it allows learners to practice skills they will need in their professional development, like cooperation in team settings.
What's in store for 2024?
In 2024, we will still hear a lot about AI—but we think this conversation of excitement will move beyond basic chatbots and more into niche fields and applications. For example, at the start of 2024, Codio has some AI-enabled features planned:
- Syllabus-based AI-assisted course constructor: Simply upload a PDF or copy-paste your syllabus into Codio, and AI will assemble a custom course from Codio's interactive courseware catalog
- Error message augmentation: Enable (with or without AI) human-readable notes to help students independently resolve up to 76% of error messages.1
- Hint bot: Provide stuck students with on-demand hints (not solutions!) via a chatbot by feeding in the context of the page they are on and their current code state.
As we wrote in an earlier blog, Codio's work is guided by the following principles when implementing AI-enabled features:
- Ensuring teacher control
- Keeping humans in the loop
- Transparency on dataset and practices
Additionally, AI will likely replace those space-shuttle-like dashboards and instead provide human-readable summaries. Dashboards can then be simplified into more easy-to-digest numbers that complement the more detailed AI-generated reports, which can be delivered straight to the instructor's email inboxes like a note from a helpful TA.
Looking back, 2023 was a busy and productive year here at Codio for our users—and us! Looking forward, we're eager to expand our understanding of how users interact with the platform and continue to improve computing education for all.
1 Brett A. Becker, Kyle Goslin, and Graham Glanville. 2018. The Effects of Enhanced Compiler Error Messages on a Syntax Error Debugging Test. In Proceedings of SIGCSE'18, February 21–24, 2018, Baltimore, MD, USA. DOI: https://dx.doi.org/10.1145/3159450.3159461
