Codio's 2022 in Review
Codio’s usage grew dramatically in 2022, with learners completing over 1 million assignments and submitting over 5 million assessment questions.
We know that different institutions use Codio differently – some as a textbook replacement with auto-graded weekly homework, others for longer-form programming projects. Many instructors leverage our assessment engine for auto-graded quizzes and tests. Of the 5 million assessments completed in 2022, over 87% of them were auto-graded by the Codio platform, usually offering learners instant feedback.
Across the different ways Codio is used, we can see that it is meaningful adoption. Learners engaged with the Codio platform, on average, for more than 15 hours. In the context of a 15-week university semester, that means a learner spent over an hour a week on Codio!
Assessment Types
Guides also allow the authoring of different assessment types, many of which are auto-graded. The most popular assessment type used in 2022 was multiple choice. While multiple-choice could be used for formative assessments in interactive textbook replacements, multiple choice being more popular than other formative assessment types such as fill-in-the-blank and parsons puzzles (i.e. re-ordering lines of code), could be due to teachers who use Codio as an exam platform.
The second most popular assessment type was code testing. Codio supports seamless integration with standard unit test libraries, scriptless Input/Output tests, and the ability to run custom, instructor-authored auto-grading scripts. Code test assessments are in addition to the 2% of all assignments with assignment-level auto-grading scripts.
The third most popular assessment type is fill in the blank, which on Codio supports Regular Expressions for smarter matching as well as using drop downs with distractors to help avoid mis-spelling.
The fourth most popular assessment type is free text. Codio supports two free text assessment types – manually graded and auto-graded. Manually graded free text is more than seven times more popular than the auto-graded counterpart.
Parsons puzzles, new to Codio in 2019, have grown in popularity and are now the fifth most popular assessment type. Codio supports not just hard-coded parsons auto-graders, but also variable checkers and unit test graders which actually test the constructed code’s functionality.
Assignment Layouts
Codio’s guides feature powers over 91% of the assignments on Codio. Guides allow instructors to design interactive educational experiences – beyond a textbook page; content authors can open a number of panels that contain code files, sudo-privileged Ubuntu terminals, Windows VMs, and browsers.
The vast majority (over 70%) of assignments contain a single panel page in guides. It is common for instructors to present markdown-formatted text, images, and embedded videos similar to how one might design an ebook. Codio usage goes beyond textbook replacement, with more than half of assignments using a 2-panel layout, typically to open a code file (or Jupyter Notebook) for learners to have hands-on practice. Almost a third of assignments use a 3-panel layout, generally opening a terminal for user-input-driven programs or a browser for previewing websites, or a virtual desktop for graphical output, in addition to a code file.
Codio started life as a cloud-based IDE for professional developers, so has all the tools typical of a fully-featured IDE, such as a debugger. This is particularly useful for larger, multi-file projects in courses beyond introductory programming courses. Content authors used a more full-feature IDE layout by revealing the file tree to learners, allowing them to create and open files in 44% of assignments.
Assignment length varies greatly from 1 page to 266 pages. Most guides-based assignments are less than 10 pages. Almost a third of guides-based assignments are only 1 or 2 pages. Almost a quarter of assignments contain 3 to 4 pages, and another quarter contains 5 to 9 pages. Only about 20% of guides-based assignments contain 10 or more pages.
Developing Code
Codio has an environment templating via our stacks feature – making it easy for students to start working on their assignments without having to struggle with the installation of the exact version of languages and libraries.
While learners have a number of options for developing code for Codio-powered assignments, including developing on their local environment and synching with their assignments container (using version control or our Desktop app) or using a third-party code editor such as RStudio or Jupyter within the assignment container, many use Codio’s built-in, full-featured code editor, which is built on Monaco, the same accessible code editor in VS Code.
Looking at the file extension of compile and run events via instrumented IDE buttons, python is the most-run programming language on Codio, followed by C++ and Java. This may be predictable given their popularity in introductory programming languages. C, a language often taught in upper-level programming courses, is the fourth most common with R coming in fifth.
Predictions for 2023
As we go into 2023, we predict seeing even more adoption of our Virtual Machine capabilities for Windows-only tools and our newly released parameterized assessments which allow randomization of parts of assessments.
We also expect to see an increase in usage of our paired programming feature with the increased ease of duplicating groups from one assignment to another. This is particularly useful for milestoned programming projects which use our copy workspace functionality to seamlessly roll students’ work from one assignment to the next – allowing more transparency into student progress through long projects.
Finally, we are looking forward to releasing improvements to our code playback feature and built-in code plagiarism tool early in the new year!
