Codio was proud to sponsor the ACM's SIGCSE 2023 conference in Toronto, Canada earlier this month. SIGCSE has a plethora of opportunities to connect with and learn from our fellow Computer Science educators, professionals, and students.
During the conference, Codio's very own Elise Deitrick led a fantastic session, "Beyond 'Textbook Replacements' & Auto-Grading: Seamless, Individualized Learning Experiences," in which Andrew Goodney of USC shared how he uses Codio for his large introductory programming courses. Elise gave a preview of our new Learner Behavior Insights tool, and demonstrated our "evergreen" assessments concept, powered by our new parameterized assessments and randomized questions features.
One of our favorite aspects of the conference is learning about the research that's been conducted, so we compiled our thoughts! Below are our team's takeaways from the speaker sessions and presentations we attended...
Student-Facing Feedback
Elise Deitrick, PhD, Vice President of Product & Partnerships
One paper in the “Understanding Programming Error Messages” session discussed a classifier that determined if compiler feedback was difficult or easy to understand. The four aspects they used as dimensions of their classifier feel applicable to student feedback in general:
- Length (# of words)
- The proportion of Basic English words
- # reserved words or "jargon."
- Is it a complete sentence?
In future work, it would be interesting to see if, similar to compiler messages; we could help instructors improve their student-facing feedback using the same metrics. Think ChatGPT is going to make manual feedback a thing of the past? In the same session work was presented that showed Codex can generate a correct compiler error message explanation only about half the time -- and can provide a correct fix only about a third of the time. So while we can automate helping instructors write feedback, the human will still be in the loop for a while.
Ensuring Student Success
Anh Le, Curriculum Developer
A recurring theme at SIGCSE 2023 was "student success" in computer science education, with speakers such as Pam Schmelz from Ivy Tech Community College, Anu Bourgeois from Georgia State University, and Patricia Morreale from Kean University stressing the importance of equitable access to resources in post-secondary institutions. They noted that resources should focus on educational aspects like extra help and hands-on activities and address external challenges students face. Examples include hosting food pantries, offering equally high-quality virtual lectures as in-person ones, and checking in on a more personal level with struggling students.
Early immediate feedback was identified within the classroom as an effective technique for promoting student success. Although there was a concern that students would rely too heavily on the feedback, which would dissuade them from developing independent testing skills, Joydeep Mitra from Stony Brook University found that early feedback improves learning outcomes, even if withdrawn later, and is particularly beneficial for underrepresented groups such as women. This approach enhances their learning experience and builds confidence in their skill development. The most successful students appear to be the ones who have their needs met, both inside and outside of the classroom.
The Relationship Between Student Stress & Plagiarism
Sharon Jason, Customer Success
I went to a couple of different sessions and one paper, and even though none of them covered the topic of student stress directly, it was an underlying theme in all of them. A few ideas I heard for alleviating stress are allocating lateness excuse tokens, allowing the resubmission of assignments, and discontinuing the tradition of grading on a curve.
Plagiarism can result from a student feeling as if they have no other choice. They might be running out of time or they don't understand the material, or there is an external issue. Are there ways to design courses to provide less of an incentive to cheat? Providing more collaborative projects might reduce the incidence of plagiarism because many people would benefit from working with others.
Educator Pain Points: Understanding When Students are Struggling, Grading, & Course Admin
Patrick Ester, Director of Content
One paper is the first to study Computer Science courses from the instructor's point of view. In particular, they want to understand the pain points of instructors and know what strategies they use to alleviate the difficulties. The authors held semi-structured interviews with thirty-two participants who teach Computer Science. There was a wide variety in institution type, institution location (by country), and the title held by the Instructors.
Regarding the pain points, six themes arose. The first among them is knowing where students are struggling. Specifically, the instructors want to identify these areas of concern before students submit graded evaluations. Another pain point is answering student questions. At times the questions can be overwhelming, especially if an instructor does not have access to TAs. Speaking of TAs, limited TA support is another pain point from the study. Even if an instructor does have access to TAs, managing them presents its own set of challenges.
Grading and sending feedback to students in a timely manner is another pain point cited by instructors. Instructors are looking to balance transparency, timeliness, and providing the “human touch” in their feedback. Instructors also noted the time and effort it takes to prepare course materials. Creating teaching aids like videos did not increase student engagement. The final pain point is administrative tasks. Instructors stated that tasks not related to teaching were burdensome. In many instances, instructors had less time to improve their courses due to administrative tasks.
Given these obstacles, what do instructors do to lessen the aforementioned pain points? One solution is to use interactive textbooks or exercises. The content is already made for the instructors, and they can update the material so that it is always up to date. Interactive textbooks are also more engaging for students.
Another solution is to use an online IDE or code visualizer. Again, the goal is to increase student engagement. In addition, these tools also provide timely feedback to students. A third common solution is to use auto-graders. These tools can be configured to provide almost instant feedback to students. It also reduces the demand for instructor and TA time.
Instructors also cited the ability to implement mastery-based learning due to auto-graders. Some instructors used a flipped classroom approach. This way, they can interact with students as they code, hopefully identifying problem areas for students. The final mechanism used to deal with pain points is peer instruction. By having students help each other, they would reinforce their knowledge of the subject while freeing up time for the instructor and/or TAs.
Improving Student Career Preparedness
Kevin Noelsaint, Curriculum Developer
I attended many sessions, one in particular that caught my attention, and the most crowded session I attended was this in-person-paper session: “What is an Algorithm Course? Survey of Introductory Undergraduate Algorithms Courses in the US.” This paper covers a survey about Algorithms courses, a core of many CS programs. Algorithms are often used to evaluate candidates, making it a very important subject in the computer science curriculum. The paper presents the results of the first comprehensive survey of undergraduate introductory algorithms courses in the US. The survey included questions about instructor information, course concepts, evaluation methods, challenges faced, and desired improvements. The survey received 87 responses from 34 different states, revealing that algorithms courses vary significantly in most areas.
One of the main issues instructors face is the lack of background in math and programming ability among their students. This poses several challenges, such as creating problems in homework and tests that are appropriate for the student's level of understanding, as well as demonstrating the relevance of algorithms to their future studies or careers.
Another challenge is that instructors must focus on teaching specific topics while introducing new algorithms and methods for solving problems. To achieve this, assigning more programming projects and exercises may be necessary to help students gain practical experience. Another approach is to reorganize the course content by removing, replacing, or moving topics from other courses to make the course more relevant and accessible. It may also be helpful to establish a more universal set of prerequisites and cover more proofs to help students build a stronger foundation for advanced studies.
The survey provided valuable insight into the current state of introductory algorithm classes in the U.S. Hopefully, these findings can be used to help instructors improve their courses and better prepare students for careers in the computer science field.
Assessments & Equity
Kendra Reed, Curriculum Developer
During this year's conference, assessments and equity emerged as two of the most pressing issues facing the industry today. With so many different sessions on offer, it was interesting to see how these themes were woven throughout many of the discussions. Dr. Geoffrey Hermon was one of the standout presenters, delivering an insightful paper session on the psychometric evaluation of curriculum assessments. Specifically, he focused on cyber security and emphasized the importance of hands-on assessments, as opposed to lectures. By prioritizing the CCA and CCI test theory, Dr. Hermon was able to help learners fully grasp these concepts and shift their focus from simply getting the correct answers to understanding why some questions were wrong.
Dr. Hermon's session highlighted the need for greater attention to be paid to the assessment process in computer science education. His approach to hands-on assessments is particularly important given the increasing demand for cybersecurity professionals. By adopting these best practices, computer science educators and employers can help create a more equitable industry, one in which all learners have the opportunity to succeed. As we continue to grapple with issues of equity and assessment, it is clear that educators and professionals alike will need to come together to create more effective and inclusive learning environments.
Improvements in Online or Remote Learning
Jairo Velasquez, Customer Success
One of the key takeaways for me was the challenges online education faces nowadays and how much progress we have made. In the past, online education was the alternative to taking classes faster or a lighter version of a course. Today, online education has leveraged to a point where it can be as good (or even better) as its face-to-face counterparts.
"Grading, plagiarism, and automation” were words I kept hearing through different sessions. Research done online is helping predict how students think and what they do. Instructors want to spend more time creating new and engaging materials rather than grading their students' work. It echoed how using the right tool is the path to discoveries and innovations. It is about gathering data, analyzing it, and relentlessly improving. Not all courses have equal requirements, and students are finding new ways of challenging the traditional model; we should find new ways to challenge and engage them.
