Raymond Spiteri, Mar 3, 2021
University of Saskatchewan

The COVID-19 pandemic has forced a re-think of many of the ways in which we deliver our courses at the university. One aspect that has arguably not received the attention it merits — especially given its traditional importance in many subjects, not just mathematics — is how to effectively administer virtually invigilated examinations.  I have administered virtually invigilated examinations in three different COVID semesters with class sizes up to approximately 50 students. This article relates some of my personal experiences and self-identified best practices with recreating the “old-normal” invigilated examination experience virtually for university students and maintaining trust in the integrity of the university degree.

Photo of exam writing by Jeswin Thomas on Unsplash

But maybe we should first start with the question: Why do we care about the concept of “virtual invigilation” anyway? The “virtual” part is clear. Until university campuses generally re-open to in-person attendance, there is no getting around it.

Sadly, it is also clear why we care about the “invigilation” part: cheating. Cheating is not a new thing, and at least awareness of it has generally been on the rise in the internet era. But the move to online learning seems to have taken cheating to a whole new level. For example, since the beginning of the COVID-19 pandemic alone we have had incidents such as allegations of over 100 cases of cheating in a first-year mathematics exam at UBC, “compromised final exams” affecting almost 50 second-year nursing students at Brandon University and an unknown number of first-year chemistry students at SFU, and at least 20 students in a 200-level mathematics course at Princeton who fell prey to a teaching assistant who posted false solutions to catch cheaters.  At the University of Malta, the Faculty of Economics, Management, and Accountancy was ready to fail an entire first-year class of students after discovery of widespread cheating in an online examination.  Perhaps even more disturbing is the feeling of growing suspicion that the problem is much deeper than the hard evidence suggests.

We also care about the invigilation part because it may be the only scenario in which assessors can be at all confident that what they are seeing is genuinely and uniquely the work of a given student. In other words, the invigilated examination has traditionally been the great equalizer. No matter what happened during assignments, projects, etc., the invigilated examination had the power to distill what a given student has to say in relation to course material without the distraction of what could be someone else’s work. It can be fine (even desirable) for students to collaborate on assignments or projects; collaborating on exams is usually not cool with instructors or universities.

Sidestepping the question of the appropriateness of examinations in the first place, if we agree on the merits (if not the necessity in some scenarios) of invigilated examinations, the next and more practical question is: how do we administer virtually invigilated examinations?

Photo of student at home by Max Shilov on Unsplash

First, I will state my firm belief that the vast majority of students (>90%) do not set out to cheat. But they are human and suffer from temptation. So the first basic principle to creating a successful virtually invigilated examination is to remove the opportunity to cheat. Students not only welcome a structure that allows them to focus on their exam, but they especially welcome the relief in the certainty that others are not cheating. So these best practices create a trusted environment in which it is easy for students to be true to their academic integrity.

The second basic guiding principle to successfully invigilate a virtual examination is to mimic the in-person experience where possible. The in-person experience, again with all its warts, has withstood the test of time and is generally viewed by students and examiners alike as acceptable. This guiding principle leads naturally to a principle of parsimony. Not making the virtual invigilation process an additional big deal in terms of technology required or stress induced is a key to success.

Here is what my virtually invigilated examination looks like. Students are asked to arrive 10 to 15 minutes early for the exam. About 5 minutes before the official start time, they are randomly assigned to breakout rooms. A TA also joins each breakout room. The required number of breakout rooms can be determined ahead of time along with the TA room assignments. Students with equity or other considerations such as being allowed more time each have their own breakout room. The numbers so far have been such that these students can be invigilated by multiple instances of me or a dedicated TA.

Ultimately, the number of breakout rooms may be determined by how many TAs are available, but in order for this process to proceed most effectively, TA micro-invigilation sessions should be limited to no more than around 10 students per TA. As the instructor, I am essentially on call in the main room for general-purpose trouble shooting, e.g., students arriving late, students having to rejoin, a TA who has to leave briefly, or other situations in breakout rooms that the TA is uncomfortable handling.  As soon as it is reasonable after entering the breakout rooms, each student shows their student card to the TA via their camera.

The requirements I have used are that the students have a working camera and that they have it display their writing surface with their hands visible. Other than that, the students have the control to set their camera up to show how much or how little of whatever else they want. Blurred or esoteric backgrounds are fine. This situation is in many ways less invasive than being in public (not that many of us even remember what that is like anymore). This approach is in contrast to other proctoring services (for example, that monitor computer activity or use eye tracking), which are significantly more invasive and generally engender mistrust. As best practice, students are asked to be muted and ask questions of the TA through chat. If the TA cannot resolve any issues in this manner, the student can go to lobby to ask their question of the instructor orally.

Students are strongly encouraged to connect only once to the virtual session. Otherwise, moving them to breakout rooms can become tricky. They are also encouraged to write on paper and not some form of electronic device. This potentially frees up a device with a camera that can be used in the session or as back up in case of unexpected issues. The general guiding principle here is that students should prepare ahead of time for the exam. A checklist of things to help them do so is also provided.

Students are advised that the micro-invigilation sessions are recorded by the TAs (as with all lectures in the course). These recordings are only meant as an impartial record of events. They are not shared, and they are destroyed after the course ends. They are only used in case of “irregular behaviour” to potentially facilitate a reasonable resolution.

In general and at the discretion of the instructor, the consequence of irregular behaviour is an oral examination with the instructor (in this case, me), where again another judgement is made whether anything untoward has occurred, and if so, what appropriate steps to remediation might be. It is clear no one will willingly choose this outcome if it can be avoided! I have yet to hold an oral examination due to irregular activity in almost 100 virtually invigilated examinations, so I cannot comment further on this untested aspect of the process.

It is important to note that the entire virtual invigilation process is primed with a generous amount of empathy, understanding, and patience in order to ensure its robustness. For example, although strictly speaking students are to keep their hands visible at all times, the policy for enforcement is not zero tolerance. Even though it would be rare to have a complete internet outage, it is equally rare for no one to have connection issues. Although it may sound ominous, the oral exam is another way to allow human judgement (again with all its frailties) to address “irregular behaviour” in a fair and appropriate manner.

At the end of the exam, the students sign a declaration of academic honesty included as the last page of their exam and finally take photos of their work and upload them to a learning management system, ready for marking. They are then free to leave the session.

A fair criticism of this approach may be that there is an element of presumed guilt until innocence is proven. However, as with many relationships, it comes down to trust. Trust that this is more about ensuring fairness and protecting the integrity of the degree and what it stands for rather than assuming the worst in human behaviour. Virtual invigilation will become much more commonplace once all parties involved trust that the system works.

I am happy to report that with no exception thus far (knock on wood), the students had no real problem with the process. From the point of view of the invigilator, I have no evidence of academic misconduct.

Am I saying this is a perfect system? Certainly not. Any system can be abused in multiple ways by those determined to do so.

And maybe I have just been lucky and have unwittingly beaten long odds. Any virtual exercise is always vulnerable to widespread internet outages. However, at some point, risk becomes quantifiable and maybe even acceptable, especially if there are backup plans. We will see because I plan to keep using virtual invigilation in all my courses. The concept may (hopefully?) be moot come September 2021. But maybe it will be a vestige of the pandemic that persists.

In summary, I believe virtually invigilated examinations are possible — as with many other complicated endeavours, it helps to have and share of best practices as well as being organized and mobilizing sufficient resources —  and there are many advantages to them — it can be a win-win-win for students, instructors, and universities — all three entities can navigate through a stressful exercise with their dignity and integrity intact.

Photo of graduation by Pang Yuhao on Unsplash