The Question to Ask First

Institutions usually arrive at proctoring through a procurement process that starts with products. The question below starts earlier. Before selecting a proctoring tool, administrators can use the Monitask overview as a contrasting example of how monitoring scope should follow a specific operational question.

The question

What are you protecting, and from what?

Named specifically, for this assessment, rather than as a general commitment to academic integrity.

What the answers point to

"The qualification's credibility with employers and regulators." A real and serious answer, and it points at professional and licensing exams where proctoring is most defensible.

"Fairness to students who did not cheat." Also real, and it points as much at assessment design as at surveillance.

"An accreditation requirement." Then read the requirement: it frequently specifies an outcome rather than a method, and alternatives may satisfy it.

"We moved online and did what we did before." The commonest honest answer, and it is a reason to reconsider the assessment rather than to reproduce the exam hall.

"Reputation if a story appears." A legitimate institutional concern that should be stated as such, because it leads to different choices from the others.

The second question

What would you do if the system flagged five hundred sessions?

If the answer is that you could not review them, the programme is not resourced and the flags will be handled badly.

Review capacity is the binding constraint in every implementation, and it is usually discovered after purchase rather than before.

The third question

What happens to a student who is wrongly flagged?

If there is no answer, there is no process, and the note on appeals covers why that is the part actually being bought.

An institution that cannot describe this has bought a detector and not a system.

Using them on a procurement

Ask them of the assessment, not of the product.

Most procurement documents in this area specify features. Almost none specify the review process, the appeal route or the review capacity — which are the parts that determine whether the programme works.

Write those into the requirement before looking at suppliers.

What frequently comes out of it

That some assessments need it and most do not.

That the ones that need it are the ones with external consequences: licensure, professional registration, credit transfer.

And that for the rest, changing the assessment is cheaper, more defensible and better educationally — which the alternatives section covers.

The collection in one line

Decide what the assessment is for, work out whether surveillance is proportionate to that, and if it is, design the human process before choosing a supplier.

Everything else here is the detail of doing that.

The point

Write the review process, the appeal route and the review capacity into the procurement requirement before looking at suppliers.

Almost every specification in this area lists features and omits the parts that determine whether the programme works.

What usually comes out of it

That some assessments need it and most do not.

The ones that do are those with external consequences: licensure, professional registration, credit transfer. For the rest, changing the assessment is cheaper, more defensible and better educationally.

Worth stating

Review capacity is the binding constraint in every implementation and is usually discovered after purchase.

Ask what you would do if five hundred sessions were flagged; if the answer is that you could not review them, the programme is not resourced.

Also worth knowing

Frequently the real question is not about time or cheating at all.

Ask what would change if you knew, and if the honest answer is nothing, the procurement is standing in for a decision somebody has not made.

And finally

Most procurement documents specify features and omit the review process, the appeal route and the review capacity.

Those three determine whether the programme works, and they should be in the requirement before any supplier is approached.

Summary

Read the accreditation requirement rather than assuming it: it frequently specifies an outcome rather than a method, and alternatives may satisfy it.

In summary

Ask the questions of the assessment rather than of the product, and write the answers down before any supplier is contacted.

The procurement that starts with a demonstration has already decided. For wider institutional context, consult Jisc.