Operations in Output checking

There is lots of discussion surrounding SDC theory and output checking tools. However, there is relatively little discussion on what makes SDC management work. One exception is the ‘RRSA’ framework, discussed below.  
In March 2024 the DRAGoN team organised a workshop on the practical side of running output checking for a secure lab. Individuals from over 15 organisations attended, and helped to draft some ‘good practice’ guidelines as well as operating tips (summarised below). The workshop report can be found here. The resulting guidelines can be found here. The next section summarises the guidelines. 

Good practice report findings

Good practice guidelines:

  • Checking statistics, not complete outputs. Don’t check the quality, and if there is a third-party checking, be very clear about both your role and theirs.  
  • For TREs, check unless there are very good reasons as to why not. Reputation is as important as risk, and if resources are limited encourage random checks. 
  • Train researchers in principles-based checking. 
  • Direct trained researchers into positive limited behaviours, using selective submission and limited formats. 
  • Using four-eyes sequential checking, and two-eyes for code. 
  • Use the runners, repeaters and strangers’ model for planning (as outlined below) 
  • Checker training in researcher management, giving positive feedback, use of autonomy, informal communication.  
  • Effective IT systems are essential 

Having effective and strong operations creates a pathway for many benefits:

  • Building links with the wider community for group learning
  • Expertise of the SDC community
  • Building a self-supportive team
  • Founding a culture of continuous improvement
  • Using process metrics effectively

Runners, repeaters and strangers – operationalising efficient SDC

There are different types of customers that will interact with a business. We can classify these as runners, repeaters, strangers and aliens. Let’s use the example of a customer in a bank.  

  • Runners have a request that can be done without human interaction. This is what SACRO achieves. In the context of a bank, it would be an individual withdrawing money.
  • Repeaters know what is happening and how to do it, but need some human interaction, such as opening a bank account, which involves confirmations and utility bills.
  • Strangers require an experienced human. They are trying to open a bank account but do not have utility bills and are unsure of their last name.
  • There is a fourth category of aliens, who have entered the bank but are unsure why they are there and do not have a name or an address. These requests are so infrequent and obscure that pre-existing knowledge is generally not applicable.

When dealing with strangers, the issue that they have will arise and be dealt with, and once this happens there can be a protocol created to recategorise these individuals into ‘repeaters’ or to ‘runners’. Therefore, there needs to be an experiences statistician on staff to help deal with these issues and optimise the business interactions.  

Scroll to Top