The editorial to this Volume ends with the hope that this journal can be part of a process that builds our understanding of speaking, judging and organising tournaments. The premise of all this – of the whole enterprise of this journal – is that what we do – whether that be speaking, organising, judging or all three – is difficult: that these are complex challenges which reward considered reflection and our collaborative effort through writing like this.
All of us who have done it – or watched others – know that speaking is difficult; most of us know that practice and reflection makes a difference. The same goes for running tournaments – the round of applause for convenors at the end of competitions confirms our mutual recognition of the challenges involved. These facts require no emphasis.
Here I make the modest claim that judging is similarly difficult. This is intended partly as an injunction to myself and others to continually reflect on what we do and thereby improve, but also as a response to recent currents in adjudication academia and practice that argue for, or implement, laypeople judging debates. [[1]. By ‘laypeople’, I mean ‘individuals without familiarity with competitive debating and, in particular with adjudicating competitive debates’. I explicitly do not include novice judges who are in the early stages of gaining this familiarity, whose involvement in judging I hold to be unquestionably essential.]
This, to me, can only be a good idea if we believe those laypeople have the skills to judge those debates, which can only be the case if those are generic, widely-held skills. I believe that is not the case, as judging is, almost without exception, extremely difficult.
This is not an argument against having inexperienced judges on a panel – this is a valuable aspect of judge development, and is to be welcomed. Rather, this is intended as a refutation of the fetishising of the ‘layperson’ and a realistic presentation of the skills judges all need to continually develop.
Following debates
Following a high-level debate requires rapid understanding of complex concepts as they are explained, and an ability to identify the relevant aspects of what is being said in order to record them. It then requires note-taking far more rapid than that generally required in, for example, a business context.
It may be pointed out that debaters are supposed to make themselves readily understood to the ‘ordinary reasonable voter’ (ORV) – surely expecting any specialised skill to follow them jars with this? This is somewhat true: we need not ask judges to understand more than the ORV would. What we do ask of them, however, is that:
They make sufficient record of the debate to adjudicate it. This means quickly identifying the key elements of what someone says, noting each analytically-distinct line in often-complex argumentative structures, and writing it down at quite some speed. This is hard. We can tell that by the very fact that experienced judges quickly develop shorthands and systems for doing it to enable them to judge to a sufficient standard. The ORV does not do this, and does not need to – she can be persuaded or unpersuaded by arguments without it. By contrast, few would claim judges needn’t keep sufficient record for a reasonably detailed discussion of the debate.
They remember exactly where lines of argument arise. Adjudications are influenced not only by how persuasive people are, but by which team gets credit for which argument or minute line of rebuttal. Particularly when comparing teams of the same bench, this can be vital. Said another way, being an able assessor of who has best persuaded the ORV is a different task to being the ORV.
Without compromising our commitment to judges judging as the ordinary reasonable voter, then, we should require a special skillset of rapidly following & recording a debate. Competent non-specialists cannot be expected to have these skills.
Applying a complex ruleset
The rules of debating are very complicated. We underestimate this because so much of this ruleset becomes internalised through practice. These rules are not adequately summarised by ‘which side you agreed with’, or even ‘who was most persuasive’. Knowing, for instance, that the debate is not about the motion, but about opening government’s definition thereof, or that we are not supposed as judges to simply discount what we deem ‘bad arguments’, are as vital to correct adjudication as they are non-intuitive to laypeople.
It is not enough to correct lay adjudicators during deliberations, because mistaking elements of the ruleset like these alters the whole way you watch a debate. Nor is it simple to simply tell a first-time judge what these rules are and expect them to adjudicate according to them straight away – if it were, judges who had attended briefings would always perfectly fulfil those guidelines. Experience suggests that is not true, intuition that it would be grossly unfair on them to expect it!
Acting as the Ordinary Reasonable Voter
In my view the hardest aspect of judging BP debates is alienating yourself from your own preconceptions. Judging not as any one individual but as an abstracted person who does not have prior opinions on the topic at contest, and who represents no particular section of society or set of experiences, requires conscious and continual effort. We get better at this by trying, by being called out on our mistakes by teams and other judges, by reflecting on this and finding ways to improve. I know that I, as a judge, read into debates now much less than I did when I started judging seven years ago, and I can track this to particular efforts I’ve made to avoid my preferences interceding in adjudication – efforts I know I need to continue.
Adjudication means only avoiding siding with the team you agree with, and equally avoiding allowing your own prioritisation of arguments (which can seem ‘just true’ to you) to influence what you reward. This is really quite distinct from efforts to be disinterested or judicious elsewhere in society. It is not just about avoiding your own personal interests impacting your judgement of what is right: it is trying to stop your judgement of what is right impacting on your evaluation of the process of disputation. Outside debating, we spend so long judging what is right that it can be difficult to avoid entering debates as a judge – adjudication asks us to think in challenging and novel ways.
While it might be appealing, then, to say the ‘ordinary reasonable voter’ standard makes lay judging a good thing, in fact judging as the ORV is very different from getting a particular ‘reasonable voter’ to make the call. Moreover, we often have debates about things where there is, in fact, a strong social consensus one way or the other; this would be represented in a group of actual reasonable voters. Our ORV does not, by contrast, come in with any particular view. As judges, we learn how to judge as that – imaginary – person.
Following debates, applying complex sets of rules and acting as the ordinary reasonable voter takes a particular skillset and reflective practice. Lay judges are unlikely to have developed the former, and cannot access the latter. Of course, it we find a layperson with those requisite skills, we should assess them just as we would a non-lay judge. What we should avoid – in academia or practice – is claiming that lay judges are suitable because they are laypeople. Certainly, sometimes a cash-strapped tournament may have to make a compromise by having lucrative sponsors judge its final, or somesuch – far be it from me to say they are wrong to make such difficult decisions. In general, however, we must recognise the technical challenge of judging debates, and select our adjudicators accordingly. This might mean being honest that what we are doing is no mimesis of public discourse; but it will mean that we do what we are doing a good deal better.