So who is benefiting from all the strange referee decisions in the Premier League

The club by club review of referees’ performances during 2011/12 is almost complete. If you have missed it (and how could you?) the reviews are listed here.

We have three clubs to go: Manchester City, Manchester United, and Arsenal.

So as we are almost there the question is: what have we found so far? What should we be looking out for in these final few reviews?

First off, through our reports on refereeing performances in 40% of matches in the Premier League last season, each conducted by a qualified ref, we have confirmed in game after game, match after match, that the quality and standard of refereeing is awful. Way, way below that claimed by the PGMOL which runs the refereeing in the Premier League.

Second, we have found incredible variance between the performance of referees in relation to certain teams. Basically for some clubs one can clearly say, “if you have got Ref X, you are likely to be stuffed”.

But, and this is the big “but”, this doesn’t happen all the time. To put it crudely, you can have situations in which clubs are getting huge bias factors against them, and then suddenly everything evens up. This is not the same as the old BBC adage that “it all events out in the end” (a mantra repeatedly endlessly without the slightest bit of evidence), but rather it is to say that the bias against certain clubs switches on and off. Indeed, it seems, it switches on and off at key moments. If Newcastle are getting close to a top four position you might find some odd ref decisions against them. If Chelsea have slipped down the league and are not going to make the top four, you might find any bias against them vanishing.

Third, it is clear that although a referee can be biased for or against a team (in term of his decisions), that is not always enough to swing a match. You can’t see bias from results alone – if you want to look for oddities you have to analyse the individual decisions and sum them up.

Thus what we have not proven as yet is:

a) which club or clubs are benefiting from dubious ref decisions

b) the cause of these dubious referee decisions

c) whether this enormous survey encompassing over 40% of the league games in the Premier League last season was itself reasonable and fair.

On point a) I am going to leave you hanging on for a little longer. Two more reports will be published later today (2 August 2012) and one tomorrow and you will then be able to make a judgement. Just chill out, relax, have your lunch (if you are on UK time), and then all will be revealed.

But for the moment I am moving on to points b) and c)

The cause of these dubious referee decisions

i) Referees can make bad decisions because they are having a really bad day on the pitch (and every one of us who is in work knows what that is like, no matter what job we have). Refs are human, and they will just have days when it all goes wrong. It happens.

ii) Referees can simply not be as good as we (and PGMOL) would like them to be. On this point, we are at one with Alan Green, the BBC radio commentator on football matches. For years he has been criticising referees, although he has never looked consistently enough at the issue to draw any statistical conclusion (or at least he hasn’t done this on air). The implication of his commentaries is that the referees in the top league are not very good. He is very careful never to ask why.

iii) Referees can be influenced in ways that they should not be influenced

This third point only comes into play if we show that the dubious decisions made by referees are so far away from being random that they can’t have happened by chance. Put another way, if everything is down to having a bad day or simply not being very good, then everything should even up in the end. If it doesn’t we have to look for a more sinister explanation.

When we first started this whole idea of reviewing referees rather than matches we had several people writing in saying, “you have proved nothing. Unless you have a recording a chairman A saying to ref B, ‘here are the keys to the villa in Tuscany – use it whenever you wish’ then we have nothing.”

I have always disagreed with this. I come from a social science background, and in the psychology experiments I was involved in, and all those that I read which came from the eminent researchers across the world the issue was always that if one could show that there was less than a one in twenty chance of the result being due to random factors, then you could say that yes, it has been proved.

In football referee errors should be at a consistent level, varying at random because of the “bad day” factor. Beyond that, we have a problem.

iv) Or maybe it is PGMOL

The organisation that runs the professional referees in the Premier League (PGMOL) has refused to engage in discussion with us – as have the referee organisations across Europe. Indeed during the course of our investigation the PGMOL have moved from having a web site with information about refereeing, open to everyone to see, to a web site that you can only enter with a password supplied by PGMOL. Did this happen because of us? Who knows – maybe it was just a coincidence.

But let’s assume for a moment that the findings we have are real (and given that we are using a range of referees to do our analysis, and their work is then checked and is open for all to see that seems more than likely), if the explanation of the bizarre and eccentric variances in referee performances through the year is NOT that a referee has been bought, then what is it?

This is where PGMOL really should be coming out with some sensible commentaries. But apart from claiming a phenomenally high accuracy level in referee decisions, with the claim being backed up by no statistical evidence at all, they say nothing. “Everything is fine, don’t worry” is their patronising view it seems.

Yet, PGMOL are running their organisation oddly. There are far fewer refs than we might expect. There is a geographical bias to the referees. Some referees are given the same team to ref over and over again. In short: if PGMOL wanted to show that everything in the refereeing world was fine, it could at least address these points. But it does not, and it makes no response to ourselves.

iv) But maybe it is all a figment of our imagination. Maybe our referees are the ones who can’t see if the ball is over the line, if a penalty is a penalty, and if a foul is a foul. Maybe it is us. The only way to find out is to look for some other organisation that assesses referees. Not PGMOL because although they say everything is rosy and they give us no evidence. What we want is to find another totally independent body, and see what they say.

And guess what, we’ve found one. Details will follow shortly.

During this series of articles we’ve had a lot of correspondence. Many have thanked and supported us – not just Arsenal supporters but fans of other clubs. We’re grateful.

Of course many have written and called us every name under the sun. “Typical whining Arsenal fans”, summarises a lot of the complaints. Some have suggested that we are picking specific matches to prove our point (although amusingly most of the time when they say, “I bet you didn’t look at ” – we find did, it is on this site, and our analysis agrees with them).

We’ve had a lot of “get a life” comments and the like – you can imagine the sort of thing. But the interesting thing is that apart from the question of the issue of cause and effect (ie is it incompetence or deliberate bias), no one has put forward a coherent argument against our methods and approaches. People call us names, but no one says, “here’s the flaw in your argument”, other than demanding a level of proof far beyond that demanded for most other investigations.

This site clearly proclaims it is a football site from an Arsenal perspective. We are Arsenal fans, and when it comes to the matches we go to, we are as tribal and committed as any Tottenham, Chelsea, Wigan, Man U, Blackburn or any other fan. But we are also informed and educated enough to be able to engage with supporters of other clubs on matters of importance. Increasingly this year we’ve had really good debates with fans of other teams on issues that concern us all, not least because we clearly proclaim who we are.

And this is my main concluding point. If our final club reviews do show that there is a problem with referees in the Premier League, then there is no doubt that the overwhelming majority of clubs in the league are suffering because of that. If there is something wrong with referees in the Premier League, it is not just one club that is being done down. It is most of them.

For that reason, supporters of the majority of teams who are suffering in this way really should be coming together. If this is the case, we are all being screwed.

Most of all we’d like to see lots of supporter groups from other clubs coming together to do what we have done – although believe me it is a huge job. But until that moment occurs, I would thank all those fans of other teams who have appreciated our work.

What is being achieved by our referee reviewers is a revolution and its ramifications will echo through football in England for a long time to come. Of course, PGMOL and their allies will ignore us. But the sensational rise in readership of this site during the last year means that as the story pops out of the box, no one is going to be able to put it back in.