Tag Archives: refereeing

Incentivising refereeing

To shorten the refereeing lag and improve report quality in economics, the natural solution is to incentivise academics to do a better and quicker job. Economists respond to incentives, but currently no salary or promotion consequences arise from good or bad refereeing, as far as I know. In http://sanderheinsalu.com/ajaveeb/?p=503, I wrote about incentives for authors not to submit careless papers (in the hope that a refereeing mistake gets them accepted). One such incentive is requiring refereeing for a journal as a precondition for submitting to that journal. If a submitted paper gets n referee reports on average, then before submitting, an author should referee n papers in a timely manner, which should balance the supply and demand of refereeing. This forced refereeing may lead to quick, but sloppy reports.

An additional incentive is needed for quality. Rahman’s 2010 paper on the question „who watches the watchmen” suggests an answer. The editor can insert a deliberate mistake in every paper and see whether a referee finds it. If not, then the refereeing of that person is likely of low quality. The mistake is corrected before publication. Alternatively, the editor can ask each author to insert a mistake and tell the editor about it. The author is not penalised for this mistake and is asked to correct it if the paper is accepted. The referees are again judged on whether they find the mistake.

The above scheme derives refereeing incentives from publication incentives, requiring minimal change to the current system. However, it is somewhat indirect. A more straightforward incentive for refereeing is to reward it directly, either paying for it or basing promotion decisions partly on it. The speed of refereeing is already slightly monetarily incentivised in the American Economic Journal: Microeconomics. If the referee sends the report before a deadline, then she or he is paid 100 dollars. If a good referee report takes about 10 hours, then the amount is certainly not enough to motivate quality provision, but it is a step in one of the right directions. A simple improvement on the binary „before vs after deadline” reward scheme is to reduce the payment gradually as the delay of the referee report increases.

If refereeing is incentivised, then lower-ranked journals need larger incentives to compensate for the fact that refereeing for these has less inherent prestige and the paper one has to read is of lower quality. On the other hand, lower-ranked journals are less able to motivate refereeing with the threat of not accepting submissions from those who have not refereed. There are more lower-ranked journals, and it is less important to get accepted by any particular one of them. Some of the less prestigious journals would find no referees under the system proposed above. This is good, because it removes the „peer reviewed” status of some junk journals and may force them to close. If authors know that quality journals require refereeing before submission, then they draw the obvious conclusion about a journal that does not require it.