Baez-Mendoza et al (2021) claim that for rhesus macaques choosing which of two others to reward in each trial, „the difference in the other’s reputation based on past interactions (i.e., how likely they were to reciprocate over the past 20 trials) had a significant effect on the animal’s choices [odds ratio (OR) = 1.54, t = 9.2, P = 3.5 × 10^-20; fig. S2C]”.

In 20 trials, there are ten chances to reciprocate if I understand the meaning of reciprocation in the study (monkey x gives a reward to the monkey who gave x a reward in the last trial). Depending on interpretation, there are 6-10 chances to react to reciprocation. Six if three trials are required for each reaction: the trial in which a monkey acts, the trial in which another monkey reciprocates and the trial in which a monkey reacts to the reciprocation. Ten if the reaction can coincide with the initial act of the next action-reciprocation pair.

Under the null hypothesis that the monkey allocates rewards randomly, the probability of giving the reward to the monkey who previously reciprocated the most 10 times out of 10 is 1/1024. The p-value is the probability that the observed effect is due to chance, given the null hypothesis. So the p-value cannot be smaller than about 0.001 for a 20-trial session, which offers at most 10 chances to react to reciprocation. The p-value cannot be 3.5*10^-20 as Baez-Mendoza et al (2021) claim. Their supplementary material does not offer an explanation of how this p-value was calculated.

Interpreting reciprocation or trials differently so that 20 trials offer 20 chances to reciprocate, the minimal p-value is 1/1048576, approximately 10^-6, again far from 3.5*10^-20.

A possible explanation is the sentence “The group performed an average of 105 ± 8.7 (mean ± SEM) trials per session for a total of 22 sessions.” If the monkey has a chance to react to past reciprocation in a third of the 105*22 sessions, then the p-value can indeed be of the order 10^-20. It would be interesting to know how the authors divide the trials into the reputation-building and reaction blocks.