The paper presents different measures of measuring reliability and which one is better than the other.
But it made me think about one particular question which is asking how much reliability we should expect when we scan the
subject again in the next couple weeks or so. Here if we expect high reliability we are excepting (prior hypothesis) that every time we do a certain task in different occasion we are in same mental state. Therefore, we are eliminating the effect of familiarity and novelty when we experience similar situations. I want to draw attention to this, this can not explained by internal physiological noise, if the underlying neural processes are dependent on internal states (mood, feeling, novelty and so on). Nowadays researchers use mock fMRI trials in fake scanners to decrease the level of anxiety. Furthermore, if the general computation mechanisms underlying our neural machinery is high dependent on the novelty as Karl Friston suggested by his unified theory of brain (free energy) model, how much does novelty effects the retest reliability is I think needs a new mathematical model not present in the paper.

I am very glad you liked the paper. You bring up a lot of valid points in your comment. As you state, we must acknowledge that no two measurements will every be precisely the same. This variability can come from a wide variety of sources, including task learning, stimulus novelty, and changes in baseline mental state. We can try to control for many of these factors, but certainly not all. New studies to investigate the influence of these factors are necessary, as are new approaches to the quantification of reliability. ~ Craig [Prefrontal]