Practically, aside theoretical considerations, the scale of the problem (N) is bounded, and this stopping rule can still be considered on some finite range. But it's important to control the Type I error for this quantity. My experiment starts here. I don't know the theoretical derivation for this case of this quantity, so I performed simulations. First observation: doubling the scale N gives a sensibly constant additional Type I error:
Code: Select all
N Type I error for t=2
from N to 2*N
500 7.92%
1000 7.96%
2000 7.87%
4000 7.96%
8000 7.90%
Second observation: the Type I error from doubling seems to follow closely the quantity Exp(-t^2/2). So the error goes pretty quickly to some small values with increasing of t-value:
Code: Select all
t-value Type I error Exp(-t^2/2) p-value
per doubling
5.0 0.00040% 0.00037% 0.000057%
4.5 0.0044% 0.0040% 0.00068%
4.0 0.033% 0.034% 0.0063%
3.5 0.18% 0.219% 0.046%
3.0 0.81% 1.11% 0.27%
2.5 2.66% 4.39% 1.24%
2.0 7.92% 13.53% 4.55%
1.5 20.23% 32.47% 13.36%
1.0 49.08% 60.65% 31.73%