Let repeated samples reveal uncertainty.

For each of B bootstrap samples, compute the mean. Their standard deviation estimates the standard error of the mean. To form a 95% percentile interval, take the 2.5th and 97.5th percentiles of the bootstrap means. The endpoints may change slightly when you change the random seed.

As a mechanical illustration, the original sample 4, 6, 8, 10 has mean 7. One possible resample is 4, 4, 8, 10, with mean 6.5. Another is 6, 8, 10, 10, with mean 8.5. Thousands of such means form the bootstrap distribution; two resamples alone are far too few to estimate its tails.

If the 2.5th and 97.5th percentiles from a run are 5 and 9, the percentile interval is [5, 9]. Its intended interpretation concerns approximate repeated-sampling coverage of the procedure. It is not a 95% probability distribution for a fixed unknown population mean after the data have been observed.

For two groups, choose the statistic D = mean(A) − mean(B). Under the null model used here, group labels are exchangeable: independent observations come from the same population distribution. For an appropriately randomized experiment, the assignment mechanism can justify a related randomization analysis under a sharp no-effect null.

Compute D for each random reassignment. A two-sided result is at least as extreme when its absolute difference is at least |Dobserved|. This lab uses that absolute-distance convention and a small numerical tolerance for ties. Other software can use twice the smaller one-sided tail; these conventions need not agree for asymmetric discrete null distributions.

Let b of B random shuffles be at least as extreme. Report (b + 1)/(B + 1), including the observed allocation in the adjustment. For b = 39 and B = 999, the p-value is 40/1000 = 0.040. Even if b = 0, the simulation does not justify p = 0; its smallest reported p-value would be 0.001.

Increasing B gives a steadier computational approximation. It does not increase the original sample size, fix selection bias, or necessarily shrink the population confidence interval. Changing the seed reruns a comparable simulation; keeping it fixed makes classroom demonstrations reproducible.

For a permutation p-value near p, a rough Monte Carlo standard error is √[p(1 − p)/(B + 1)]. If p ≈ 0.05 and B = 1,999, that error is about 0.0049. A simulated answer close to a decision threshold may need substantially more repetitions than this teaching explorer supports.

The bootstrap standard error shown in bootstrap mode describes variation among resampled means. The Monte Carlo standard error shown in permutation mode describes numerical uncertainty in a tail-probability estimate. These are different quantities and should not be substituted for each other.

A bootstrap cannot recover population features absent from the observed sample. Tiny samples, extreme skewness, influential observations, and a degenerate sample can make percentile intervals inaccurate. More advanced intervals such as BCa address some limitations; the transparent percentile method here is a teaching tool, not a universally best interval.

Dependence also changes the correct resampling unit. Paired data should retain their pairs; repeated observations within a cluster may require cluster resampling; time series may need blocks. This lab implements independent observations only.

For the unstudentized permutation difference used here, equal means alone are not enough to guarantee validity when group distributions differ, especially with unequal variances and sample sizes. Use a method that matches the design and the null being tested. Describe conclusions in terms of that model, rather than calling resampling assumption-free.