Depending on the statistical power desired, a sample size of 7 or 8 is generally considered a good starting point for determining whether or not samples (a shot, or shot group in this case) are an accurate representation of the true mean. This is why 5 shot groups are a weak indicator of a barrels precision (it's true mean/population mean).
Great points Molon!