Save money & improve scores - real examples
Tests were carried-out using a B25 with 1/2 choke (Teague long choke) at 25 yards distance. Again, this distance was chosen to ensure almost all of the pellets were captured.
The shells tested were:
For the sake of thoroughness the average pellet count of each cartridge was found to allow traditional measures of pattern efficiency to be calculated if needed. This produced the first surprise. According to the Eley Shooter's Diary a 28g load with No.7.5 (English) should contain approximately 400 pellets. The three-shell average was found to be 417.7, 396, 398.7 and 291 for the Express HV, ProOne, DTL300 and World-Cup respectively. The World-Cup shot size was actually nearer to 6/6.5 shot - this makes direct comparisons regarding pellet coverage pointless because they are effectively different shells best suited to different distance and choke scenarios. However, consistency in terms of variation in the spread can be compared.
The second big surprise came regarding the Express HV shells. The first four targets were shot about four weeks before the last six and seemed to be completely different shells! The first batch had ~417.7 pellets as counted by hand, the second batch had ~350 pellets as captured by the pattern plate. Again referring to the Eley Shooter's Diary, ~350 pellets is the load expected of a number 7 sized shot load. Because of this the two groups of results have been kept separate.
Some salient points from the above data:
Two 24g 7.5 loads were also tried:
Those who are recoil sensitive might like to consider 24g loads for sporting. Some people attach mythical powers to 24g loads due to the shorter shot column. Now this can be investigated.
Again, the average pellet content was counted by hand from three shells. And again, the 'premium' shell used oversize shot. The Eley Diary suggests ~335 pellets in a 24g 7.5 shell. The Hull Chevron yielded 331.7 pellets but the Gamebore Patriot only 260.7. This was a shame because the Gamebore had the most beautiful shot of all the shells tested. The pellets were polished, seemed "non-stick" and were noticeably better formed. They were almost like ball-bearings! As will be seen the performance was good and in due course No.8s will have to be tested to see how the shell performs with smaller pellets. This is important because it could simply be the larger shot deforms less and this gives tighter and more consistent patterns rather than the powder/wad/shot combination being particularly suited to the gun. The Patriot also had a lovely recoil action - crisp but not heavy.
The above results show that the Patriot load groups significantly tighter but there is not much difference in repeatability of spread (as a percentage of the average spread). Beyond that, it's difficult to compare the two other than to say that at distances where true 7.5 shot will reliably break a clay the Chevron load will have a higher likelihood of a kill largely due to the higher pellet count. In fact, bearing in mind the difference in cost the Chevron looks really good.
See sporter shells ongoing sumary for the latest status of testing for sporting shells!
A calibration target can be used to confirm the measurement process. Shown below is a screen-grab of a hand prepared target with seven points each separated by 5" aligned horizontally and seven points aligned vertically (also spaced by 5") with the centre point of each line coincident.
The true value for the spread (calculated by spreadsheet) is 7.64". There is a small change in the reported results dependent on where the scale markers are placed. This is due to the distortion caused by the digital camera. This is worst at the extreme edges/corners of the target. The last of the 'Pattern statistics' tables shown above is for the case when the right hand side of the paper is used as the vertical scale. The camera 'pinch' at the corners of the target is the worst case. The reported vertical spread of 7.82" is 0.18" (2.35%) too high. This is tolerable in the context of shotgun patterns. Furthermore, since the typical user will be comparing one shell against another, if the scale points are taken from the same position on each target the relative error will be much smaller. Alternatively, scale markers could be added to the target in-board of the edge of the paper to improve the absolute accuracy - that is if knowing the pattern to better than 0.2" really is a requirement!
More data is not necessarily good data! Having got the ability to measure the pellet placement, what indicates that one shell works better than another? First, one needs to understand how the pellets are distributed - the Normal Distribution; second, one needs to understand the outputs from Shotgun-Insight and finally one needs to appreciate statistical significance.
The following terms used in the average results panels shown in the earlier screen shots will also be explained:
The Normal distribution features strongly in this analysis of shotgun patterns. The picture below illustrates its main features:
The vertical axis of the graph shows the relative probability of an 'event' happening versus its distance from the mean (average) which in this example is shown as zero. It will be seen that the likelihood of an event occurring is greatest nearest the mean. The distance +/- "1-sigma" accounts for approximately 68% of events, the distance +/- 2-sigma for 95% of events and the +/- 3-sigma range accounts for 99.7% of all events. This type of distribution model fits many naturally occurring observations, for example, heights of people, test scores etc. Oberfell and Thompson also found that the average pattern distribution from a shotgun could be modelled by the Normal distribution. It is important to note that the Normal distribution does not define the pellet distribution, it is just a model that fits the observed distributions quite well. The closeness of the pellet distributions (shown by the histograms of Shotgun-Insight in the Average Results Panel and Analysis Graphic) and the ideal Normal distribution should be convincing enough that the Normal distribution is a useful tool for characterising the pellet distributions.
The value 'sigma' is used to indicate the width of the distribution. The value of sigma can be used to compare one shell against another for a given choke to see which shell delivers the tightest pattern and hence which will be able to carry a useful pattern the longest distance. Alternatively, skeet shooters or fixed choke users may be interested in obtaining the widest spread possible.
To demonstrate that the shell and gun combination work well together, the patterns should be as repeatable as possible in much the same way as a rifle shooter looks for minimum variation shot to shot. The big difference is that rather than measuring a single distance (the maximum spread of a group of rifle bullets), the shotgunner's measure of repeatability is how all the pellets behave together. One way of doing this is to fit all the pellet positions to a model and then look at how the parameters of the model vary. In this case the Normal distribution is employed as the model.
A measure of repeatability of the pattern is given by how the spread - sigma - varies from shot to shot. Perhaps surprisingly to the uninitiated, the distribution of the values of the spread (or sigma) is itself a Normal! (If you are interested do a Google search for the phrase 'Central Limit Theorem' with a few words like 'explanation' included) Thus we can have a 'sigma' value for the variation of the spread of the pellet distributions as measured shot to shot! However, even when using perfect computer generated data there is some difference between successive measurements. Considering the flipping of a coin, although the long term average will be 50:50 heads to tails, a sample of 10 flips may yield something quite different. Statistical significance is a term used to describe how confident one can be that the measured result is genuine and not a statistical outlier.
Spread x/y: This is the 'sigma' value of a Normal distribution made to fit the real pellet distribution as closely as possible. From the above it is seen that a spread of 5" means approximately 68% of the shot falls within +/-5". The pellet spread is described in terms of horizontal and vertical spreads just in case there is a systematic problem with the gun and the spread is significantly larger in one dimension than the other. The '75% shot diameter' converts the spread x/y values into a diameter that on average would contain 75% of the shot. This is a convenient single number that can be used to summarise the spread.
Probability of hit (0-10", 0-20", 0-30"): This is the likelihood that the clay would be hit by at least one pellet. 0-10" considers only a 10" diameter circle about the centre of the pattern. A consistently good shot who centres most of his shots would want to see as high a likelihood of a hit in this region as possible, perhaps at the expense of a poorer probability of a hit further away from the centre. 0-20" and 0-30" are the same but consider larger diameters. A poor shot or someone shooting game that follows a very unpredictable flight might be more interested in a good coverage across the whole 30" diameter circle and be prepared to trade some coverage in the centre 10" to achieve this. The spread in the average values of each of these figures is also given. Obviously 100% with a sigma of 0.0 is the best one can get! An average hit probability of 95% with a sigma of 1% is better than one with an average of 95% but a larger sigma.
Centre weighted sum: The 0-30" probability of a hit treats all areas of the 30" diameter circle equally. An area with no pellets has the same affect on the hit probability figure regardless of whether the pellet free area occurred in the centre or on the edge of the pattern. Instinctively, one can appreciate that this is not right. If the shooter centres the pattern on the target he expects the pattern to do the job. It is not unreasonable to think that the centre of the pattern is 'worth more' than the edge. The 'Centre weighted sum' takes the 0-10" hit probability, the 10-20" hit probability (not shown individually in the results screens) and the 20-30" hit probability (also not shown explicitly) and gives the simple average of the three. Because the centre 0-10" area is one third of the 10-20" area and one fifth of the 20-30" area the simple averaging of the hit probabilities weights the centre much more than with the 0-30" figure. An alternative way of interpreting this number is that for a shooter who places the target in the centre 10" of the pattern 1/3 of the time, the 10-20" region 1/3 of the time and the outer 20-30" area of the pattern 1/3 of the time, the 'Centre weighted sum' shows the average effectiveness of the shell for this shooter.
Centre weighted norm: This weights the centre even more heavily than the centre weighted sum. The centre is weighted at ~68%, the 10-20" annulus at ~27% and the outer annulus at ~5%. The rationale for this is: If one thinks of a rifle shooter shooting at a bulls-eye target, the distribution of shots about the centre of the target will be approximated by a Normal distribution. That is, the natural variation of the shooter also leads to a Normal type distribution of the shot. Assuming the shot-gunner is the same, the relative weighting of the shot pattern should reflect the likelihood that the clay/bird falls in that part of the pattern and this relative weighting is the Normal distribution (again!). I assume that the shooter can keep 99.7% of his shots within the 30" circle (a tall order but probably not far off for good trap and skeet shooters). For good shooters of these disciplines who get high scores the 'Centre weighted norm' shows how the shell supports them. For these disciplines where high scores are common and needed to win, the centre of the pattern is obviously very important - hence the 68% weighting!
'sigma', '+/-' or 'shot to shot variation': For all of the averages the standard deviation or 'sigma' is calculated. Once again it is assumed that the distribution of the values being averaged follow a Normal. An average hit figure 90% with sigma of 2% means the average of the shells is 90% and ~68% of the shells fall within +/- 2% of this. Clearly an average of 90% with a sigma (or shot to shot variation) of 1% is better than an average of 90% with a sigma of 5%. The +/- or sigma value for the averages gives an indication of how repeatable the performance is.
Firstly, statistical significance is a tricky concept to get one's mind around. It's not needed for basic analysis but will be needed to decide finer points such as whether one choke design is better than another.
Consider the flipping of a coin. We all know that in the long run over many 100s or 1000s of flips, half the time it will land heads-up and half the time tails-up. However, for a smaller sample, say 10-flips, a quite different ratio of heads to tails might be obtained. Further 10-flip samples will give different ratios. It does not mean there is anything wrong with the coin, flipping or our counting, its just down to chance exactly what we get with 10-flips. In the long run if all the 10-flip samples were averaged, they too would give 50:50 heads to tails. In the short term there will be some variation about this.
Now consider the spread of a shot-shell. We know that in the long-run the distribution of pellets about the centre of the pattern can be described by the Normal distribution with a width 'sigma'. In the short term our estimates of this long term sigma value will tend to vary a bit. Again, it does not mean anything is wrong, it is just what happens when a small sample is used to estimate a long-term trend.
In the 'Average cartridge results' display Shotgun-Insight reports the average spread of the shells and also the 'shot-to-shot' variation about this average. The variation is expressed as the '1-sigma' value. For example, if a sample of shells had an average spread of 5.7" with a 1-sigma value of 0.45", this means 68% of the shells should have a spread within +/-0.45" of 5.7".
As more shells are tested the estimate of the average spread becomes more reliable. Intuitively this is obvious since averaging more shells will tend to reduce the effect of a 'one-off' bad shell.
Now there is a little bit of maths to quantify the benefit of testing several shells: The 1-sigma uncertainty in the average is given by the shot-to-shot 1-sigma value divided by the square root of the number of shells tested. If nine shells were tested for the example given above, the average would be estimated as 5.7" +/- 0.15" (1-sigma). This means that for each subsequent group of nine shells tested 68% should have an average within 5.7" +/-0.15". If we averaged 100 shells, the estimate of the average spread would be 5.7" +/-0.045".
It is not proposed for one minute that anybody average 100 shells. But the key point is if you want to have a very accurate estimate of the long term width of the spread of a pattern you need to average a (possibly large) number of shells.
In practice you probably do not need a very accurate estimate of the long term spread. If you find that two different shells have a very similar spread based on averaging a few of each, you will (correctly) decide that the difference for all practical purposes is irrelevant.
Statistics can help when you want to decide if two different shells are 'genuinely' different given the data available. Recall that for a given event or group under study the '+/- 1-sigma' value accounts for 68% of the occurrences. The '+/- 2-sigma' value accounts for 95% of occurrences. If we take the 2-sigma value, we can say that an event occurring just beyond this limit has a 5% chance of belonging to the first family of samples, or we are 95% confident it does not. This is known as the 2-sigma or 95% confidence limit. Similarly there is the 3-sigma or 99% confidence limit. This means that if the average of one group of shells is 2-sigma (sigma found by using the shot to shot sigma divided by the square root of the number of samples) away from another average, we can be 95% confident that the difference is genuine and can be repeated.
This is statistical significance. You may have heard of it in the news, now you know how to calculate it.
These are powerful concepts but do not worry if it seems a bit opaque! Many an engineering student has been rent asunder by statistics. For the most part, simply reading the average spread and '1-sigma' values from Shotgun-Insight along with some practical sense will suffice.
Eley Shooter's Diary. Eley Hawk Limited. www.eleyhawkltd.com
(c) Dr A C Jones