Just About Right Scales and Penalty Analysis
Just About Right (JAR) scales are a commonly used question format when trying to identify the performance of a product or experience against a certain attribute. The scale assumes that there is an ideal position for an attribute, and the possibility of being over or under the ideal. For example, the sweetness of a chocolate bar could be Just About Right, or it could be either too sweet, or not sweet enough. To quantify this, we create five unique positions on a 5-point scale ranging from much too sweet, to not at all sweet enough for example (Figure 1).
When answering the questions, the respondent can only select one of the five answers, making this a single code question.
At the end of the study, you will be able to create a mean score on this data, finding your average position in relation to being just about right. A mean close to zero will illustrate whether you are in line with consumer expectations for this attribute, but it is important to look whether this mean is derived from polarised or unanimous scoring.
The next stage is to assess the influence that this attribute is having on your overall appeal, and the penalty encountered when you under-perform. This will tell you whether this attribute is critical to optimise, or of secondary importance. For this we use Penalty Analysis.
How to calculate it:
Firstly, you need to have a question that measures the consumer’s overall appeal of the product e.g. How much do you like the product overall, where 5 is like a lot and 1 is dislike a lot? This could also be asked on a 7 or 10 point scale if preferred.
You must also have a series of JAR scales against which to measure the influence. For example: sweetness, thickness, colour and strength of smell.
Looking at each of your JAR scales, you need to group consumers who rated the product too sweet, or not sweet enough. So those who selected ‘a little too sweet’ (4) and ‘much too sweet’ (5) we will call group ‘A’. Next those who code ‘just about right’ (3) will be group B, and those who selected either ‘not at all sweet enough’ (1) and ‘not quite sweet enough’ (2) will be group C. So, you now have 3 groups, A, B and C.
Next you will need to calculate the size of each of these two groups, by dividing the number of respondents in each group (A,B,C) by the total number of respondents that answered your study.
Now you will need to look at how each of these groups answered the overall appeal question. This can be done in a raw Excel data file or another data package. You need to create one mean score for each of your groups. For example, Group A who considered the product to be too sweet, gave an average overall appeal of 4.15. Group B, who considered the sweetness Just About Right, have an average overall appeal of 4.55, and so on.
Once you have your mean scores for overall appeal, for all three groups, you are ready to calculate your penalty. Take Group B, those rating Just About Right, to be your starting point. Let’s assume an average of 4.55 for overall appeal. Now deduct the overall average for Group A from this total. E.g 4.55 minus 4.15 equals 0.40. This tells us that the penalty for rating the product as too sweet, is a decrease of 0.40 on the overall appeal. Then do the same for Group C.
Repeat this for each attribute you tested.
Now you have calculated the penalties for each attribute, and the percentage of consumers in each group, you can now plot these on a graph to see where they lie. Looking at this graph, the top right-hand corner is the Critical Corner; this is where you will find the attributes that have the largest penalty on overall appeal. To create the Critical Corner we would typically intersect the X axis where attributes are effecting overall appeal for more than 20-25% of respondents, and the Y axis where the penalty drop is 1 point or greater.
This graph shows us that the product being not sweet enough, the colour being too light, the thickness being too thick and the smell being too strong all have a large penalty on consumers overall liking of the product. These need to be addressed immediately.
So, is Penalty Analysis the right tool for you? Well there are many pros to Penalty Analysis. Firstly, it is cheap to run and can be done by anyone with Excel. You won’t need to employ a statistician or invest in expensive software. It is quick to run, and quick to chart, making it efficient for analysis and reporting. And it is easily understood, meaning you can demonstrate the role each attribute is having on the product without complicated explanations.
However, there are cons, the biggest of which is that it is looking at the relationship between attributes and overall appeal in isolation; tampering with one attribute could have a knock-on effect on other attributes. And as discussed earlier, some attributes are polarising which could go unnoticed by the Penalty Analysis. Finally. there are other more robust statistical techniques out there that could be used to better understand the role each attribute is having on overall appeal, but Penalty Analysis is a good starting place.
Has this been useful? Please feel free to share this article and give us a thumbs up.