Loyalty program can be an expensive investment. Once you decide to take that plunge, it is important to take the pulse of your program at regular intervals to make sure it is doing what you want it to do. In this series, I would like to discuss how to gauge loyalty program success and ROI.
Success Metric Depends on Your Goal
Different loyalty programs set out to achieve different goals. The best metrics for gauging your program performance depend on the specific goals you have for your program. Broadly, most companies decide to start a loyalty program for one of the following five reasons:
- To grow customer spending. With this goal, you are trying to make consumers increase their spending at your business. This can happen either because they pull their purchases from other places to concentrate more of their purchases with your business, or because you are expanding their absolute demand. The latter one is more likely to be the case when your product category is a flexible one, e.g., travel, entertainment, etc.
- To reward your best customers and strengthen their loyalty. You create a loyalty program to better understand and cater to their needs, and to make them feel truly appreciated. The delayed reward nature of such programs can also decrease your promotional cost to these customers.
- To be on par with competition. This is a scenario where many of your competitors have loyalty programs. Although you may have felt ambivalent about having such a program, you realize the pressure the competitors’ programs are creating to your business and the bleeding of your customers because of it. In this case, your main focus is to catch up on competition and stop further loss of customers and their spending.
- To attract new customers. This is almost the opposite to the last scenario. In this case, you may be a loyalty program pioneer among your competitors, or you may have a better designed loyalty program. Either way, you are hoping to lure at least some customers away from your competitors, or otherwise attract consumers who were not buying from you before the program was in place.
- To gain customer insight. Your business may not offer a natural way of tracking consumer purchases. Starting a loyalty program can help you solve this problem by offering a mechanism of tracking at least some consumers’ purchases. You then leverage the insights learned to improve your products and marketing messages.
Do you see your program goal fall into one of these types? Good. We can now look at what metrics you should use to make sure that your program is reaching its goal. I will discuss the first scenario in this article and address metrics for the other goals in later parts of this series. But before I continue, I should mention that sometimes a program may be created to achieve multiple goals. In that case, you will want to identify the proper metrics for each goal and monitor them simultaneously.
Loyalty Program Performance Metrics When Spending Growth is the Goal
The obvious metric in this case would be spending per customer, which you can further break down into transaction size and purchase frequency. An example of such an assessment can be found in my paper on the long-term effects of loyalty programs. If the program is successful in achieving its goal, we should see per customer spending to increase after they join the loyalty program. Although this may seem rather intuitive, the implementation of this metric is actually not that straightforward. To gauge increase, you need a baseline to compare the current spending level with. Two natural baselines come to mind:
- per customer spending of those who are not in the loyalty program; and
- per customer spending of program members BEFORE they joined the program.
Each of these baselines has its own problems. For the first one, there is a significant self-selection problem. To understand why, imagine two customers named Mary and Jane. Mary spends $100 per week in your store, and Jane spends $50 per week in your store. Mary happens to be a loyalty program member and Jane is not. A simple comparison between Mary and Jane may make you think that the program led to the $50 extra spending per week by Mary. But this would be wrong, because we do not know if Mary decided to join the program because she was already spending more. In general, customers who buy more from your business are likely to find your loyalty program to be more attractive. Therefore, your better customers are more likely to join your program than those who don’t buy a whole lot. If you simply compare program members vs. non-members, you may simply be capturing these pre-existing differences in spending that happen no matter with the program or without.
The second option compares the same member’s behavior before vs. after he or she joins the program. This tracking of same individuals’ behavior over time does not suffer from the self-selection problem. But it has its own problem too. For example, if you have been improving your marketing effectiveness beyond the loyalty program (e.g., better messaging, more attractive product mix, etc.), these improvements are likely to cause an increase in spending over time too. It is not easy to tease apart these other time trends from loyalty program induced growth. So you may conclude incorrectly that your program is causing spending growth when it is not. Imagine an opposite scenario, where for some reason your products are becoming less attractive to customers over time. Although your program may be increasing customer spending on its own, that increase would be offset by the increasingly unattractive products. This may result in a wash, making you believe that your program is not doing anything, when in fact it’s other parts of your marketing process that’s broken.
How to Address Self-Selection and Attribution Problems
There is no single perfect way of addressing these issues. One thing that can help is to combine the two baselines above and use them both in your growth assessment. Let’s assume spending per member per month before joining the program is M0, and after joining the program is M1. Similarly, assume spending per non-member per month during the same pre-program period is NM0, and during the same after-program period is NM1. Instead of comparing M1 and M0, or comparing M1 and NM1, the better formula for calculating program induced spending growth is:
By considering the time trend for the same customers in conjunction with the member vs. non-member comparison, we are able to tease out the over-time increase in spending that is unique to program members. Any baseline differences in self-selection would be reduced by the M0 and NM0 in the equation. In the meantime, any growth over time due to other non-loyalty program related marketing changes is factored out by comparing the time trends between program members and non-members, as they both would have experienced the same other marketing changes.
The disadvantage of the above approach is that you need quite a bit of data, not only of members’ and non-members’ spending now, but also their spending prior to the program. You may not have such data available, and all you have may be M1 and NM1. In this case, you can try to use data analytics to deal with the self-selection problem. Two common approaches are using propensity score matching and using instrumental variables. I won’t go into the mathematical details of these here. But very quickly, the propensity score matching approach tries to identify a group of non-members as similar to members as possible. This way, any difference in spending between members and non-members are less likely due to pre-existing differences between the two types of customers. A good reference book on this method is Propensity Score Analysis. The second instrumental variables approach tries to use some other factors unrelated to spending to explain program joining decision and then factor that into the overall model. You can take a look at this academic paper on loyalty program behavioral impact for an example of this approach. Plenty of econometrics books cover this method as well.
What If You Only Have Members’ Spending After They Joined the Program?
It is not uncommon for a business to have only M1, that is, spending history of program members only after they joined the program, and nothing about non-members. Obviously the lack of data significantly limits what you can do. But it is not a completely hopeless situation. You can still try to model the time trend in your program members’ spending, time being how long a particular member has been in the program. However, you’ll want to take into consideration other influences such as the start of an important new marketing initiative, product pricing, new product introduction, seasonality, macroeconomic trends, etc. This will help control for the other influences that could also cause a trend in purchase behavior.
Another possibility is to model program vs. non-program sales at the business level. Presumably knowing M1, you can figure out how much of your sales were from loyalty program members, and the rest from non-members. You can then model the sales data over time. You can see an example of this on p.30 of my loyalty program effects paper (section labeled “Store-Level Trends”). At the time of my work, I did not have accurate information about member count over time for the program I studied. Usually your business will have that information, and you will want to control for the (hopefully) increasing number of active program members in your analysis.
I will stop here for this part of the series. In latter installments, I will discuss the proper metrics for the other four common program goals. A lot of the issues discussed here, such as the self-selection problem, exist in those other contexts as well. Although I won’t reiterate the same points, please keep in mind of these limitations in your own metrics and analyses so you do not grossly overestimate or underestimate the impact of your loyalty program.
I hope you find this discussion on loyalty program performance measurement helpful to your business. I welcome comments and feedback from you. If your program has a goal that is not covered by the five types here, I would like to hear that as well. Finally, to receive notification about new posts in this series, you can subscribe to my blog by filling out the short form below.