Have you ever wondered why so many rollouts are disappointing? Three of the authors' recent DM News articles established that a significant part of the answer lies with statistical sampling theory. However, there is more to the story than sampling theory! Two other articles have explored how merge/purge can affect reported list performance. We will continue to focus on the non-statistical side this month, but will branch out beyond merge/purge.
Statistical testing theory is predicated on the assumption that all things are equal except for the test parameters. Unfortunately, in the direct marketing business nothing is ever equal. Some of the problem is caused by the poor decisions of circulation professionals, who often have not been trained in proper experimental design. This is something that we will focus on in a future article. A significant portion, however, is driven by fluctuating environmental influences, both competitive and non-competitive. And, you as a direct marketer must understand and be mindful of these phenomena in order to correctly interpret rollout behavior.
Non-Competitive Environmental Influences
Consider the following article, "Piece's Eerie Timing Shatters Response Goal," that appeared in the January 25 issue of DM News:
"The question, 'Can Islam and democracy mix,' and the photo of a Muslim woman covered except for her eyes on a direct mail piece from The Economist that dropped Sept. 4 contributed to a response rate 30 percent greater than expected ...
"Our September piece did better than any campaign we did in the last three years, both gross and net," said Hilde Sprung, promotion marketing director for North America at The Economist, New York ...
"The 1.2 million pieces began arriving in mailboxes Sept. 13 ...
" 'Our acquisition costs will be very low on this,' Sprung said. 'I didn't anticipate the response. It blew all the other creative packages we've tested it against out of the water ...' "
The Economist should be thankful for its fortuitous timing. However, it should be mindful of the fact that similar response boosts were experienced by certain direct marketers during last decade's Persian Gulf War, and that these boosts did not hold up over time.
Consider, for example, the veteran's organization that built a prospecting model off a fundraising mailing that dropped during the Gulf War. As the organization learned when the model flopped during rollout, response patterns during this atypical time in our nation's past were not representative of future behavior!
Competitive Environmental Influences
What the competition does, and when it does it, can have a major impact on the performance of your direct marketing promotions. The following case study illustrates this by focusing on the significant overlap that can exist between the customer universes of you and your competitors:
Several years ago, a statistics-based model was built to predict the future purchase volume of a direct marketing client's customers. This model continues to be very effective in determining which individuals to promote, and how often.
Subsequent to the construction of the model, the client purchased its largest competitor. Currently, certain administrative functions have been consolidated. However, when it comes to customer and prospect contact strategies, the two direct marketing companies continue to operate as if they are still standalone operations. We will refer to the original business as the "original division," and to the purchased business as the "sister division."
Recently, the overlap between the two divisions was quantified, with a mind to determining the level of cannibalization. The study focused on the significant number of individuals who are customers of both divisions. The goal was to determine if historical purchase information pertaining to the original division is predictive of purchase behavior from the sister division. Specifically, the predictive model for the original division "which by definition employs only historical information from the original division "was measured for its accuracy in estimating order activity from the sister division. Here's how:
First, the overlapping customers were scored by the original-division model. Then, the customers were sorted by score and broken into ten equal groups called deciles. With this approach, Decile 1 corresponds to the best performing ten percent of the file, Decile 2 to the second best, and so forth. Finally, responses received by the sister division within a six-week timeframe were appended to the file, and tabulated for each of the ten deciles.
It was remarkable just how effective the original-division model was in identifying which customers were likely to make a sister-division purchase:
Original-Division Model Deciles: Percent Purchasing Sister Division Merchandise
The results illustrate the strong interconnectivity that can exist between you and your competition.
Controlling for Environmental Influences
To the fullest extent possible, you as a direct marketer must anticipate and control for environmental influences when evaluating results. Without this perspective, you are evaluating list performance in a vacuum.
Clearly, it is more difficult to do this for competitive than for non-competitive environmental influences. However, it is not impossible. As long as you are receiving promotions from the competition, it is a straightforward matter to estimate the degree of overlap.
You can increase the probability of receiving the competition's promotions to its house files by ordering multiple times from the opposition camp. As just about anyone who has ever ordered from a direct marketing channel has learned, a purchase or two pretty much guarantees multiple follow-up contacts.
You can also achieve perspective on the competition's prospecting efforts by ordering from firms whose rental lists have consistently performed well for you in the past. The list business is very incestuous, and what works for you is likely to have also been effective for the competition!
