[Note : Despite dramatic increases in raw computing power and a proliferation of end-user software tools since the publication of this series of six articles, virtually all of the content remains highly relevant. The occasional obsolete point is highlighted.]
Statement of Purpose
In a series of six articles, we explain a number of the key concepts that mailers should understand about merge/purge, as well as reviewing (in the first article) a methodology that could be helpful in evaluating the effectiveness of either present or prospective merge/purge systems. While our comments are primarily addressed to mailers, merge/purge vendors can benefit by measuring themselves against the criteria that we have identified as important.
Our objective is to describe new and specific tools that can be used to evaluate and improve the performance of the merge/purge process. Through commentary and examples, we will attempt to translate into layman's terms the technical jargon that baffles many mailers. In the process, practical applications should become apparent.
This month's article, Part Four, focuses on unduplication at the household versus individual level, additional issues for business-to-business direct marketers, and advanced duplicate recognition techniques.
Unduplication at the Household vs. Individual Level
Most consumer mailers choose to unduplicate at the household level rather than at the individual level, because it is generally believed that one mailing package to each address is optimal. A few do opt for unduplication at the individual level, because it is to their advantage to have more than one customer per household. These mailers are faced with a number of additional decisions concerning the definition of a duplicate. All of these focus on the first name:
For example, should these be duplicates if all other aspects of the record are identical?
- Robert Smith and R. Smith "One record contains a first name and the other an initial.
- Robert Smith and Bob Smith "The first record has a proper name and the second a commonly used nickname for that proper name.
- Bob Smith and R. Smith "One record has a nickname and the second an initial of the related proper name.
- Mrs. Robert Smith and Paula Smith "Here, first names indicate opposite sex, but the title indicates the same sex. Are there two people here, perhaps mother and daughter, or merely one individual?
- Robert Smith and Robert Smith Jr. "One record contains a suffix and the other does not.
- Roberto Smith and Roberta Smith "Here, a slight difference of one vowel in the first name could indicate either different people of different sexes, or merely a single input error. Many parents give names to their children that are similar to their own. Again, there is no right or wrong answer.
- Robert Smith and R. Smith (gender code) "The interaction of gender codes with first name permutations can make a difference. For example, the handling of Robert Smith and R. Smith in the first example will depend upon whether R. Smith has been coded "male," "female" or "unknown." And, the Roberto/Roberta decision in example six would also be affected by the sex code.
Issues for Business-to-Business Marketers
So far, we have discussed issues that apply to both business-to-business and consumer unduplication efforts. A number of additional, unique issues must be faced in a business merge/purge, making it far more complex than the consumer version:
Different Levels of Unduplication
First, unduplication can be at the company, location or individual level:
Here are three records with three individuals at the same company: two at one address, and the third at a different address.
- With unduplication at the "company level," all of the records are duplicates, because all are located at CNL, Inc.
- With unduplication at the "location level," number one and number two are duplicates, because only these two both work at 29 Woodsons Avenue.
- And, with unduplication at the "individual" level, none match, because all refer to separate individuals.
Multiple Company Names
A second issue in business-to-business unduplication is that companies frequently have several names or abbreviations of names.
Imagine for a moment that you want to send only one mailing piece to each company you contact. Ask yourself how you would know whether or not these are separate companies within a single address, multi-office complex:
- The first record contains the parent company name: CNL, Inc.
- The second, The Software Sales Group, is a division of CNL, Inc.
- The third record contains a frequently used abbreviation of the division in record two.
- The fourth is a second division of CNL.
Multiple Addresses
A third issue is that many companies have multiple street, building name, and/or post office box addresses:
- The first record is the standard address for one division within the firm.
- The second is the warehouse address for this division. It is, however, at the same location, with the only difference being the street entrance.
- The third record is one of many post office boxes maintained by the company.
- The fourth is the research and development center for this division, located several miles away.
The business-to-business problems illustrated in these sets of examples are extremely difficult to handle by themselves. When several such problems appear within a single group of records, however, the challenge becomes almost overwhelming.
Additional Business-to-Business Complications
There are, in fact, still more complicating factors:
- Executives often hold several titles, spread among multiple divisions and geographic locations:
- Harold Abbott is listed as a vice president on the roster at the CNL corporate headquarters.
- Abbott, however, spends most of his time at S.S.G.'s research and development location on Henry Street.
- In addition, business records frequently contain non-address characters within the individual name field, such as "care of" and "attention." Sometimes, only a title will accompany a non-address character (e.g., Attn. Warehouse Mgr.)
- Transposition of fields is a very common problem.
It is for these reasons that good consumer performance does not necessarily guarantee strong business capability. A business-to-business merge/purge is so different that there are merge/purge vendors who perform only this type of job.
Advanced Duplicate Recognition Techniques
Advanced duplicate recognition techniques are being developed to deal with the complex problems found in a business-to-business merge/purge. Central to these techniques is an attempt to catch additional duplicates by "linking" together multiple records that have various but different fields in common. Some vendors are successfully extending these advances to the consumer arena.
Assume, for example, that you want to send just one mailing piece to each company:
- Record #1 is the first to be examined. The mail piece will be addressed to this individual.
- Record #2 has the company name in common with number one. It is therefore a duplicate, and should not be mailed. In addition, we have established, for the first time, that Allen Services is in Suite 406.
- Record #3 has the suite number in common with record number two, which indicates that Greg Sterrett is also an Allen employee, and should therefore be suppressed. Notice that all three records have been indirectly linked together, which would not normally be possible.
- Record #4 has the suite number in common with number three, and should therefore also be suppressed as a duplicate. It also establishes, again through this linking process, that Johnson, Allen & Associates is an alternate company name.
- Record #5 has this alternate company name in common with record number four. This indicates that Mike is also an Allen employee, who should therefore not be mailed. As the linking process continued, it is now known that Allen Services has a post office box.
- And finally, via a post office box match with Record #5, we can establish that M.T. Pepitone is an Allen employee, who should also not be mailed. The overall result of this linking process has been that only one of these six records has been retained.
Linking techniques are at the leading edge of merge/purge technology, and are still being perfected. We cannot promise that any currently existing software could handle these six records perfectly. It is clear, however, that the technique has impressive potential to uncover duplicates that would normally go undetected.
