I have been a direct and database marketing consultant since 1984. In all that time, one consistent verity is that most internal IT departments think they can - and should - be responsible for the marketing database. In many instances, the IT department has no idea what it is talking about.
Why is this? I think it has to do with the term "marketing database." IT professionals hear the word "database," and say, "Ah ha! That means a system, and systems are in my bailiwick."
Well, the IT guys are partly right, but they are mostly wrong. This is because, for the majority of direct marketers, the systems component of a marketing database is relatively trivial. Sure, there are some multiple-terabyte systems with near-real time update cycles, and dozens of users who need simultaneous access. But, most databases are much smaller, and with no more than one of two users accessing it at any given point in time.
For these smaller applications, the real challenge lies with the content; that is, the "stuff" of which the database is constituted. This "stuff" can be very difficult to render consistent and usable because of three challenges, none of which lies within the bailiwick of an IT professional:
Challenge #1 : Name and Address Processing
B2C account information must be aggregated to the individual and household levels. Likewise, B2B account information must be aggregated to the individual, site and organizational levels. These multiple levels of customer (and, when applicable, prospect) definition are required to:
- Perform accurate analysis, scoring, promotional selections, and response attribution.
- Properly allocate marketing-spend to each customer.
In order to pull all of this off:
First, address standardization, ZIP Code correction, parsing and unduplication technologies - guided by carefully-constructed business rules - must be employed to match accounts on a combination of names, company names, addresses, phone numbers, and - when applicable - bill-to/ship-to relationships.
Then, the matches must be unified into a single non-circular cross-reference that:
- Assigns each account to one and only one individual.
- Assigns each individual to one and only one B2C household or B2B site.
- For B2B, assigns each site to one and only one organizational entity.
Finally, all of this must be maintained over time so that it is easy to make adjustments and enhancements, and re-consolidate the data, per ongoing quality assurance that is conducted on the matches.
Challenge #2: Transaction Processing
Hopefully, your customers are doing lots of buying. Most likely, the purchases are taking place across multiple channels. You almost certainly have at least one e-commerce site, and you probably have an in-bound call center. If some of your revenue comes from B2B, then you are likely to have an outbound sales team and/or field sales force.
The data from each of these sources will have its own structure and anomalies. Multiple divisions often mean even more permutations of data structures and anomalies, especially when company mergers have taken place.
The bottom line is that transactional data is not particularly usable in its raw format. In order to make the data usable, the semantics must be rendered historically complete, consistent and accurate, and correspond with core business concepts. Also, the data must be time-stamped and maintained down to the atomic level, and must not be overwritten, archived or deleted. Finally, the following must be included:
- Demand, as opposed to "shipped" or "completed," transactions.
- Promotion transactions, even for those that did not result in a purchase.
- The following, when applicable: inquiry and post-demand transactions, and supplemental sources such as demographics, "firmographics" and social networks.
Challenge #3: The Creation of Past-Point-In-Time Views
A modern database must support - on-demand - any calculation, aggregation or subset that logically can be generated from the underlying data. This requires a mechanism to allow the efficient and rapid re-creation of multiple past-point-in-time ("time-zero" or "time-0") views. Time-0 views are necessary because all of the dimensions to be analyzed cannot be known and "frozen" in advance. These views form the basis for virtually all meaningful analytics, by allowing customers to be classified based on detailed histories only up to the appropriate past-points-in-time.
Cohort analysis such as lifetime value is an important example of data mining that depends on the re-creation of multiple time-0 views. Likewise, the analysis and validation files required for predictive models are based on time-0 views.
Another application of cohort analysis is the monitoring of changes in customer "inventories," such as fluctuations in segment sizes and performance over time. Still another is the analysis of historical trends within subsets of promotional channels, products and services offered, etc.
Final Thoughts
How many internal IT departments have the chops to handle these three challenges of marketing database content? Not many! Those that do are typically concentrated among companies in which the scale of the application is such that it makes sense to hire a team of experienced professionals.
Things are different for smaller database applications in which it is not cost effective to hire multiple experienced professionals, much less staff to the level of job-function redundancy required to counteract the inevitable resignations and terminations. All of this, to return to my opening statement, is why most IT departments have no idea what they talking about when they think they can - and should - be responsible for the marketing database.
