Big Data Holds Big Potential for Marketers

Marketing data used to be simple to acquire but straightforward to organize in simple datasets, like the names and addresses of potential customers compiled into precious mailing lists.

Today the scope of data in the marketing sector is much, much broader. Not only might you need contact information, but also household buying habits, political affiliations, and number of hamsters in the home. Because of the sheer diversity of this data, acquiring it is a much more complex exercise than the old mailing lists, because at least in that instance you would have known what you wanted. Today, sometimes it's hard to know what is the right question to even ask.

In such an environment, it is easy to hear about a concept like "big data" and start seeing it as a potential life-saver for marketing. But before you pin all your hopes on big data, first it's important to understand what big data really is.

First off, there's no cut-off amount of data that suddenly defines a dataset as "big." Part of this is because that ceiling would be constantly moving. A decade ago, the technology world was all a-flutter over the presence of Microsoft's geographic database TerraServer (now TerraServer-USA), the world's first multi-terabyte dataset. Today, terabyte drives are easily purchased for PCs.

Another reason is that "big" isn't always the determining factor in what defines this new area of storage technology. Sometimes it's how the data is stored. Most data these days is stored in what are known as relational databases: Data sits in tables, which are in turn connected to each other (through relationships), so the data can be found later through a query. So, if you have a table of customer information (with a customer ID), and another table of sales information that also has customer ID numbers denoting who bought what product, you can easily run a query to find out who bought gum in your store last year, because the customer ID number connects these tables together.

If you have done any sort of Web work, then you already use relational databases. Most Web content managers use relational databases to manage the content.

Big data -- also referred to as "NoSQL" or "unstructured data" -- does not use tables and queries to store data. That's because after a certain size, tables can get very unwieldy. And, if you want to change or otherwise update a very large table in a relational database, you could be in for a world of pain, or worse: downtime. And in this day of 24/7 e-commerce, downtime might be the kiss of death. So, data is stored in special file systems that use advanced, streamlined control nodes that keep track of where data is, even if data gets so big you have to add more machines to the database.

From a technology standpoint, non-relational databases are a lot cheaper than relational databases and can be easily scaled to meet the demands of a growing data store. They can also use queries built on something other than the Structured Query Language (SQL) that's been a part of database technology for decades. (Hence, "NoSQL.")

From a marketing standpoint, the benefits of this technology are potentially huge has well. In the past, corporations could only afford to keep a certain amount of data, which means that modeling might be done with a limited dataset. Now, with big data storage so relatively inexpensive, companies can store, well, everything. This means if you want to run models on every conceivable marketing scenario, you very likely can.

The capability to store so much data also means that your data can also come from something other than customers: the products themselves. Smartphones, in-vehicle assistance systems, and other electronic devices can already "phone home" usage data that give a very complete picture of what's going on with the product. RFID tags are now small enough and cheap enough to put on nearly anything. Couple this with near-ubiquitous WiFi in homes, and trillions of new Internet addresses thanks to the coming adoption of IPv6, and you will have what's been coined "The Internet of Things." And big data is read to handle this influx of information.

If all of this talk has you salivating at the prospect of knowing how often that 45-year-old man in Indiana changes razor blades, or other such fun data, some words of caution. First, and foremost, privacy issues are a real concern, and once businesses figure out they can store everything about a customer, they will try. Some common sense will work wonders if you want to start tapping into such datasets.

Second, not every company is equipped to handle that kind of data flow, nor are they intending to implement it. The presence of cloud computing, with the capability to expand to a network of virtual machines at the flick of a proverbial switch, pretty much solves the hardware problem, but right now NoSQL data techniques are still in their early adoption phase, and there is a distinct shortage of the types of specialists who can work with this kind of data.

It's not just the database administrators, either. Data scientists are also in short supply. These are the folks who not only manage the big datasets proper, but also work with the tools to extract real information from that same data. It is here that a real gap exists, one that poses a current obstacle for marketers interested in mining all this data. Gathering and storing the data is coming along nicely. Visualizing the data is something that's behind the curve. And without proper visualization and analytics, all the data in the world can't do much to help you.

With the steadily growing crop of data scientists and the growth of data marketplaces with their own visualization tools, this is an area that's improving fast. Once it's matured, marketing should be able to tap into a wealth of data to assist their decision making and improve their bottom lines.

Related posts:

• Using Analytics to Get the Most From Content Marketing
• CMO University: The Digital Ocean -- Using Bit Data Effeciently
• Using Big Data for Marketing Success

— Brian Proffitt is a veteran journalist and analyst.