The age of big data

This is impressive:
This morning, if you opened your browser and went to NYTimes.com, an amazing thing happened in the milliseconds between your click and when the news about North Korea and James Murdoch appeared on your screen. Data from this single visit was sent to 10 different companies, including Microsoft and Google subsidiaries, a gaggle of traffic-logging sites, and other, smaller ad firms. Nearly instantaneously, these companies can log your visit, place ads tailored for your eyes specifically, and add to the ever-growing online file about you.
…
Adnetik is a standard targeting company that uses real-time bidding. They can offer targeted ads based on how users act (behavioral), who they are (demographic), where they live (geographic), and who they seem like online (lookalike), as well as something they call "social proximity." They also give advertisers the ability to choose the types of sites on which their ads will run based on "parameters like publisher brand equity, contextual relevance to the advertiser, brand safety, level of ad clutter and content quality."  
Adnetik also offers a service called "retargeting" that another … company, AdRoll, specializes in. Here's how it works. Let's say you're an online shoe merchant. Someone comes to your store but doesn't purchase anything. While they're there, you drop a cookie on them. Thereafter you can target ads to them, knowing that they're at least mildly interested. Even better, you can drop cookies on everyone who comes to look at shoes and then watch to see who comes back to buy. Those people become your training data, and soon you're only "retargeting" those people with a data profile that indicates that they're likely to purchase something from you eventually. It's slick, especially if people don't notice that the pairs of shoes they found the willpower not to purchase just happen to be showing up on their favorite gardening sites.
…
AdExpose, now a comScore company, watches where and how ads are run to determine if their purchasers got their money's worth. "Up to 80% of interactive ads are sold and resold through third parties," they put it on their website. "This daisychaining brings down the value of online ads and advertisers don't always know where their ads have run." To solve that problem, AdExpose claims to provide independent verification of an ad's placement.
All three companies want to know as much about me and what's on my screen as they possibly can, although they have different reasons for their interest. None of them seem like evil companies, nor are they singular companies. Like much of this industry, they seem to believe in what they're doing. They deliver more relevant advertising to consumers and that makes more money for companies. They are simply tools to improve the grip strength of the invisible hand.

But not as impressive as this:
Andrew Pole had just started working as a statistician for Target in 2002, when two colleagues from the marketing department stopped by his desk to ask an odd question: “If we wanted to figure out if a customer is pregnant, even if she didn’t want us to know, can you do that? ”
…
… the marketers said they wanted to send specially designed ads to women in their second trimester, which is when most expectant mothers begin buying all sorts of new things, like prenatal vitamins and maternity clothing. “Can you give us a list?” the marketers asked.
…
The only problem is that identifying pregnant customers is harder than it sounds. Target has a baby-shower registry, and Pole started there, observing how shopping habits changed as a woman approached her due date, which women on the registry had willingly disclosed. He ran test after test, analyzing the data, and before long some useful patterns emerged. Lotions, for example. Lots of people buy lotion, but one of Pole’s colleagues noticed that women on the baby registry were buying larger quantities of unscented lotion around the beginning of their second trimester. Another analyst noted that sometime in the first 20 weeks, pregnant women loaded up on supplements like calcium, magnesium and zinc. Many shoppers purchase soap and cotton balls, but when someone suddenly starts buying lots of scent-free soap and extra-big bags of cotton balls, in addition to hand sanitizers and washcloths, it signals they could be getting close to their delivery date.

As Pole’s computers crawled through the data, he was able to identify about 25 products that, when analyzed together, allowed him to assign each shopper a “pregnancy prediction” score. More important, he could also estimate her due date to within a small window, so Target could send coupons timed to very specific stages of her pregnancy.        

I’m agnostic about some of these privacy concerns but I believe that consumers need to be aware of them. I also use four different browsers for different purposes (Opera, Chrome, Firefox and IE). All are set to delete cookies on exit - except for Chrome which I use solely on Google sites.

My response to cookies is that I find them more annoying than a breach of privacy. The Internet is so cool that all they can come up with is serving ads? Zuckerberg’s response when Eduardo Saverin wanted to have ad companies link up with Facebook was an emphatic no because Facebook was so cool. Look where Facebook is now - they may not be serving ads but they’re definitely trying to sell something.

If users were really upset about this one thing they might be able to do is to render their databases invalid. Deleting cookies is one way. Randomly clicking away at ads is another. Sometimes I amuse myself by responding to pop up surveys and giving responses that are way out of whack.