Sears embraces Hadoop to get better read on customers

Retailer seeks competitive advantage--and revenue--in big data initiative

Sears Holdings' revenue was topped by's (NASDAQ: AMZN) revenue last year for the first time, and it is facing harsh competition from the more conventional retail behemoths Target and Wal-Mart. To fight back, Sears launched a big data campaign to get closer to its customers, reports Doug Henschen at InformationWeek.

A couple years ago, Sears set out to increase customer loyalty with rewards programs, personalized marketing campaigns and individually targeted coupons, but the effort required greater IT expertise than its legacy systems could offer, Phil Shelley, executive vice president and CTO said. In a highly innovative move for the retail sector, Sears started moving batch data processing from its mainframes to Apache Hadoop in early 2010.

It used to take Sears six weeks to analyze rewards club marketing campaigns, using SAS servers, Teradata and a mainframe computer, and the models used only 10 percent of the data that was available. With Hadoop, the analysis can be done on a weekly basis, and in the online and mobile arenas, it can be done daily--and all of the data is used. What's more, Shelley said, the company has lowered its mainframe costs by $500,000 annually.

"The Holy Grail in data warehousing has always been to have all your data in one place so you can do big models on large data sets, but that hasn't been feasible either economically or in terms of technical capabilities," Shelley said. ""With Hadoop we can keep everything, which is crucial because we don't want to archive or delete meaningful data."

Sears runs financial reports, SEC reports, supply chain analyses, customer data and other critical reports on Hadoop. In addition to saving weeks of work, the new system enables business units to develop many of the reports themselves.

Sears set up a subsidiary, MetaScale, which leverages the company's data center capacity to help non-retail companies take advantage of Hadoop. Shelley is CEO of the division, which provides services, including data management, modeling, architecture and consulting, on a subscription basis.

For more:
- see Doug Henschen's article at InformationWeek

Related Articles:
Will Hadoop make an open-source convert out of you?
Hadoop gains traction in the enterprise