Book Review Retail Analytics, The Secret Weapon by Emmett Cox

by Emmett Cox Wiley and SAS Business Series 2012 

The book cover opens with promising references and from Tom Davenport and all this on just 142 pages of 10 point Times new Roman with double spaced interlines. Wow! This needs further reading I guess. Chapter One gives a very high level introduction to the main process: sales and replenishment and chapter two introduces the ins and outs of retail data management. A diagram of the transaction log files could improve the didactic impact of this chapter and the comparison between denormalised and normalised data is at best superficial. But I guess the readers that, according to the editor are helped with their critical management decisions don’t bother much about technical mumbo jumbo, so let’s move on. Although this inspires me to publish an in depth blog on the pros and cons of these datamodels that might interest senior-level management. Because it is about money, time and quality and there is no one size fits all choice to be made.

Inspiring introduction

Cox sums up a few interesting case studies ranging from trade area modelling, site selection modelling, competitive threat analysis, merchandise mix modelling, marketing effectiveness tracking, brand analysis, clicks and mortar analysis, cross sell analysis to market basket analysis. And he does so in plain English which at least will trigger the interest of senior management. The schema on page 25 gives a quick and clear view on how the analytical domains are part of a chain driving important retail and merchandising decisions. While reading these case studies you are confronted with a very mild and unobtrusive form of product placement with sentences like “The statistical software company SAS has a strong set of utilities within its SAS Enterprise Miner that we used heavily” (p. 33) Yet, the case studies are very credible and well presented, even if they don’t go in depth into the statistical details. I guess the target group of this book will tell guys like me: “Find out how we can apply this method in our stores”. Chapter three pays attention to the specifics of the apparel industry and does so in a didactic manner, illustrating clearly the constraints apparel buyers have to work with: long order lead times, optimum price elasticity management, optimum sales promotion management,… Having worked in sales promotion for a mail order company doing 80% of its sales in fashion I recognised the stressful situations even the best retail analytics won’t attenuate. The fourth chapter spends quite a few pages on explaining what a GIS tool can do and how it works which is a bit redundant in the days of Google and Wikipedia, leaving only a few pages for the application of GIS in retail. For example: the combination of category management and GIS analytics can produce powerful results I am sure the author could illustrate to his readers. It is as if Mr. Cox apologises for this in the fifth chapter opening with “Chapter 4 was an extremely technical illustration of the GIS tool.” Er.. well,… I wasn’t bothered with points, vectors and surfaces of the GIS world, I just wanted to know more about the applications…. 

 A 101 on in store promotion 

 In-store marketing and presentation get a lot more attention with 31 of the 142 pages. Pricing strategies are explained very well as they are the meat of retail analytics. But Cox’ emphasis on convenience shopping hides its counterpart: fun in shopping which also needs tailor made analytical insights to enhance the profitability potential. His reference to DB2’s datablades brought me back to the heydays of Informix, once the leading RDBMS before Oracle displayed a superior sales and marketing strategy. All in all this chapter is a little too narrowly focused on the US situation. Phrases like “Most retail chains are now open 24 hours a day…” (p. 98) will raise eyebrows in Europe and sometimes the author gets lost in details like the description on how floor graphics can be removed “without leaving any residue behind”. I wonder who in the analytical community will bother about this. And when I read stuff like “another form of subliminal messaging” (p. 101) related to clearly detectable things like baking smells I get confused. “Subliminal” means a stimulus the audience is not aware of. And literature and research are not in unison about the effects of subliminal messages in stores. Chapter 6 handles the store operations, an underestimated source of information opportunities to be used; Think of workforce management where you optimise the match between labour cost and service level based on POS data and longitudinal time series. Think of differentiated messages to consumers, HVAC control, intrastore communication, replenishment: Cox explains these operations and the link with retail analytics in a clear and concise way. 

Introduction to loyalty marketing

Last but not at all least, the chapter on loyalty marketing. Since Reichheld and Sasser’s publications on customer loyalty we are aware that the link between excellence, customer satisfaction and loyalty is not always obvious and in many cases it is not even existent! Measuring and managing customer loyalty is a real pain in retail. To me, attracting customers to loyalty programs is not equal to creating customer loyalty. It is just a form of prolonged sales promotion that –if stopped- would cause an immediate drop in sales. True loyalty is quite different. It took me one and a half page to read about it: “(…) as long as the consumers do not just cherry-pick the discounts and rewards offers.” Spot on Emmett! Weeding out the cherry pickers is the true challenge for retail analytics. The author offers a checklist before you begin a loyalty program.

To conclude

This book will certainly deliver value for novices in retail marketing and analytics. If you're a CEO or CMO of a retail chain and wonder whether you should invest heavily in an analytical environment, you are probably too late as the competition will have obliterated your obsolete business.That doesn't mean other C-level executives couldn't benefit from this book.  But to be really relevant for other profiles like a CIO or COO, it should have related data and process management to analytics with diagrams and models. 

A Short Memo for Big Data Sceptics

In an article in the NY Times from 17th August by James Glanz, a few Big Data sceptics are quoted. Here is a literal quote: Robert J. Gordon, a professor of economics at Northwestern University, said comparing Big Data to oil was promotional nonsense. “Gasoline made from oil made possible a transportation revolution as cars replaced horses and as commercial air transportation replaced railroads,” he said. “If anybody thinks that personal data are comparable to real oil and real vehicles, they don’t appreciate the realities of the last century.” I respectfully disagree with the learned scholar: the new oil is a metaphor for how our lives have changed through the use of oil in transportation. Cars and planes have influenced our social lives immensely but why shouldn't Big Data do so in equal or even superior order? Let me name just a few: 

• Big Data reducing traffic jams (to stick close to the real oil world) 
• Big Data improving the product-market match to the level of one to one, tailoring product specifications and promotions to individual preferences, 
• Big Data improving diagnostics and treatments in health care combining the wisdom of millions of health care workers and logged events in diagnostics epidemiologic data, death certificates etc... 
• Big Data and reduction of energy consumption via the smart grid and Internet of things to automate the match between production and consumption, 
• Big Data in text mining to catch qualitative information on a quantitative scale improving the positioning of qualitative discriminants in fashion, music, interior decorating etc... and of course... politics Ask the campaign team from 44th President of the United States and they will tell you how Big Data oiled their campaign.

As soon as better tools for structuring and analysing Big Data become available and as soon as visionary analysts are capable of integrating Big Data in regular BI architectures the revolution will grow in breadth and depth. Some authors state that entirely new skills will be needed for this emerging market. If I were to promote training and education I'd say the same. But from where I stand today I think the existing technological skills in database and file management may need a little tweaking say a three or five day course but no way is there a need for an MBD (Master in Big Data) education. On the business side of things there may some need for explaining the works of semi structured and unstructured data and their V's which already add up to seven. I believe it is going in the same direction as the marketing P's where Kottler's initial four P's were upgraded to over thirty as one professor of marketing churned out this intellectual athletic performance. Let's sum them up and see if someone can top them: Volume: a relative notion as processing and storage capabilities increase over time Velocity: ibid. Variety: also a relative notion as EBCDIC, ASCII, UTF-8 etc... are now in the company of video and speech thanks to companies like Lernhout and Hauspie whatever the courts may have decided on their Language Development Companies, Volatility: I have added this one in an article you can find on the booksite on "Business Analysis for Business Intelligence" because what is true today may not be true tomorrow, so it is not about the time horizon you need to store the data as some authors claim because that will be defined by the seasonality. The problem with these data is there might not be any seasonality in them! Veracity: how meaningful are the data for the problem or opportunity at hand? Validity: meaning Big Data can only be useful if validated by a domain expert who can identify its usefulness. Value: what can we invest in recording, storing and analysing Big Data in return for what business value? This is one of the toughest questions today as many innovative organisations follow the Nike principle: "Just do it". And that, professor Gordon is just what all the pioneers did when they introduced the car and the aeroplane to their society, ignoring the anxious remarks from horse breeders and railroad companies. Remember how the first cars where slower than trains and horses? I rest my case.

Time to muse over “holiday”

Greetings from a 30°C office, in the north of Belgium, Flanders. It’s late in the evening, a good time for some lighter philosophising before packing the suitcase.

The DATE dimension is a very simple flat table (although some designers and DBA’s prefer third normal form). It contains surrogate keys, the date, the name of the weekday, month, day number, week number, month number, quarter, trimester and semester number, IsLastDayOfMonthFlag and what have you. There will also be a IsAHolidayFlag somewhere and in case you only work in a few countries, you simply add a column per country.

In case you operate on a global scale  you may want to snowflake a bit to support truly global analytics and compare apples with apples.

Example of a snowflaked Holiday model

Holidays can explain a lot

Imagine you are comparing last year’s June sales with June of this year. Last year, June had no holidays and 21 working days. This year it has 20 working days and one holiday, i.e. 19 working days. In case your revenue is directly linked to the number of working days, this explains for a revenue drop of no less than ten percent in June this year! And in case you sell holiday related products like ice cream or beer it may well run into the other direction explaining a sales increase of over twenty percent or more…

So, here’s my advice: enjoy your holiday but don’t forget to integrate it in your analysis.

When will transaction systems and analytical appliances converge?

 There is no harm in being a bit visionary sometimes…

For the last three or four decades, the gap between online transaction processing (OLTP) and business intelligence(BI) was impossible to cross. Neither technology nor the architecture of both systems allowed integration on data level. So cumbersome and expensive technical infrastructure was needed to extract, transform and load (ETL) data into data warehouses and data marts to exploit the information assets, hidden in the records of the OLP systems.

One of the main problems was that OLTP applications were never conceived with a view on BI. For example, the first Customer Relationship Management (CRM) Applications stored the customer address  as an attribute of the CUSTOMER entity instead of treating it as an entity in itself. When the first geographical information systems and geographical analytical systems came along, CRM developers remodelled the customer database and made ADDRESS a separate entity.

Imagine…

… a greenfield situation where you could develop any OLTP application from scratch and with an added BI perspective. What would it look like? What would be the guiding design principles to develop an OLTP application that plugs in seamlessly into a BI infrastructure.

Because, don’t get me wrong, there will still be a need for a separate and dedicated BI infrastructure as well as a specific BI architecture. But the ETL , the master data management (MDM) and data quality (DQ) management should no longer be a pain.

What would it take to relieve us from ETL, MDM and DQ chores?

Or how the simplest things are the hardest to realise. It would take:

• A canonical target data model and subsequently,
• A function in any OLTP application to drop off its data in that target format for the BI infrastructure to pick it up, be it via an enterprise service bus or via a bulk load procedure,
• A hub and spoke system where the master data objects are managed and replicated in both the OLTP and the BI applications,
• A uniform data quality policy, procedure and checking of accuracy, consistency, conformity and being in line with business rules of lower order (e.g. “a PARTY > PHYSICAL PERSON must have a birth date older than today”), but also of a higher order (e.g. a CUSTOMER who is under age should have a parent in the customer database whose co-signature must be on the ORDER form”)

So why is this not happening?

One explanation could be that application vendors claim they have full BI functionality built in in their solution. I remember a client with an ERP solution that contained over 300 standard analytics and reports of which the client used less than 3 %, eight to be exact. Not a great BI achievement I guess, but certainly an attempt to have 100% account control...

Another explanation could be the that we are still far away from a canonical model integrating transaction processing with BI, although data warehouse appliance vendors will tell you otherwise. Nevertheless, the dream has been kept alive for decades. I remember a data warehouse guru from the nineties who developed what he called verticals for BI to be used in telco, retail and finance, among others. They were just target models in the third normal form to set up a corporate data warehouse, so no OLTP included. He managed to sell them to a database vendor, bought a nice sailing boat (or rather a yacht) and disappeared from the BI stage. The database vendor peddled these models for a few years but in the years after 1999 I never came across these templates. And the vendor never mentioned their existence since then. Conclusion: we're still far away from a one-size-fits all transaction and analytics model.

What is the second best choice?

I can come up with a few ideas that will still be hard to realise.  Think of an industry wide data type standard for transactions and BI purposes with the transformation rules documented. For example: a timestamp in a transaction database should always be in the form of 'YYYY-MM-DD HH:MM:SS' and two numbers are added for BI purposes: the standard day number, counting from an internationally accepted date like 1900-01-01 and the second number, which is a figure between 0 and 86.400 to represent the lowest grain of the time dimension.

The next step is to design industry dependent star schemas for basic analytics every organisation needs in that industry. There is after all, a growing body of knowledge in retail analytics, telco, finance, production, supply chain etc… If we can already achieve that, I will probably see my retirement date coming which is 2033-01-04 00:00:00.

Managing Choice, a BI and CRM Challenge

Introduction 

When I was visiting countries behind the iron curtain in the period before November 1989, it struck me how simple life was there. You had the choice between two cheese types in the supermarket (if they were available) one pickled gherkin flavour,… The car market showed a little bit more choice: two Trabant types, a few Wartburgs, Skodas, Ladas, Tatras, FSO, Dacia and other Yugos… They all shared the same features: low quality and no evolution in safety, design or luxury…

Back to 2013: hundreds of cheese types in the better supermarkets and car maker Volvo alone has the capability of building over 5,000 product configurations of its mid-market model. Today, managing choice has become a shared skill, shared between producer and  retailer on one hand and consumer on the other hand.

About Choice Stress

Choice stress is a common phenomenon in developed markets because the differentiation becomes so low in granularity that we get stressed because we want both: a bio low fat yoghurt with strawberry flavour in a reusable cup with the chance to win a trip to Disneyworld but also another bio low fat yoghurt with strawberry flavour in a recyclable plastic cup with a cash back promotion. And then you ask yourself: “But where’s the pineapple variety?”

More and more, Customer Relationship Management becomes the art of dialogue with your customers to help them make the right choice in a stressless environment.

Retailers know that consumers’ main sources of stress are store related (like staff, queues, parking, products sold out, messy presentation, regular changes in the aisles,…) and choice related (mainly brand clutter and information clutter). But what are they doing about it and how does this relate to shopper marketing in the store and online?

One side of the coin: Business Intelligence in the virtual and the real world converges

From needs, occasions and solutions, how do you make the transition to the most profitable brand on your shelves?

And how do you make sure both showroomers and webroomers end up on the right web page or in the right aisle?

Business Intelligence solutions for retailers need to converge both web clicks and store visits per customer to come up with answers to these questions.

Let’s examine the enablers for these advanced analytics.

First there is an organisational aspect: make sure there are no splits in your hierarchy between online and store marketing management. Phew, that’s going to be a hard one for some organisations. You may be enthusiastic about the internal turf wars but your customer doesn’t make the distinction between your click and mortar presence, so why should you?

Second: the balance of power is shifting, so how do you adapt? In the pre Internet era when information was in the hands of producers and retailers the consumer was subjected to  their agenda. Now it is the other way around. Consumers create their own information about products and brands and managing this flow of dispersed blips on the radar is quite different from the traditional broadcast, one-way marketing communication. The consumer’s knowledge on product ranges of his choice is sometimes better than the shop assistant’s. Social media may not be a good vehicle to promote any brand but they sure are effective vehicles to break down reputations… fast and irreversible. That is not to say that there aren’t brands and retailers effectively using social media to manage sentiments and content about their products and brands. But they are still a minority, which is the only positive message I have for the laggards: there is still time to catch up. But don’t wait too long. Initiatives like Amazon Birthday Gifts using Facebook to have friends chip in for a birthday gift card are just the beginning of a set of ploys coming along to digitise classical real world interactions and channel them to the retailer who has the creativity and excellence of execution to take the first mover advantage. Big behavioural data[i] will become more and more a topic on the retailer’s agenda but this is only one side of the coin. The other side is a new form of customer relationship management (CRM) where the social aspect is altering the classical CRM processes.

The other side of the coin: social CRM

Paul Greenberg, a recognised CRM expert for decades, cornered the term in a handsome and useful definition:

"CRM is a philosophy and a business strategy, supported by a technology platform, business rules, workflow, processes and social characteristics, designed to engage the customer in a collaborative conversation in order to provide mutually beneficial value in a trusted & transparent business environment. It's the company's response to the customer's ownership of the conversation."

I’ll add my two cents to that definition: it is an extension of the existing CRM process support in that sense that it interacts sooner with the customer in the sales funnel, trying to convert information seekers and information producers into consumers.

Technology vendors like Salesforce.com and Sugar CRM have been working hard to produce support for social CRM and others are following their lead. Social CRM is about a meaningful dialogue as researchers of Penn State, Duke and Tilburg University found out.

Establish a meaningful dialogue with your customers

In their article “SOURCES OF CONSUMERS’ STRESS AND THEIR COPING STRATEGIES” (European Advances in Consumer Research Volume 4, 1999, Pages 182-187, by Mita Sujan, Harish Sujan, James R. Bettman and Theo M.M. Verhallen) talk about facilitating choice both online and in store, only five years after the Internet became available for commercial purposes:

Marketer Interventions for Consumers Stress and Coping.

As suggested earlier, marketers can help consumers cope with their stresses by enabling them to use more effective strategies for coping. For example, retail stores can provide more in-store personnel that stressed consumers can approach for help. Additionally, marketers can facilitate the development of consumer self-efficacy through the environments they create. One way to achieve this may be through consumer educational programs (at the point of purchase, over the web) that teach consumers skills by which to make better buying choices, use products more appropriately and to dispose them more responsibly

 

Conclusion: retailers become information brokers, in collaboration with producers.

Managing information online and in store and presenting this information in a timely and accurate manner will help the shopper cope with choice stress. Convenience shoppers will greatly appreciate this approach and are ready to pay a premium price for this service. Our self-service economy has become so time consuming that consumers with spending power have become more than ever aware of the time = money equation.

Combining data from producers about product perception and experience with shared information between producers and retailers about product preferences and what I call “the shopping logistics of product choice”.

If you want to know how this is done, don’t hesitate to contact us at contact@linguafrancaconsulting.eu    

---

[i] I refer to my definition of the term in the article “What is Really “Big” about Big Data” which you can find here.  

Gurus of BI in Oslo

Sound and vision in the country of Grieg and Munch

The tenth June, Oslo Spektrum was packed with 400 attendees for the second Gurus of BI conference. Yours truly made a small contribution. My presentation on BI and workforce management can be found here.

I guess both Edvards would not have been impressed by the quality of the sound and vision but at least the PowerPoint slides are self-explanatory.

Don't you love KD Nuggets?

I know, it's just a pop poll, but if the big commercial and self-proclaimed market leaders in statistical analysis can't incite their users to vote for them, then I consider this poll as interesting information.
Click on the KDNuggets link for the full story.