Our first disappointment was that the most recent data for retail industries such as electronics and appliance stores was two years old. These days, for studying anything related to online, 2013 is ancient history. The most plausible defense is that budgets are tight, but surely it is more important to devote resources to measuring in a timely fashion the parts of the economy that are changing rapidly than to keep current on sectors that move glacially.
Worse, though, was when we looked at the published statistics. Scratching our heads, we saw the Census tables showed that general merchandise stores — which is where they code Wal-Mart — had only $88 million in online sales in 2013. That couldn’t be right, since Walmart.com was a multibillion-dollar operation that year.
So we started digging. We pored over the surveys and documentation. We emailed the Census official in charge of the retail trade data. We published our results in an online working paper to help other users of those data and to describe how we tried to use other sources to improve the Census estimates. Then, finally, the Census public information office contacted us, but not to offer congratulations or thanks. They demanded that we retract our paper, and we learned that the Census official with whom we had been emailing had been directed not to directly communicate with us anymore. The public information office retracted, in writing and in a long telephone conversation, what the Census official had told us in multiple emails. They said the process the Bureau followed wasn’t really the process described in the documentation that we had been able to find online.
Oh, and by the way, the data on online sales they report for general merchandise stores — well, that’s not really online sales for general merchandise stores. We were told that their procedures called for online sales of general merchandise stores such as Wal-Mart to be included under non-store retailers along with the sales of food trucks and the like. You may be as surprised as we were to learn that Wal-Mart is a non-store retailer. You might be even more surprised, as we were, to learn that online sales of manufacturers such as Apple and Nike are also supposed to be included under non-store retailers. Think about a food truck selling iPads out of one window.
What, then, is the $88 million reported as online sales of general merchandise stores in 2013? As nearly as we can figure out, that’s the total online sales of small general merchandise stores that didn’t separately report they also operated a non-store retail operation. That is, it’s junk data containing no valid information. If you took it at face value, you would be surprised that giant companies such as Wal-Mart seem to think e-commerce is worth their attention at all.
We retracted our paper, of course, since it had been based on the assumption that the Census e-commerce statistics by industry were biased only because, as the Census official told us, some large companies didn’t report those sales, not because, as we subsequently learned, the industry-specific numbers were essentially meaningless. Census public information did not retract the Census official’s statement that e-commerce estimates were understated.
We think we understand all this better now, though it doesn’t make a lot of sense. But it is hard to be sure. No one audits the work, and the Census people themselves seem to have different views on what was done. Because the rise of e-commerce is such an important development for so many businesses, and because reliable statistics are thus so important to businesses and researchers alike, we’ve called for an independent review of the Census’s retail sales data.
This rather painful episode — which started with us trying to get sensible numbers for one paragraph in a 262-page book — woke us from our slumber when it comes to federal data. We have always had great respect for the professionalism of the federal statistical agencies, and we recognize their chronic budget problems, but the government does suck up massive amounts of time from businesses and consumers collecting data. Relative to that effort, the costs of turning all that data into information useful to businesses and researchers and providing it in a timely fashion must be tiny. The agencies clearly do exactly this in some cases, but we have learned that they don’t do it in all cases. And the public has no idea which published, official data sets are as meaningless and potentially misleading as the $88 million figure that first caught our eyes.
Let’s go back to the question we raised. Brick-and-mortar retailers are now rapidly trying to adapt to a world in which people want to be able to shop where they want, when they want — sometimes in-store, sometimes online, sometimes using their mobile device. A massive part of the economy is undergoing stress as some retailers are being cut down by a buzz saw, while others are making a lot of progress transforming how they do business. Reliable, current information on what’s going on would be valuable, and it could be available but is not. So entrepreneurs, investors, and analysts don’t have much to go on. The federal government has, or could get, the right data and could tabulate it sensibly and report it quickly without adding to the third digit of the deficit.
Given today’s technology, there is no reasonable explanation for why the government needs two years to process data and report results on retail sales. If any data geek in private business needed two months, never mind two years, to go from survey data in hand to final results, they’d be tossed into the unemployment line.
Sadly, the only way to learn which statistics are as reliable as GDP and which are as useless as reported e-commerce sales by general merchandise stores may be to give federal data collection and reporting a top-to-bottom overhaul, perhaps along the following lines:
- Government statistical agencies need to design coding and classification systems to deliver useful information. Why can’t we be told how much general merchandise stores and other retail industries sell online and how much they sell offline? How about firms that are mainly manufacturers? Census already has the necessary data, after all.
- Government statistical agencies need oversight from the research and business communities for everything from what they ask to what they report — that’s why we recommended an independent review of the Census retail data.
- Government statistical agencies need to move quickly to get data out in weeks or months, not years.
- Government statistical agencies need to make more raw data available to researchers in ways that maintain confidentiality. Census and other agencies already do this in some areas; they could do it across the board. If we had access to the raw data on retailing, we could unscramble some of the eggs that the current classification system has so thoroughly scrambled.
It is time to break the government’s stranglehold over all the data that we, as citizens and businesses, hand over to them. Let’s take a small step first: support our call for an independent review of how the Census Bureau collects, analyzes, and reports retail trade data.