Wednesday, January 14, 2004
Nine plus three does not equal twelve
Let me just say, as an opening salvo, that while a few data points over the short run won't tell you anything about where Toto is staked, it might tell you something about where the mail carrier is walking.
Now, back to the beginning. This is going to be a narrative, so put on your story-time hats.
In a discussion of Bush's prospects for re-election I offered the banal observation that it will come down to two factors, the economy and Iraq. With regard to the economy, I claimed that: "As far as the economy goes, Bush desperately needs jobs. While job growth had been slowly trending up, it trended down in November and December. Retail, where most job growth has been taking place, always takes a hit in January. So I'm betting that the economy will lose jobs this month. If that scares the markets or damages consumer confidence, then Bush will have a severe headache. But I have mixed feelings about this, since I hope to eat this summer."
While I knew that I didn't really have the expertise to start making economic forecasts, I did feel pretty clever. But then a certain biophysicist (we'll call her Rosemary) slapped me down:
"Whoa, wait a sec. I'll agree with you that the economy is sucking and that Bush desperately needs jobs, but you're looking at a second derivative which is within the noise of your sample. The trend of the growth for the past three years has a mean of 285 more jobs created (or fewer jobs lost) each month than in the month previous, with a standard deviation of 138,000 more jobs created OR lost each month than in the month previous: it's noisy as hell, trending up only very slightly more than it trends down. It's more meaningful to point to the fact that short-term growth would have to be enormous to recoup the overall jobs lost; and that people are leaving the labor force unable to find jobs."
Ouch. Luckily, a certain physicist (we'll call him Matt) was standing near me when I got Rosemary's message and was able to make the problem clear to me. (Rosemary, by the way, has since put up an in-depth math-illiterate-accessible analysis here)
What I had done was look at the job growth figures from the last three months and noticed that the number of jobs added had declined each month. That is, in October the economy created about 100,000 jobs, in November about 40,000, and in December about 1,000. To me (a math illiterate) this looks like a trend. source
Rosemary's point, as I understand it, is that employment numbers are so volatile that the appearance of short-term trends is practically meaningless. In fact, if you follow that link to her analysis you'll see a graph which vividly shows that, "generally speaking [the short-term job growth trend] doesn't stay positive or negative for very long. In any given month saying 'the trend is positive' or 'the trend is negative' is pretty much meaningless since in the next month or so it's likely to reverse itself wildly." The upshot is that in order to see any kind of meaningful trend you need to take a long enough view to filter out the noise in the sample.
Matt illustrated this with the following example: Suppose that you have a dog chained to a stake and that the chain is fifty feet long, and suppose that you can observe the dog but you can't observe the stake. With enough measurements you'll be able to say precisely where the stake is. But just because you get a few measurements in some direction, that doesn't tell you that the stake is over there, or that somebody has moved the stake, say, six inches to the right.
To go a little deeper into the case, keep in mind that where the dog is at any given time is the result of a tremendous array of factors. Maybe something smells nice over here, or makes noise over there. Those factors--if you knew them--might explain why the data is ordered in the ways that it is. But knowing them doesn't help you figure out where the stake is. (if I'm parsing Matt's metaphor correctly, then the stake represents the underlying trend)
Knowing I was out of my depth, I resorted to an argument from authority. "Economists," I said, "are supposed to know a thing or two about math, and they use these sorts of numbers to talk about trends all the time. Do they know something you don't, or are they just being sloppy?"
Matt said, "they use those numbers because they are the only one's they have."
As a reply this is flip, but not without merit. One of the great problems in economics is to develop metrics which accurately report reality. For example, the job creation numbers we're talking about here have been harshly criticized because the Bureau of Labor Statistics only updates its survey annually, which may lead to underreporting of job growth. But if economists had to develop perfect metrics before doing anything else then they wouldn't do anything except econometrics. So, as Matt says, they make do with what they have.
Rosemary's reply was different, and pointed to the political uses of economic data. The data can be represented by a graph of a line with a, "very slight *positive* slope -- which is to say that on average since Jan 01, more jobs have been created (or fewer jobs have been lost) in each month than in the previous month." This graph could then be used to argue that, "the economy is growing -- fewer jobs are being lost and now jobs are even being created -- and that that growth is, in fact, accelerating." Or, on the other hand, you could create a graph which showed the dramatic loss of total jobs during the course of the Bush Administration (or, you could have this graph, which contrasts the loss of jobs with the rosy predictions BushCo used to sell their economic policy). Rosemary points out that, "since a voter's conclusion will be based on which of these two depictions stick with him, the choice of depiction presented by a given economist or journalist is a political one."
While I don't think anything Rosemary says is wrong, I don't want to believe that it's the whole picture either. I want to believe that when economists muck about in short-term forecasting they're sometimes doing something other than promoting a political agenda.
Which gets us back to the mail carrier. If our data on Toto's location shows that Toto spent some time off to one side this won't tell us that Toto's stake has moved. But the location of the stake might not be the only thing we're interested in. We might, for example, be interested in discovering whether the latest issue of Body and Soul has arrived.
In terms of job growth, we aren't interested only in the question of the (medium) term underlying trend. We're also interested in various short term trends and in the ability to make good predictions. So how can economists develop meaningful results? It seems to me that there are two paths that might be pursued.
First, assume that we are limited to the current data with regard to job creation. This doesn't mean that we can't appeal to other data in order to notice trends or make predictions. For example, we know that retail stores routinely add workers for the holiday rush and shed those workers during the early months of the year. This sort of information may not help us spot a genuine trend, but it might allow us to make good predictions.
But, second, we aren't limited to the current data. The Bureau of Labor statistics publishes this data monthly, but gathering the data is an ongoing process, and the data could be significantly refined. It seems likely that we could track job growth from day to day. This would give us a data set for the October-November-December period which is larger than that produced by looking at monthly numbers over the last three years. Given this larger data set, an economist might be able to make meaningful statements about trends over relatively short time periods.
Of course, even if I'm on the right track with this, my own forecast is pretty much indefensible. But that doesn't mean I'd turn down a bet.