EconTech

Gabriel is taking requests, and someone wants to know where to get data.

Request #10: Data Sources

Starting on the requests. Today, Mike request (comment #10):

There are a number of key macroeconomic indicators generally used, such as GDP, CPI, and others I can’t begin to name. Is there some master list of who publishes what, and when, and where these can be found? I’m curious to see it “from the horse’s mouth” rather than through various news filters.

For EU data, I usually use EuroStat. You should make an account (free) which allows you to use the data extraction wizard (the little “HTML” icon) which I find more convenient than the alternatives. My main frustration with this site is that you can’t see how far a series goes back, or whether it’s available for a particular country, without actually downloading it.

For the US, a good place to start is St. Louis Fed: Economic Data – FRED® The only complaint I have here is that the series definitions and method could be more visible. Other than that, it’s a great resource. The Bureau of Labor Statistics offers a large set of time series. I haven’t used it, personally, but I saw it referenced a lot.

The World Economic Outlook Databases (from the IMF) might help with the rest of the world, although I never took a good look at it.

Finally, the American Economic Association’s Resources for Economists page has a Data section, a directory of data sources, including micro data.

If someone knows any good sources for Asia or Africa and maybe for real-time data and whatever else I’ve missed, please do comment!

Unfortunately, a ton of data is locked up in nasty formats and incredibly clunky interfaces. I remember some data sets that I required for my thesis required hours of clicking (this was before I knew scripting languages that were capable of crawling the web. oh the time Python could’ve saved me on that). Maybe someday, Hans Rosling will make easy access to data, until then, if you really want to work with data, you’ll have to put in some time.

Data sources (many of these may be gated, go to your library and you may have access):

• Journal and author websites. This is not incredibly common, but is becoming more so. If you see data in a paper you can’t find somewhere else, contact the author or their secretary and ask for the data. If they have a scientific outlook on the production of knowledge, they should be happy to send it to you. I hope your sarcasm detector was on. I have never received a response from a request for data. Notice how I publish source code for everything I do on here? Few people do that.
• Penn World Tables
• OECD: Holy shenanigans. This is open now. I am pretty sure this is usually gated. If you want OECD data, NOW is your time to grab it: “OECD.Stat contains complete databases and is freely available as a beta release via OECD’s iLibrary, SourceOECD for the first half of 2008.” Can you say tempted to scrape those pages for all their data?
• Current Population Survey: Maybe more micro-y, very US centric.
• Economic Census: Very detailed data on the US, with things like firm size.
• Bureau of Economic Analysis: Some international data. Some relatively old data, to 1929.

What I want to know is, where do you get historical data? You see papers written with the use of privately collected data. I don’t know that I’ve ever seen any such data just available to download.