The FEC’s Disclosure Data Catalog
October 28th, 2009 | Published in Campaign Finance, Fed Data | 9 Comments
The good folks at the Federal Election Commission launched a disclosure data catalog recently, continuing the federal government data catalog trend. And while there are few (if any) people better at explaining campaign finance data than the FEC’s Bob Biersack, the data catalog is a work in progress and has room for improvement.
It should be noted that the FEC has been giving away campaign finance data in bulk form since 1979, and has remained admirably consistent in its approach. Text files stored on FTP servers still work quite well, thank you, and while that system could use some improvements, too, plenty of campaign finance reporters rely on it.
But in launching a new product with new data, the FEC has a chance to really do things right. Take, for example, the definitive listing of leadership PACs, those committees used by politicians to help boost their standings within the party. The new data catalog has a listing of leadership PACs, which sure beats tracking these by hand like I used to do. But it underscores one of the fundamental issues involving FEC data: the lack of an ultimate candidate ID.
Take a look at that list – notice how the “sponsor name” and “affiliated committee name” are interchangeable (and, what’s more, leadership PACs aren’t supposed to be affiliated with an official campaign committee)? The FEC has no single equivalent of the unique committee ID that can exist across multiple election cycles. There is a unique candidate ID, but if a candidate runs for more than one office he or she gets additional candidate IDs, with no easy way to tie them to each other or back to the person.
Until that system exists, it’s difficult to track the full extent of a politician’s campaign finance activity – yes, most of them only run for a single office, but that happenstance is not a pillar of good information management. The consequence is that organizations that want to tie together a politician’s activity need to invent their own ids for doing so, which means duplication and an inability to share across data sources. Now that campaign finance APIs exist, this is a bad idea that needs to be fixed.
Congress itself has a permanent unique ID for members regardless of whether they serve in the House, Senate or both. The FEC should do the same, beginning with the current election cycle and working backwards for current candidates. Might be something worth taking to the FEC’s new data blog.
October 28th, 2009 at 11:05 am (#)
What’s the onus for this move by FEC? Was there some new requirement passed into law?
October 28th, 2009 at 11:14 am (#)
Bob Biersack was put in charge of better coordinating the FEC’s web offerings a little while ago, and the leadership PAC list in particular is a response to a law passed by Congress mandating broader lobbyist disclosure.
October 28th, 2009 at 2:04 pm (#)
Thanks for the info.
I support watchdogs more closely evaluating whether government data disclosure efforts live up to the letter and spirit of their legal mandates. Thank you for doing this.
Politicians can get a ton of good press for putting these things into law, but if the data products aren’t executed successfully then the whole thing doesn’t amount to much.
Will any of these new FEC products replace or greatly change your current campaign finance data collection process?
October 29th, 2009 at 8:13 am (#)
Admirably consistent? Are you kidding? There’s nothing admirable about those text files, which have changed exactly once since 1979.
* Not when the FEC has been collecting full donor addresses since 2002 for all but Senators and the DSCC (last I checked the NRSC files electronically — only DSCC is a holdout), and cannot seem to bother itself to get those data out there except through the extraordinarily complex electronic filing system.
* Not when there is a mandate for committees to report names as five discreet data fields, yet it continues to shove them into a single field.
* Not when they STILL have employer/occuptation (again, separate data fields) shoved into a single field in the FTP data that is too small to accommodate them.
Frankly, my patience is wearing thin on this front. This has been the one thing people have been asking for for years, and it’s the easiest thing in the world to figure out: Keep the existing files, and create a new set of data files that actually reflects the data they collect. Done!
Hell, I’ll even volunteer to write the SQL to export those files if the FEC promises to actually produce them.
October 29th, 2009 at 9:30 am (#)
The consistency is admirable in terms of people being able to use data across time. It’s not admirable in terms of overall usefulness, as you rightly point out. And electronic filing is still an unwieldy mess that has made for two classes of campaign finance data users: those who have the time and ability to figure out the electronic data and those who cannot.
October 29th, 2009 at 1:44 pm (#)
Derek,
The “two classes” of users description is apt. I’ve been working daily with this data for months and every day I discover a new quirk. I can’t imagine anyone — even a seasoned data analyst — diving into these data sets and doing anything moderately sophisticated (and accurate) in a short amount of time. The learning curve is just too high.
The Sunlight Foundation folks are on the money with their message to the FEC: Focus on data quality and accuracy first, then worry about the bells and whistles.
October 30th, 2009 at 7:22 am (#)
Hi Derek,
The quality of the data isn’t about to change. Why? The FEC isn’t going to enforce ‘quality’ reporting. This isn’t a criticism, but a realization that the law is so quirky, with so many exceptions, that errors are going to be common. Unfortunately, those few who chose to obfuscate in a critical reporting period will likely be able to do so in the short run.
Unless the FEC mandates donors to have unique identifiers (say for instance, SSN – something never likely to happen), there will always be uncertainty when aggregating donors by name. The same is true for ‘intent’ of donation…is it because of employment or ideology?
Although we in the data world like clean, constrained data, it’ll never happen with public records. The agencies have too small a budget and too little time to make timely checks on filed information.
In other words, it is what it is, and only good follow up reporting will make for uncertainty in data with so many anomalies.
BTW, almost all of the recent changes mandating ‘improved’ disclosure have in fact confused things. Much of the ‘extra’ data is duplicate (lobby bundling, for example).
One huge improvement would be to limit itemization of donors aggregating over $1,000 per cycle…that’s right, actually reducing disclosure. It would instantly make the data more manageable, protect privacy and let us zero in on the true heavy weights in our little world.
October 30th, 2009 at 7:24 am (#)
Oops! I meant good follow up reporting will make for certainty…not uncertainty.
November 10th, 2009 at 12:49 pm (#)
Weighing into a discussion between Aron and Derek is like getting between two elephants. In this case, they are both right.
It seems trivial for the FEC to move the address of donors fields from the electronic files into the FTP files and most of the other things Aron suggests.
But as one who has worked with these things for a long time, I also am glad that the FTP filings remain stable. The single identifier would be a great thing (try doing a history on Hillary or Obama without it).
What I fear is that the Senate will finally be forced to go to electronic filing and the FTP files will go away. The electronic filing system, dictated by the software companies, is beyond horrible.
The other point I would make is that you should not expect “data quality” from the FEC. As a regulatory agency it is purposely weak (mainly regulating members of Congress), and I don’t think the agency uses the public disclosures in its regulatory audits of campaigns.