not everything is public– that’s a lot of work.
I spent most of the week researching state government sites to see what if anything I could do to help Sunlight Labs get the 30 remaining states [or so] working with their API. This included adding Python plugins to my NetBeans IDE and reviewing their existing code.
Most of yesterday, Friday– I spent looking at the Iowa legislative site- mainly at legislator data. I even called their data center– explaining I was a private software consultant [with QiSoftware] and wondered if they had a separate means for developers to access their data. I had a very nice return call from a gentleman that told me the data was not their data but instead purchased.
Over the years, I would like to think I have become expert at finding a way to access most functions on any given web site. I spent most of the day yesterday trying to access specific Iowa data via direct html requests– to no avail.
Around 5:00 pm I gave up and went in search of something else. It so happens, last week [think last Saturday]– I requested several API keys from different providers. In one case I was not approved until Monday. I did nothing with the new API access from the last, thinking I would volunteer for SunLight Labs to find ways to access those states they had not setup calls for.
Iowa helped me to decide all that work [programmers call that scraping a site]– and if they changed something– I would be right back where I started.
That’s when I reexamined the API access provider of Monday. I was able to get all data insofar as state legislators working with the new interface on the WiredPages Government & Education page by 8:00 pm last night. I had a little bit of overhead to learn how the API worked– but once I started.. it came easy. I installed the new interface last night- and still beta testing, so let me know if you have any problems.
I am still using SunLight Labs resources for federal legislator data– and have no plans to change this. I also like the fact- Sunlight is a partner with the new data source.
In my research of some of the code– I discovered some of the real code was not made public. I get that.. I seldom allow my code to be public. A lot of work goes into “scraping” data from public sites like state government sites. I repeat.. I get why…
Again let me know if you have a problem with the new interface.