….::: VOX POPULI :::….

New Year, New Blog

Posted by Mark Headd under General Discussion

The final days of 2011 are ticking away, and so too are the final days of this blog.

I’ve been writing about voice technologies, phone apps and open government under the Vox Populi monicker for a long time.

I first registered the voiceingov.org domain in 2002, and built a very basic website to share my excitement and enthusiasm for an emerging set of standards for building telephone applications using web technologies.

My interest in this area was spurred by my involvement in helping create the e-government initiative in the State of Delaware, and launching projects like the Delaware Internet Access Locater.

I have always believed in the importance of communications technology in general and telephone applications specifically for engaging citizens and encouraging an open dialog between governments and the people they serve.

This interest was cemented when I left state government to work as a technologist, using the latest and greatest tools for building communication apps of every stripe.

I’ve continued to write posts on this blog, trying to emphasize the core message that these new technologies can make it easier and more efficient for citizens to connect with government, even as my interest and work has expanded more broadly into other areas, like open government and civic hacking.

More and more, I’ve felt like I’ve been shoehorning posts on these topics into this blog which has an admittedly more narrow focus.

In addition, I’ve had much less time to devote to writing posts for this blog - I write extensively for my company’s blogs as well as for other blog sites covering open government and the civic hacking space.

With all of that said, I’m happy to say that I will soon announce a new direction in my writing and blogging efforts. This announcement, sadly, means the end of Vox Populi as it is currently constituted.

It’s hard to let go of something I’ve been working on for so long, and that is central to the way that I think technology should work to bring government and citizens together.

But the tension that currently exists between my current work and interests, and the narrow focus of this blog can’t continue.

In 2012, I’ll launch a new site where I will write more extensively about open government, open data, civic hacking and more of things that will change the way that government works and they way we interact with those that represent us.

It’s been a great run with this blog, and I’m still proud of many of things I wrote and all of the things I tried to do with it.

Looking forward to 2012.

Apps for SEPTA Recap

Posted by civic_io under Open Government

At some point in the next few days, I’ll have time to do a proper recap of the civic hacking event I organized with the gang from Devnuts to build transit apps for the Philadelphia region, but in the interim I’ll provide this short post.

The event was awesome!

We had a full house of hackers, and a good number of non-technical participants joined us on the first day to offer ideas. Our partners and sponsor stepped up big time to make the event a pleasure from start to finish, and also to provide some neat incentives to the winners to continue to develop their new transit apps. (I’ll be writing more about these incentives in a follow up post.)

For now, here is a round up of the media coverage from before and after the event:

• Real Time Data (Philly.com)
• Apps for SEPTA hackathon features new data sources and mass transit projects (TechnicallyPhilly.com)
• SEPTA sets a good example for open government, seriously (Philly.com)
• SEPTA-Hacking: Philly Transit’s Digital Upgrade (CityPaper.net)

A video put together by one of our terrific sponsors - TicketLeap:

And some pictures I took at the event throughout the weekend.

Be on the lookout for a follow up post with some thoughts on lessons learned from this event that can be used to improve the results of civic hackathons in the future,

Innovations in Civic Hacking

Posted by Mark Headd under General Discussion

There are some exciting things happening in the world of civic hacking, and some cool innovations are being used to make civic hacking events more exciting, and to perpetuate the value of hackathon projects.

Last night in San Francisco, a forum of mayoral candidates was presented with the top projects from a Summer-long series of hacking events called (appropriately) the “Summer of Smart.”

I wasn’t able to be at this event, but I did attend an earlier San Francisco mayoral candidate forum that kicked off the Summer of Smart. By all accounts, the event last night was a great success and there were some awesome projects presented to those who would be Mayor of San Francisco.

I love the idea behing Summer of Smart - holding an intensive, tightly clustered series of events over several months and then “jacking in” to the political process by presenting the outcome directly to mayoral candidates in a public forum.

I’m also really interested to learn more about the new “civic residency program,” which will provide resources and space for hackathon participants to continue to develop their projects. If it works, this could become a model for other parts of the country.

Another innovative civic hacking event is taking place in Reno, NV - the Hack4Reno event gets more interesting as the days go by.

The event’s chief organizer - Kristy Fifelsky, of GovGirl fame - has started a video series providing insights on how to organize and run a civic hackathon. Insights, tips and tricks from someone who is actually hands on with organizing a civic hackathon could be extremely valuable to other municipalities that want to hold similar events.

One of the most interesting tidbits about the Hack4Reno event from the first installment in this series - the hackathon will take place out in the open, literally. The current plan is to hold the event outside the Pioneer Center in downtown Reno. How awesome is that!

There appear to be contingency plans if the weather doesn’t cooperate, but I love that the organizers are thinking outside the box.

Another cool detail from the Hack4Reno event is that almost all of the planning for the event is being done using GitHib to assign and track work. This is a great way to encourage collaboration and underscores the commitment of those involved to the idea of transparency. Everything about this event is open.

Really looking forward to seeing more innovations from civic hacking organizers.

Keep em coming!

Speech Recognition for Open311

Posted by civic_io under CouchDB, Open Government, Open Source

Really excited about a new project I started recently to enable phone-based speech recognition for 311 service requests.

Here is a screen cast demonstrating the solution.

Saying Goodbye to TweetMy311

Posted by civic_io under Open Government, Twitter

Sometimes in order to move forward into the future, you need to let go of things from the past.

This weekend, I’m officially decommissioning the TweetMy311 project, an Open311 project I launched over a year ago. The application is no longer active, and the TweetMy311 twitter account won’t respond to any more mentions.

The process of building and launching this application was a great learning experience for me. I got to eat my own dogfood in a sense - I had previously developed some libraries for using the Open311 API and I got a chance to use them in this project - always a satisfying thing.

The original idea behind TweetMy311 isn’t going away. In fact, it’s the impetus for several other things I’m working on, including the Mural Art Mapper project that is active currently in San Francisco and soon will be in Baltimore.

Ultimately, however, usage of the service never really took off. The goal of this project was stated plainly on the project website:

“… to learn more about building Open311 applications, and to share that information with developers that want to improve their communities by participating in the Open311 project, or to build Open311 applications.”

I’ve certainly had the chance to realize this goal.

I’ve worked with developers at Code for America this Summer (as part of the Google Summer of Code project) to develop PHP libraries for a number of civic APIs, including the newest version of the Open311 API.

I’ve developed libraries for the Open311 API in other languages (e.g., C#, Node.js) and I’m currently working on a speech-enabled address capture application that can be incorporated into 311 phone apps.

It is also deeply gratifying that TweetMy311 was able to inspire other services - like MarkASpot - to incorporate Twitter into their reporting services.

The TweetMy311 project was a great opportunity for me to be active in a civic hacking space that I care about and that I want to contribute to.

But the time to wind the project down has now arrived. Thanks to all of those that have supported the project in the past, and thanks to the good folks at OpenPlans for their leadership in moving the Open311 standard forward.

Onward, upward.

The Road Ahead for Open Baltimore

Posted by civic_io under Open Government

A couple of months ago, I wrote about the state of the open data program in the city of Baltimore.

At the time, the buzz from a day of civic hacking with data released by the city was still palpable and the developers of an application built in the wake of this event stood ready to release it for public use with a full marketing push.

Questions remained about how developers would be authorized to use city data for commercial applications, and - perhaps more importantly - how often the data released as part of the Open Baltimore initiative would be refreshed and maintained.

At the recent Baltimore Data Day event - organized by the Baltimore Neighborhood Indicators Alliance - I had an opportunity to meet Rico Singleton (CIO for the City of Baltimore) and some of his staff to talk about Open Baltimore.

Based on what I heard from Mr. Singleton and his staff, I think that significant questions about the open Baltimore initiative still remain, but I am now more convinced of the city’s commitment to open data and to the long-term success of the program.

An Outsider’s Perspective

I don’t live in Baltimore so I sincerely hope that my perspective on the city’s open data program will be interpreted for what it is: a representative view of the many people who have friends or family there, who visit often, who care deeply about the state of the city, and who are interested in building civic apps for Baltimore.

To the extent that there are political undercurrents to what is happening with open data in Baltimore, I’m detached from them. I won’t (and can’t) vote for any political representative in Baltimore and I don’t (and won’t) contribute money to any of their campaigns.

I have, however, worked with other cities to provide advice and support for their open data programs - most notably the cities of San Francisco and Philadelphia - and I enjoy being active in the civic hacking ecosystem.  Outside of the city of Baltimore, I don’t think you’ll find a bigger supporter of the Open Baltimore program or someone rooting harder for its long-term success.

I was encouraged by what I heard from Mr. Singleton and his staff last week at Baltimore Data Day, but long-term success for Open Baltimore still seems somewhat in doubt.

Here is what I heard, and what I would suggest to Mr. Singleton and MOIT for making Open Baltimore a smashing long-term success.

An Open Discussion

Baltimore Data Day provided an opportunity for myself and Mike Brenner (a passionate advocate for open data in Baltimore and organizer of the Civic Hack Day) to meet Rico Singleton in person.  I wanted to use this opportunity to discuss what I thought about Open Baltimore.

Mr. Singleton was very approachable, friendly and eager to talk.  His time was somewhat limited at the event, but we had a chance to chat briefly about the Open Baltimore initiative and I was able to express some of the concerns that I have heard from others in and around the open data movement in Baltimore.

Chief among them was that the data in the Open Baltimore site is quickly becoming stale.  For example, some of the more prominent data sets are now quite old - data on 311 service requests is only current to January, and parking citation data is only current to March.

If developers are going to use city data to build civic apps (or if activists and community leaders are going to use it to advocate for those they represent) this data must be kept more current.  Mr. Singeton (and MOIT staff at the event) seemed to understand this concern, but none offered a clear and convincing response to how the city would address it.

Two representatives from MOIT staff were on hand to talk at the Baltimore Data Day event as well - Heather Hudson (the full time program manager for Open Baltimore) and Tom Jones (technical support for Open Baltimore and other MOIT initiatives).  Both discussed in some detail an effort by the city to provide more realtime updates to the Open Baltimore site, particularly for high demand data sets like parking citations.

Both Hudson and Jones seemed excited and passionate about this effort, but neither provided specifics on when this process or system would be put into regular use.  While I was initially very excited to hear about this effort, it was pointed out to me by others afterwards that this discussions has been going on for some time.

In the limited time that was available, I asked both Hudson and Mr. Singleton if the city would be willing to endorse and actively participate in another data hacking event focused on the use of city data to build civic apps.

Both thought it was a good idea and accepted the offer.  Mr. Singleton shook hands with myself and Mike Brenner on it before departing for another event.

The Road Ahead for Open Baltimore

I’ve said this before and I truly believe it - the City of Baltimore’s open data program has some of the key ingredients needed for real long-terms success.

Chief among them - in my opinion - is the selection of Socrata as the platform for serving data sets.  The Socrata platform is just awesome and the company is a leader in the open data movement, helping develop standards that benefit any government wishing to start down this road.

In addition, there is a passionate and active developer community in Baltimore - as evidenced by the attendance at and enthusiasm from the Civic Hack Day organized earlier this year.

But to capitalize on these things, I believe that Mr. Singleton and his staff need to do some things differently.

Communication

Engagement

In my opinion, the City should do more to reach out to the developer and tech communities in and around Baltimore. They are your best possible allies for making Open Baltimore a success.

I look forward to seeing the Open Baltimore initiative become successful in the long term, and I hope that this advice (from one interested outsider) can help make a difference.

The Future of Real-Time Communications

Posted by Mark Headd under Standards, VoIP

Great interview of Jeff Pulver and Jonathan Taylor by Chris Pirillo on the future of real-time communications.

Local Focus Drives Apps for Metro Chicago

Posted by civic_io under Open Government

Really like the information I am getting about the Apps for Metro Chicago contest.

This OpenGov app development contest seems to have taken a lot of the learnings from other contests and incorporated them into some of the things that define how it will operate.

Case in point, a focus on “local usefulness” - here is a snippet from the judging criteria:

The judges are looking for apps that are built with the input and support of not-for-profits, community organizations and businesses. If you can include a letter from a partner group saying “we’ve used this app and it furthers our mission and/or business” in your submission, you get extra points. If you work closely with a partner group to develop an app that helps them solve a problem and build a plan to integrate your app into their online presence, your app will automatically rise to the top of the pile.

That’s smart, and could result in some first rate apps beng submitted for this contest.

It’s a definite departure from the focus of other OpenGov app contests - like the app contest that was conducted as part of OpenGov West, which focused on apps that crossed over government boundaries and used data sets from multiple governments.

It will be interesting to see how this local focus affects the kind (and number) of apps submitted.

I may have to roll up my sleeves and get an entry together for this one.

“Phind It For Me” Live in Philly

Posted by civic_io under Cell Phones, CouchDB, Development Tools, Digital Divide, Open Government

Really excited to launch a new OpenGov project in Philadelphia - Phind It For Me.

The service is built on PHLAPI and the point data sets it houses. As such, one could understand why I’d be interested in enhancing the data sets currently in PHLAPI.

I’m really excited about this project - source code available on GitHub - and would love to see if there is an interest in launching in other cities with CouchDB-based geospatial data repositories, like Baltimore.

It’s built on the awesome new SMSified platform from Voxeo (disclaimer, I work there) and uses a Node.js module I built for working with the SMSified API.

As always, dear readers, any comments or feedback is welcomed.

Do head on over to the project website and check it out!

Operation Data Liberation

Posted by Mark Headd under General Discussion

I’m working on a neat new OpenGov project that has grown out of a few things I’ve worked on the past.

I’m hoping to launch a new SMS-based service this week that will let anyone quickly and easily find important locations in their neighborhood using an ordinary cell phone (i.e., a non-smart phone).

I’ll launch the new service in Philadelphia, but I plan to open source the code for it so that anyone that wants to launch a similar service in another location will have a code base to start from.

As a data source for the new service, I’m planning on using data sets that are available in PHLAPI. The only drawback to this approach, currently, is that there aren’t a whole heck of a lot of point data sets available in PHLAPI.

That’s not necessarily a deal breaker, as there are plenty of data sets out there just waiting to be liberated and added to the PHLAPI instance. So, as a way of getting the process rolling, I decided to start hacking away at liberating some open data.

One of the more interesting data sets I came across in looking for candidates (pun intended) was a listing of polling locations in the City of Philadelphia. I found a PDF document containing a detailed list of locations (prepared one presumes for the primary election in Philly last week) on the site of the Committee of Seventy.

The PDF document I started with can be found here. There is also a web-based app available on the Committee of Seventy site for finding polling locations, but I wanted the data in as raw a format possible (ideally, CSV).

It turned out to be surprisingly easy to do this. Here’s how I did it.

Converting PDF to Text

Since I run a variety of *NIX machines in my home office, it was pretty easy to use pfdtotext to convert the Committee of Seventy PDF document into a text document. Using the -layout option with pdftotext allowed me to maintain the nice table layout of the original document, and helped with further processing.

From Plain Text to CSV

Once I had the document in text format, it was time to fire up Google Refine. This is an indispensable tool in the arsenal of any OpenGov hacker, and it is enormously powerful for cleaning up and enhancing messy data.

Google refine comes with a built in scripting language called Google Refine Expression Language (GREL) - very much like JavaScript in it’s syntax - that lets you manipulate and refine the data in a project.

I won’t go over all of the steps I used to convert the plain text file I started with to CSV, but if you spend any time playing around with Google Refine you’ll see how easy it is to enhance a data set.

Adding Locational Information

Next it was time to geocode the address of each polling location - something that is not included in the original Committee of Seventy document. One of the really awesome features of Google Refine is that it allows you to add a new column to a data set that is created by making a call to a web service.

This functionality makes it pretty straightforward to use Google’s Geocoding API to get the latitude and longitude for each address in the original file. There are some really good screencasts and demos of this technique on the web, and if you look for them you’ll find some good stuff.

Exporting to CSV

One of the last steps in the process, once the data was cleaned up and enhanced, was to simply export it to CSV.

I’ve loaded a copy of the CSV file I created to GitHub. This format is nice because just about any software program or development tool can consume it (you could, for example, simply open this file in Excel).

But what I really hoped to do was to be able to make this data easy to import into a CouchDB instance, like PHLAPI. Not only would this support my new OpenGov project, it would let anyone that wanted to use the data simply replicate it from a publicly available CouchDB instance.

To do this, I turned to Node.js.

Inserting into CouchDB

What I needed to do with my CSV file was to parse it and turn each row into a JSON-formated document that I could then insert into CouchDB via HTTP POST. This a snap to do with Node.js and a few of the handy modules made available by the Node community (most notably, node-csv and the cradle module)

Here is the script I used (also available on GitHub):

A bit of a hack, but it works just fine. So I now have all of the polling places in the City of Philadelphia in one of my CouchDB instances.

Want to replicate it and use it for yourself? Just run this at the command line (assumes you are running CouchDB locally, and have created a DB named phl_polling_places):

curl -X POST "http://127.0.0.1:5984/_replicate" \
-H 'Content-type: application/json' \
-d '{"source": "http://markh.couchone.com/phl_polling_places", "target": "phl_polling_places"}'

If this post has inspired you to try and liberate some data yourself, let me know. I’d love to help with more efforts like this.

I’m also still in the process of checking my converted data, to ensure that everything looks correct. Once this is done, I’ll work on getting it inserted in PHLAPI.

Stay tuned for my new service, launching soon…!

UPDATE:

Right after I posted this I learned of an even easier way to export data from Google Refine to CouchDB. Max Ogden’s Refine Uploader - worth checking out if you are using data from Google Refine to a CouchDB instance.