Greater Greater Washington

What's up with NextBus, part 2: A pile of APIs

Why do some apps for getting bus predictions work with some DC-area bus services, like WMATA and the Circulator, but not others, like ART and Ride On? Why couldn't the NextBus DC app just use the same data source that other apps do? Why is this all so complicated? The answer lies in APIsapplication programming interfacesand fragmentation.


USB cables are standard; real-time bus APIs need to be. Photo by osde8info on Flickr.

Previously, we talked about how the NextBus DC app went away because they were getting their data from NextBus Information Systems, which lost its relationship with NextBus Inc., the company powering the WMATA bus tracking web and phone tools known as NextBus.

The plethora of things called NextBus aside, my first question when the NextBus DC app went down was, why can't they just reconnect their app to a data source that isn't broken? If the bus locations still exist, and the bus predictions still exist, and there's nothing wrong with the app's code itself, we should look at why it's not easy for them to simply bypass the broken link in the chain.

To understand what's going on, we have to delve a little more into APIs. An API, or application programming interface, is a way for one computer program to contact another computer and get certain information directly, in a structured format, without a human having to be involved.

For example, Twitter has an API, and if you're writing a software program that accesses Twitter, you can have it talk directly to Twitter to post tweets, search tweets, and so on. I put code on the Greater Greater Washington system so that when a post goes live, it also automatically posts a tweet that the author or editor have written ahead of time, without a human having to go onto the website and click around.

Each API has a certain vocabulary. The asking computer users certain terms, and gets back data in a certain format. Other APIs have different words and different formats. If one API breaks but there's one using the same vocabulary and formats on another system, it's trivial to just have the app connect somewhere else. If the API is different, the software writer has to redo the code, maybe just a little, or maybe quite a lot.

NextBus DC app was not using the "official" API

WMATA contracts with NextBus Inc. to run the bus prediction section of wmata.com and a text message and phone service, but not for an API. For other systems that contract with NextBus, it also offers an API for developers as part of its package of services. However, that is not available for WMATA Metrobus predictions.

A few years ago, WMATA embarked on a pretty ambitious project to offer all kinds of data, including bus predictions but also rail predictions, rail station locations, bus stop locations, schedules, elevator outages and more. Because they have this service, said WMATA spokesperson Dan Stessel, they have asked NextBus not to offer its own, different API.

However, that NextBus API is actually what the NextBus DC app was using, because of the legacy agreements between NextBus Inc., NextBus Information Systems, and AppTight. When those expired, that API went away. AppTight could have probably redone its app to use the WMATA API, but that would not have been an easy task.

Is WMATA right not to let NextBus use its own API? There are definitely some valid reasons for this. Stessel explained that if WMATA let app developers use the NextBus API and then WMATA decided to end its contract with NextBus, all of those apps would break. Plus, there is a lot of other information in the WMATA API, so people building apps on the WMATA API would find it very easy to also show next train arrivals, for instance, while anyone using the NextBus API couldn't.

We need standardization

API formats are particularly important because there are a lot of transit agencies, across different cities and even within our region. If they use incompatible APIs, then it's difficult for app writers to support all of them, and smaller bus systems get left out.

The bigger the potential audience who might pay a buck or two for an app, the more app developers will build transit apps. If they can build one app and have it help riders in DC, New York, Chicago, Los Angeles, etc., that's a lot more incentive to build something than if it just works for one city. Small cities especially benefit here, because not as many people will want to build an app for the bus system in Charlottesville, but if the Chicago app works for Charlottesville too, great.

The same logic applies to bus systems here. Some apps work with the WMATA API but don't support any of the regional bus systems. The DC Metro Transit Info app has Metrobus and also supports Circulator, Fairfax CUE and PG The Bus, all of which work with NextBus and support the NextBus API. ART and Ride On have real-time APIs, but they're not the WMATA or NextBus APIs, and the author of DC Metro Transit Info hasn't done the extra work to integrate those as well.

What needs to happen is that all transit agencies and app developers need to coalesce around one API format. WMATA should modify its systems to offer apps the option of making their requests and getting data back in this standard format. So should NextBus. So should ART and its provider, Connexionz, and Ride On, and New York MTA, and Chicago CTA, and everyone else.

It's similar to power chargers for cell phones. Once, every phone had a different plug. You had to use a special charger just for that phone, and if you got a new phone, your old chargers were junk. Now, almost everyone except for Apple use micro-USB, and all the chargers for my 2½-year-old Android phone work on my brand new one as well.

Fortunately, WMATA is open to changing its API to a standard. Stessel said,

Over the course of the next six months, we will be reviewing our API effort in full, and determining ways to improve the service. Standardizing the format is a definite consideration. However, current applications must be taken into consideration... Short answer: Yes, it is something that is being considered.
If WMATA just switched its API, all existing applications would break, just like NextBus DC did. They could simply offer 2 APIs, but for how long? It creates extra work to have to maintain multiple APIs far down the road. They could switch APIs and offer both for a transition period, perhaps a year, but no matter what some apps won't make the switch.

There's a big obstacle to all agencies moving to a standard API, however: it's not yet clear what the standard should be. If the USB of real-time bus data is out there, there isn't the consensus around it. In upcoming parts, we'll talk more about the API standards that exist today.

Plus, having a standard API is great, but it's useless if the actual bus locations are not good, and many say WMATA's data is just not up to snuff. We'll talk about that and their efforts to fix the problems with bus tracking.

David Alpert is the founder and editor-in-chief of Greater Greater Washington. He worked as a Product Manager for Google for six years and has lived in the Boston, San Francisco, and New York metro areas in addition to Washington, DC. He now lives with his wife and daughter in Dupont Circle. 

Comments

Add a comment »

Thanks for explaining this, David. But I'm always cautiously optimistic about developing standards:

XKCD comic

by Gavin on Jan 18, 2013 11:49 am • linkreport

I was actually planning to use this comic in a later part. I think of it every time a discussion of standards comes up.

by David Alpert on Jan 18, 2013 12:08 pm • linkreport

Would using General Transit Feed Specification-Real Time (https://developers.google.com/transit/gtfs-realtime/) be a solution?

by GP Steve on Jan 18, 2013 12:09 pm • linkreport

Luckily some open standards (free to use) for real-time information already exist:
SIRI is a public transit standard for real-time information that is used in Europe and also used by NYMTA.
TCIP is a standard developed by APTA that has a real-time information component but is designed to connect lots of transit ITS systems.
GTFS-realtime is a standard developed by Google.

The problem for a lot of transit agencies is that they are beholden to NextBus or whoever their provider is, and those companies want to use THEIR API rather than produce an API in a standard format. Smaller agencies especially do not have the expertise or staff time to create and manage their own APIs nor do they have the expertise to put this stuff up in the initial contract.

by MLD on Jan 18, 2013 12:17 pm • linkreport

WMATA should be using GTFS-realtime. Enough said.

by Kurt Raschke on Jan 18, 2013 12:28 pm • linkreport

GTFS-realtime helps huge data consumers like Google, but it isn't very helpful to small developers.

The agency or another public sector partner needs to provide a SIRI interface as well, or very few apps will get built.

The nice thing about NextBus and (to an extent) the existing WMATA API is that they give you the basic functionality you need to build a real-time app.

by Matt Caywood on Jan 18, 2013 12:55 pm • linkreport

What I don't quite understand is why ART buses don't show up in realtime on Google Maps. (The static schedules have been there for a long time.) Arlington County provides GTFS and GTFS-realtime feeds (http://www.arlingtontransit.com/pages/rider-tools/tools-for-developers/). Do county transit staff need to work with Google to make this happen?

by Zach on Jan 18, 2013 1:06 pm • linkreport

What's really annoying is that the NextBusDC app for iPhone works PERFECTLY for all the services in the area (that use the technology) *AND* it's feature-rich with stop locations, bus locations, maps, etc. But now that I'm on Android the two best NextBus apps I could find, both suck pretty hardcore. You can't search based on stop, stop number, route, or anything. It just suggests some routes that might be nearby you but the list is hardly ever inclusive of all routes nearby and I find myself frustrated by it. I'm not going back to the iPhone just to get bus predictions so I hope the Android developers step up their game...

by Matthew on Jan 18, 2013 1:25 pm • linkreport

@Zach - ART has made their data widely available in open formats -- see the Mobility Lab list of local transit open data.

As I understand it, ART has made every effort to be in Google Transit but cannot be at present. The reason is that Google requires indemnification in their contract to accept the data, but Arlington is prohibited by the state from doing so. A statewide change is in the works but it is currently stuck in AG Ken Cuccinelli's office.

I hope he makes it a priority :/

by Matt Caywood on Jan 18, 2013 1:28 pm • linkreport

@Matt Caywood
The reason is that Google requires indemnification in their contract to accept the data, but Arlington is prohibited by the state from doing so. A statewide change is in the works but it is currently stuck in AG Ken Cuccinelli's office.

Interesting, but there are tons of Virginia transit agencies on Google Transit so it seems weird that they would need some immunity from the state to participate.

by MLD on Jan 18, 2013 1:53 pm • linkreport

I'm pretty sure at #Transpo camp the fellow from ART mentioned that "we're lucky that our real time prediction provider decided to support GTFS-Realtime". While i think it's great that they support it, it's more or less by luck.
Here's the scoop. There are two standards. SIRI and GTFS-Realtime. Both are fairly new (the folks at nyc told us they modified SIRI by "Breaking" it to provide more functionality, which is what a lot of the early adopters in europe were looking to do).
GTFS-Realtime is geared towards a complete snapshot model. Developers would subscribe to the snapshot and individual requests would go to the developers.
SIRI is more of the standard API model. Query on each stop just like the WMATA or Nextbus API works.
Honestly, providing both standards and letting the developers decide which one works for them with the same data source would be a bit more work, but will pay off in the long run.
Just realize that the standards have been in the works for years, but when the WMATA Api came out, there was no clear standard and GTFS-Realtime was just a twinkle in a researcher's eye at the time.

by RC on Jan 18, 2013 3:50 pm • linkreport

GTFS-realtime would be great since it can integrate into Google Maps, but as RC says it works in a fundamentally different way.

As for the concern about breaking apps, it's only a concern if you change APIs all the time. You could introduce the new API, and keep the old one for 6 months while spamming your development list saying the old API is going away on a certain date. Then just turn it off. People can move on to other apps.

If this NextBus "crisis" has taught anything it's that transit agencies should:
1. Have an app gallery PROMINENTLY on their website
2. Go out there and actually look for apps that exist and put them on your gallery
3. Have a convenient way for app developers to submit their apps for consideration

WMATA fails on all three, they do have an app gallery but I bet you can't find it!

by MLD on Jan 18, 2013 4:45 pm • linkreport

@MLD - I wish I remebered the explanation but it was something to do with Arlington being a county and a city.

by Matt Caywood on Jan 18, 2013 5:44 pm • linkreport

Us Fairfax Connector riders are lucky... we don't have any of these API data feed issues since there is no real time arrival information .

by Transport. on Jan 18, 2013 5:49 pm • linkreport

@Matthew
I highly recommend DC Metro Transit for Android. It's an excellent app. The paid version is well worth the price at $2.99. JazzMoon Studio developed it.

by Phil LaCombe on Jan 19, 2013 4:57 pm • linkreport

Our Nextbus feed is working fine. Why isnt DCs?

by Steve on Jan 21, 2013 7:32 am • linkreport

Unifying on a single standard is a hopeless idea. As RC stated, NYC MTA rightfully found the SIRI "standard" to be completely lacking and therefore modified it creating yet another "standard". Plus SIRI is managed by a committee, which is not the most effective way to deal with rapidly evolving technologies.

And GTFS-real time, as mentioned by Matt Caywood, is for large data consumers like Google, not for smartphone apps. Google created the format specifically for their own internal needs. And look how few users there are in the US.
The NextBus API is used by far more end users than GTFS-real time.

And it does seem rather ironic that WMATA is breaking apps now by not allowing other feeds, such as the NextBus feed, because they are worried about possibly breaking them in the future. Really quite short sighted.

But there is a simple solution. Promote open data, not specific APIs. And then see which APIs the app developers gravitate towards. Then emphasize those few APIs.

by Ski on Jan 21, 2013 12:24 pm • linkreport

If a standard is not a standard, another approach is to use a gateway that exposes one API to consumers (of the API) but connects to other APIs in the background.

by @OneAPI4SMS on Jan 21, 2013 2:38 pm • linkreport

@Ski:

Pray tell, how would you access "open data" without an API? APIs in this sense are basically just a way to tell you where to find the open data and how to read it. Without a standard to do that, you basically focus every developer to start from scratch to read data from every provider, and you have fixed absolutely nothing.

by Joshua Cranmer on Jan 21, 2013 3:37 pm • linkreport

The NextBus feed is good, but it's vendor specific. If WMATA chose to replace nextbus, all the apps would break, thus it created it's own. That and Nextbus deals with bus predictions. Last time i checked there were a good real of riders who depended on the rail side of things.
SIRI v2 (whatever nyc is using) is a good suggestion for WMATA to move to. A good way to entice app developers to move and ask for a standard is the fact they can resell their apps for other areas across the country by re-pointing the data source instead of having to develop a whole new interface.
If you back off the WMATA focus for a second and do a cross comparison on what transit agencies around the country are doing, you'll see that it's a complete mix of non standardization.
NYC - SIRI (for bus) and GTFS-realtime (for rail)
LA - internally developed API and Nextbus API
CTA - Clever Devices BusTime API, separate rail API by QuicTrak
SEPTA - internally developed API
Muni - nextbus API
WMATA - internally developed API that includes bus and rail
comparatively they're not better or worse off. Since no one has a standard that's universal they went with a solution that works. Is it the best one? hard to say, but it's in line with the rest of the country.

by RC on Jan 21, 2013 6:35 pm • linkreport

So, does any of this explain why the RideOn "Real Time" data are so unreliable?

by alurin on Jan 22, 2013 1:57 pm • linkreport

Add a Comment

Name: (will be displayed on the comments page)

Email: (must be your real address, but will be kept private)

URL: (optional, will be displayed)

Your comment:

By submitting a comment, you agree to abide by our comment policy.
Notify me of followup comments via email. (You can also subscribe without commenting.)
Save my name and email address on this computer so I don't have to enter it next time, and so I don't have to answer the anti-spam map challenge question in the future.

or