Greater Greater Washington

Transit


What's up with NextBus, part 3: Where Ride On is the leader

Which Washington-area bus system was the first to offer its bus position data in an open standard? Would you believe it's Ride On?

In part 2, we talked about how there are many different APIsapplication programming interfaces, the way one computer system, like an app, gets data from to another, like bus positions from a transit agency. The fact that there are so many APIs means many apps don't include all of the types of buses in the region that have real-time positions and predictions.

Prince George's The Bus, Fairfax City CUE and the DC Circulator are available using NextBus, Inc.'s API, which is one of the most common because many agencies contract with NextBus, Inc. WMATA also contracts with NextBus, Inc. but doesn't use its API; WMATA built its own. ART has a different one entirely.

Since NextBus is most common, some residents asked Montgomery County officials why RideOn is not part of NextBus, too. One was Evan Glass, who tweeted last May:

Why is MoCo's Ride On bus system not accessible on the #NextBus app when all other jurisdiction are? #14bus cc @hansriemer (@EvanMGlass)

Note that Glass was talking about the "NextBus DC" app, the one that died this past December and, people discovered, actually wasn't from the same company as the one that provides bus prediction services to many transit agencies.

Councilmember Hans Riemer passed the question on to Ride On officials. Carolyn Biggins replied:

Recently, our staff met with a representative of NextBus to discuss products and costs. Although NextBus has not yet given Montgomery County a firm price quote, they offered a ballpark figure of approximately $55,000 per year for operating costs. This would cover a barebones system which would only have their mobile and desktop web site along with a suite of management tools. There are also undetermined setup fees, probably starting around $15,000 but possibly much higher. ...

At this point the inclusion of NextBus into the Ride On Real Time customer information product line is actively open for discussion. Feedback from our customers and industry critics point us in various directions and toward various apps; and, interestingly, NextBus is not at the top of our customer's request list.

Besides our Eastbanc/Nerds Ride On Real Time App (available for iPhone and android) which, by the way, includes integrated real time data from Metrobus, Metrorail and several Northern Virginia jurisdictions, our customers have asked us to integrate into the "DC Metro Transit App" and "OneBusAway."

We have been working with developers for DC Metro Transit App who recently responded to us with a very encouraging post about our open data: "This seems well thought out and documented. It is also nice that you can get the data in both JSON or XML [the 2 most popular formats for getting data from APIs] in a restful service [basically, a way of making APIs easier for the app developer to use]. I'll give it a try in the app and let you know if I have any questions. You guys are ahead of the curve compared to other agencies."

As you mentioned in your recent e-letter, Open Data and public/private initiatives, such as 3rd party app development, is the wave of the futureto "disrupt and create." 3rd party app development not only unleashes the initiative of the private sector but also provides varied choices for our citizens: the delivery of information in many different formats to suit different consumers with varied needs and tastes.

In developing our Ride On Real Time system, Transit Services has taken this approach, both through internal product development but also by providing its data in as many different formats as possible while trying to maintain fiscal responsibility. We will continue to work with NextBus and other vendors to try and provide Montgomery County citizens the very best in transit information and customer service.

(Notes in brackets added.)

Biggins is right. The solution to the problem of Ride On not being part of many existing apps is not to work with any particular vendor, but to provide open data in more formats.

It's particularly good to hear this from Ride On, because at first they did it wrong, and contracted with a software developer just to build them a website where people can track buses, but with no way for 3rd party app developers (in other words, people who aren't the agency or one of its contractors) to access the data.

Following prodding from Kurt Raschke, us, and others, Ride On started offering an API, and even fairly quickly improved it based on feedback from Raschke and other developers.

Why doesn't everyone just use GTFS?

In the area of transit schedules, one standard has largely emerged as the most common, and one all transit agencies ought to offer: the General Transit Feed Specification, or GTFS. GTFS is basically a set of big files that contain every single stop location and all of the schedules for the transit system. You can download it, write code to analyze it, and then do whatever you want.

There's an analogue of GTFS for real-time buses, called GTFS-realtime. However, real-time is not the same as schedules. With schedules, you can download the whole thing once and it basically won't change except every few months. With real-time bus tracking, the positions change every minute.

GTFS-realtime lets you download the entire set of bus positions as they constantly change. It's a huge amount of data. For some applications, like if you're making a live map showing buses, that's what you want. For the typical smartphone app, where you just want one bus position at a time, it's too much. That much data would overtax the user's data plan and burden the phone trying to deal with it all.

Other APIs, like the NextBus and WMATA APIs, work differently. For those, an app sends it only the very specific question it wants answered, like asking for next arrivals at a particular bus stop.

Twitter, as an analogy, has both types of APIs. For most uses, you use a more transactional API. You ask Twitter for a list of recent tweets matching a hashtag, or ask it to post a specific tweet. But Twitter also offers a "firehose" API where certain users, who have to be approved ahead of time, can get the entire stream of all tweets, everywhere.

We need GTFS-realtime AND a transactional API

Ultimately, for transit, there needs to be both. If you're building a smartphone app, it's too hard to get the firehose of all bus positions, and easier to ask one simple question. But if you're designing a real-time screen, it's a burden to ask for each possible bus and bus stop every minute; you'd rather just get all the data at once.

WMATA's API also goes through another service, called Mashery, which limits how many of these API questions you can ask in a set period of time. The intent is to keep someone from overwhelming WMATA's systems and crashing them. But when Eric Fidler was building the real-time screen demos, he found that just asking for a few bus lines at nearby bus stops every minute, his system quickly hit the limit.

Plus, since one server was running many screens at once, the more screens, the quicker you hit the limit. We kept asking WMATA to increase the limit, and they did, but for many applications these limits will quickly become untenable.

Every transit agency ought to provide GTFS-realtime feeds for those that need them. ART's vendor, Connexionz, now also offers it, making 2 area agencies that do. Others should join Ride On and ART and offer this feed as well. Often it will be the agency's API contractor that offers it; agencies that pay NextBus for bus tracking services should require NextBus to offer a GTFS-realtime feed.

What's the common transactional API?

At the same time, we need a transactional API, ideally a common one. If everyone used the same API, it would be really easy for app developers to support all of the region's (or the nation's or world's) bus systems.

Unfortunately, there is no consensus here, unlike with GTFS. Most APIs are nonstandard ones an agency's IT staff or its contractor devised. New York uses the European standard SIRI, but had to make some changes of its own, and few US agencies use that. NextBus's is pretty widespread since that company serves a lot of agencies.

What to do? There are a few solutions.

First of all, everyone could get together and try to coalesce around an existing standard. It doesn't really matter which. It doesn't have to be the best one. Most standards are pretty imperfect; we type on QWERTY keyboards, which are one of the least efficient keyboard layouts you could devise, but any effort to come up with something else has failed. There's a strong lock-in, but to some extent, it doesn't really matter; we manage to type fine.

We could use SIRI; Europe does. Or NextBus could make their API a standard. Google did this with GTFS. Google initially created GTFS, but then they stopped controlling it and let the community of developers and agencies take control. They changed the "G" to stand for "General" instead of "Google." Many standards in computing started out as some company's property, but they transferred it to some national or international committee to shepherd.

If NextBus wanted to do this, they would probably want to give it a different trademark, so an agency offering the API wouldn't be saying they offer "NextBus" (we've had enough problems with NextBus trademark confusion already). And they would need to let other agencies and developers make changes, through some process, without the company having control.

Another approach would be to not worry about this at all. It's not all that hard to write some code to interact with multiple APIs, as long as they have a few features that you need to make them interoperable, like common identifiers. In the next part, we'll talk about this.

Some other company or entity could also set up an intermediary computer system that takes in all of the data on one end, and lets app developers connect to it. It would have to get the "firehose" style data from the agencies, and can then even offer 5, 10, or 50 different styles of APIs on the other.

What has to happen for that to be possible? For one, someone has to maintain it and pay for the bandwidth. An organization like COG, or a partnership of the DC, Maryland, and Virginia state DOTs, could do it. Or, to go national, a group like APTA or a federal agency could provide it. Or, perhaps some private entity would find it worthwhile, though the amount of revenue they could make is probably limited.

But for that to happen, the agencies have to offer the "firehose" of GTFS-realtime. For that reason, while there isn't consensus around all of the APIs, our region's transit agencies can and should take one step now, to offer GTFS-realtime, as Ride On and ART now do.

David Alpert is the Founder and Editor-in-Chief of Greater Greater Washington and Greater Greater Education. He worked as a Product Manager for Google for six years and has lived in the Boston, San Francisco, and New York metro areas in addition to Washington, DC. He loves the area which is, in many ways, greater than those others, and wants to see it become even greater. 

Comments

Add a comment »

Gosh, I wish we had something like a federal goverment that would just impose these standards -- like having a standard socket set -- on various agenices that get funded by federal money.

Silly rabbit.

by charlie on Jan 23, 2013 2:50 pm • linkreport

What's the cost to WMATA of adopting the API standard/standards that developers need?

There's a standing offer from the Riders' Advisory Council to help raise the profile of this issue and look for solutions. If the developer community wants to use this channel and there's a concrete proposal/ask we can promote, we're willing to assist.

by Ben on Jan 23, 2013 4:43 pm • linkreport

Sigh. Yet another article not yet related to the Fairfax County Connector buses. No real time, no algorithms, nothing on our bare bones system. What is the Connector doing with this sort of improvement?

by Transport. on Jan 23, 2013 5:54 pm • linkreport

The "intermediary computer system" that David describes is exactly what OneBusAway is. Feed in GTFS, and GTFS-realtime or SIRI feeds (where real-time data is available), and get out a merged GTFS-realtime feed, and the OneBusAway and SIRI 2.0 transactional APIs (along with much more, including desktop and mobile Web interfaces, IVR, SMS, etc.).

I have been working on bringing OneBusAway to the Baltimore/Washington area, but there are considerable challenges in doing so, mostly surrounding data quality and availability. It would be great to hand this work off to an institutional partner, but as far as I can tell they are not interested.

SIRI is in fact more mature and more widely-supported than the article suggests; version 2.0, which includes the changes made by the MTA and its partners for the Bus Time project in New York, has now been adopted and is now making its way through the CEN standardization process. People will continue dithering about standards for transactional APIs for transit data until someone makes the point that we already have SIRI 2.0, which provides a robust and mature API for both bulk and transactional real-time transit data exchange.

by Kurt Raschke on Jan 23, 2013 6:18 pm • linkreport

Ben,

The major issue for WMATA right now is data quality. Transforming data between various standards (and between proprietary formats like the current real-time API and existing standards) is not terribly difficult. The challenge is that once you perform that transformation, the output must be useful.

In this case, the identifiers for routes, trips, and stops in the real-time API do not correspond at all with those in the static data (that is, the GTFS feed), which is one of the greatest hurdles in getting WMATA's data into GTFS-realtime. There are, of course, many more problems, but that's a start.

It's difficult to say what the "cost" of adopting standards like GTFS-realtime and SIRI would be for WMATA, simply because in institutional IT projects the costs and time required tend to become bloated relative to what it would take an independent developer to do the same work.

by Kurt Raschke on Jan 23, 2013 6:30 pm • linkreport

Finally, on NextBus: The current NextBus API should be deprecated; for some applications it's a nightmare to work with. NextBus should instead offer a GTFS-realtime feed, along with SIRI for those applications that are better suited to a transactional API.

by Kurt Raschke on Jan 23, 2013 6:32 pm • linkreport

I'm going to just mention two improvements that could be made to Ride On Real Time's website:

1) It doesn't work on Firefox Mobile. No reason for this, it works with default Android browser just fine and is just hyperlinks as far as I can tell.

2) Every request is a post form submission, which makes it useless for bookmarking a route.

I get that apps are the wave of the future, but a simple no frills website can be very useful. Small improvements can make it more so.

by CorbinDallasMultipass on Jan 23, 2013 10:41 pm • linkreport

A few comments on Montgomery County's RideOn Real Time API.

The main issue I found is that it does not support something called JSONP. Basically, that means that if I enter http://rideonrealtime.net/stops/3504.json (plus my API Key) in the URL bar of my browser, I can get the requested stop data in JSON format. However, if I have a web application that makes an XMLHttpRequest, using that same RESTful API, it fails. Web applications use this technique, more commonly known as an AJAX requests, all the time. But the server at the other end of the request needs to support this, and rideonrealtime.net does not. It's not difficult to do and would allow for so-called mobile hybrid apps to be built, which are easy (er) to get to run on multiple kinds of mobile devices.

The other issue I found is in the data structure of the API itself. If you request the estimated arrival time for a stop, you get that information, plus something called the trip id. You can use the trip id to get additional information, which your app can use to display info to the user. But the trip API call does now say which bus is coming (the bus id). There is an api call that gives you bus positions, but you need to know the bus id. That's unfortunate, because if it seems that a bus is a long way away, it would be nice to show the user about where the bus was on a map.

Lastly, the ability for a developer to contact the folks running the RideOn Real Time system is a bit lacking.

by Steve Mallory on Jan 24, 2013 9:42 am • linkreport

It was mentioned that RideOn provides GTFS-RT. However, I don't see indications in Google Maps of their real time status in Google maps. Does Ride-On need to take further action to enable this? Or am I missing something?

by GP Steve on Jan 24, 2013 10:16 am • linkreport

@GP Steve
I see some buses that have real-time arrival info, check this link:
http://goo.gl/maps/QSakz

The little blinking dot next to a time means real-time information.

by MLD on Jan 24, 2013 10:26 am • linkreport

@MLD

Thanks, you are right. I do see one, but oddly ONLY one. While some of them are Metrobuses, a lot are RideOn that are as soon as 1 minute and yet not listed as RealTime.

by GP Steve on Jan 24, 2013 10:38 am • linkreport

I swear that RideOn's "realtime" tracking is just a countdown to predetermined time-table times. Because there have been many times that I have been waiting at a stop 5-10 minutes before the scheduled arrival time, and then watched as the time ticked down to 1 minute, and then "Due" and then back to 20 minutes, without a bus ever having come by. So while it's great that they were the first to offer their bus position data in an open standard, their data seems to be pretty bad.

by Daniel on Jan 24, 2013 11:59 am • linkreport

this is all Very educational, thank you. One question: How would John or Jane Doe find the right app for their smartphone to use this wonderful datqa?

by SRY on Jan 24, 2013 1:20 pm • linkreport

Daniel, for Ride On accuracy, that depends on the stop. If you are waiting at a stop near the beginning of the route, many times it will be just the scheduled time, which is really frustrating. If you're checking a stop a bit into the route, my experience is that it is usually fairly accurate. For example, my scheduled bus is 7:15, and it will say in the app that it is 7:12, and that 7:12 arrival is usually accurate.

However, sometimes the data can be terrible, like this morning. The app said 7:19, right up and after the bus went by at 7:11. Yes today may have been an "S" service day, the Ride On real time data should still be "real time," right?

My only solution to this is to keep filing 311 complaints with the county every time the bus is was off schedule and off the times given on the android app.

by AB on Jan 24, 2013 2:44 pm • linkreport

i would love a bus app that shows Scheduled time and actual time.
that way i can gauge performance and more importantly, i could scroll and see when the last bus or train is coming.

by pat b on Jan 25, 2013 3:34 am • linkreport

As the project manager for Ride On Real Time, I appreciate this article and the feedback. The prodding of David Alpert, Kurt Raschke and everyone that seends us feedback have had a hand in helping us move this project forward. As a public agency with limited funds and normal bureaucratic hurdles we move much slower than we would like, but we are making progress. Anyone with comments or questions about Ride On Real Time please send us an email to info@rideonrealtime.com and I will answer personally.

by Alfie Steele on Jan 25, 2013 8:14 am • linkreport

Add a Comment

Name: (will be displayed on the comments page)

Email: (must be your real address, but will be kept private)

URL: (optional, will be displayed)

Your comment:

By submitting a comment, you agree to abide by our comment policy.
Notify me of followup comments via email. (You can also subscribe without commenting.)
Save my name and email address on this computer so I don't have to enter it next time, and so I don't have to answer the anti-spam map challenge question in the future.

or

Support Us