The Washington, DC region is great >> and it can be greater.

Posts about GTFS


Annapolis gets on board with open transit data

Many transit agencies around the Washington region and the nation have published their routes and schedules online in formats that apps and web tools, including Google Maps, can use. Now, Annapolis is joining the club.

Photo by Frida Archibold.

The General Transit Feed Specification (GTFS) is a format for computer-readable files that list all of a transit agency's routes and schedules. Google and Portland's TriMet created the format so Google Maps could include transit directions, but it's open and public, so anyone can write software that uses the data, and many other people have developed their own apps.

For example, the Car-Free A to Z tool, which lets people compare various options like driving, driving to transit, biking to transit, transit to Capital Bikeshare, and so forth, uses GTFS files for the transit portion of its data.

Image from Car-Free A to Z.

WMATA released GTFS files for its services in 2009, and became part of Google Maps in 2011, Since then, most regional transit agencies have followed suit.

A new feed for Annapolis Transit will let users of Google Maps and the other tools that use GTFS files plan trips on Annapolis' eight bus lines.

New tools help small transit agencies use GTFS

Small agencies, like Annapolis', don't have big IT departments. Some transit agencies even plan bus service with spreadsheets instead of dedicated transit planning software. They may not have the resources to make GTFS files and keep them up to date as schedules and routes change.

Luckily, tools are emerging to help small agencies. One such tool is GTFS Builder, a product of the FTA's National Rural Transit Assistance Program.

GTFS Builder is a simple set of Excel workbooks that compiles information like schedules, routes, and fares, and processes them into GTFS files. These workbooks allow transit agencies to digitize their data using Excel that agency staff are already familiar with.

Maryland gets a statewide program to make GTFS files

One of Maryland's Transportation Demand Management agencies, called TRIP, is taking open data one step further, running a program to help Maryland's transit agencies publish data to GTFS.

All data created through the TRIP program is open to software developers and available on the TRIP website.

So far, Annapolis Transit and the Regional Transportation Agency of Central Maryland have published data via TRIP. Others are sure to follow.

Having interactive transit information that includes schedules, fares, and bus stops available online is a prerequisite for many potential transit riders to learn how and when they can ride. GTFS files, and participation in tools like Google Maps that result, can encourage new people to try transit.

The more agencies join, the better.


WMATA might offer open data for all regional transit

WMATA planners helped STLTransit create an animation of transit across the entire Washington region. That's possible because WMATA has a single data file with all regional agencies' schedules. They hope to make that file public; that would fuel even more tools that aid the entire region.

Click full screen and HD to see the most detail.

One of the obstacles for people who want to build trip planners, analyze what areas are accessible by transit, design visualizations, or create mobile apps is that our region has a great many transit agencies, each with their own separate data files.

Want to build a tool that integrates Metrobus, Fairfax Connector, and Ride On? You have to chase down a number of separate files from different agencies in a number of different places, and not all agencies offer open data at all.

The effect is that many tool builders, especially those outside the region, don't bother to include all of our regional systems. For example, the fun tool Mapnificent, which shows you everywhere you can reach in a set time from one point by transit, only includes WMATA, DC Circulator, and ART services. That means it just won't know about some places you can reach in Fairfax, Alexandria, Montgomery, or Prince George's.

Sites like this can show data for many cities all across the world without the site's author having to do a bunch of custom work in every city, because many transit agencies release their schedules in an open file format called the General Transit Feed Specification (GTFS). Software developer Matt Caywood has been maintaining a list of which local agencies offer GTFS files as well as open real-time data.

We've made some progress. Fairfax Connector, for example, recently started offering its own GTFS feed. But while DASH has one, you have to email them for it, and there's none for Prince George's The Bus.

The best way to foster more neat tools and apps would be to have a single GTFS file that includes all systems. As it turns out, there is such a beast. WMATA already has all of the schedules for all regional systems for its own trip planner. It even creates a single GTFS file now.

Michael Eichler wrote on PlanItMetro that they give this file to the regional Transportation Planning Board for its modeling, and offered it to STLTransit, who have been making animations showing all transit in a region across a single day.

This is one of many useful ways people could use the file. How about letting others get it? Eichler writes, "We are working to make this file publicly available."

Based on the STLTransit video, WMATA's file apparently includes 5 agencies that Caywood's list says have no public GTFS files: PG's TheBus, PRTC OmniLink and OmniRide, Fairfax CUE, Frederick TransIT, and Loudoun County Transit. It also covers Laurel Connect-a-Ride, Reston LINK, Howard Transit, the UM Shuttle, and Annapolis Transit, which aren't even on that list and which most software developers might not even think to look for even if they did have available files.

Last I heard, the obstacles to the file being public included WMATA getting permission from the regional transit agencies, and some trepidation by folks inside the agency about whether they should take on the extra work to do this or would get criticized if the file has any errors.

Let's hope they can make this file public as soon as possible. Since it already exists, it should be a no-brainer. If any regional agencies or folks at WMATA don't understand why this is good for transit, a look at this video should bring it into clear focus.


What's up with NextBus, part 3: Where Ride On is the leader

Which Washington-area bus system was the first to offer its bus position data in an open standard? Would you believe it's Ride On?

In part 2, we talked about how there are many different APIsapplication programming interfaces, the way one computer system, like an app, gets data from to another, like bus positions from a transit agency. The fact that there are so many APIs means many apps don't include all of the types of buses in the region that have real-time positions and predictions.

Prince George's The Bus, Fairfax City CUE and the DC Circulator are available using NextBus, Inc.'s API, which is one of the most common because many agencies contract with NextBus, Inc. WMATA also contracts with NextBus, Inc. but doesn't use its API; WMATA built its own. ART has a different one entirely.

Since NextBus is most common, some residents asked Montgomery County officials why RideOn is not part of NextBus, too. One was Evan Glass, who tweeted last May:

Why is MoCo's Ride On bus system not accessible on the #NextBus app when all other jurisdiction are? #14bus cc @hansriemer (@EvanMGlass)

Note that Glass was talking about the "NextBus DC" app, the one that died this past December and, people discovered, actually wasn't from the same company as the one that provides bus prediction services to many transit agencies.

Councilmember Hans Riemer passed the question on to Ride On officials. Carolyn Biggins replied:

Recently, our staff met with a representative of NextBus to discuss products and costs. Although NextBus has not yet given Montgomery County a firm price quote, they offered a ballpark figure of approximately $55,000 per year for operating costs. This would cover a barebones system which would only have their mobile and desktop web site along with a suite of management tools. There are also undetermined setup fees, probably starting around $15,000 but possibly much higher. ...

At this point the inclusion of NextBus into the Ride On Real Time customer information product line is actively open for discussion. Feedback from our customers and industry critics point us in various directions and toward various apps; and, interestingly, NextBus is not at the top of our customer's request list.

Besides our Eastbanc/Nerds Ride On Real Time App (available for iPhone and android) which, by the way, includes integrated real time data from Metrobus, Metrorail and several Northern Virginia jurisdictions, our customers have asked us to integrate into the "DC Metro Transit App" and "OneBusAway."

We have been working with developers for DC Metro Transit App who recently responded to us with a very encouraging post about our open data: "This seems well thought out and documented. It is also nice that you can get the data in both JSON or XML [the 2 most popular formats for getting data from APIs] in a restful service [basically, a way of making APIs easier for the app developer to use]. I'll give it a try in the app and let you know if I have any questions. You guys are ahead of the curve compared to other agencies."

As you mentioned in your recent e-letter, Open Data and public/private initiatives, such as 3rd party app development, is the wave of the future—to "disrupt and create." 3rd party app development not only unleashes the initiative of the private sector but also provides varied choices for our citizens: the delivery of information in many different formats to suit different consumers with varied needs and tastes.

In developing our Ride On Real Time system, Transit Services has taken this approach, both through internal product development but also by providing its data in as many different formats as possible while trying to maintain fiscal responsibility. We will continue to work with NextBus and other vendors to try and provide Montgomery County citizens the very best in transit information and customer service.

(Notes in brackets added.)

Biggins is right. The solution to the problem of Ride On not being part of many existing apps is not to work with any particular vendor, but to provide open data in more formats.

It's particularly good to hear this from Ride On, because at first they did it wrong, and contracted with a software developer just to build them a website where people can track buses, but with no way for 3rd party app developers (in other words, people who aren't the agency or one of its contractors) to access the data.

Following prodding from Kurt Raschke, us, and others, Ride On started offering an API, and even fairly quickly improved it based on feedback from Raschke and other developers.

Why doesn't everyone just use GTFS?

In the area of transit schedules, one standard has largely emerged as the most common, and one all transit agencies ought to offer: the General Transit Feed Specification, or GTFS. GTFS is basically a set of big files that contain every single stop location and all of the schedules for the transit system. You can download it, write code to analyze it, and then do whatever you want.

There's an analogue of GTFS for real-time buses, called GTFS-realtime. However, real-time is not the same as schedules. With schedules, you can download the whole thing once and it basically won't change except every few months. With real-time bus tracking, the positions change every minute.

GTFS-realtime lets you download the entire set of bus positions as they constantly change. It's a huge amount of data. For some applications, like if you're making a live map showing buses, that's what you want. For the typical smartphone app, where you just want one bus position at a time, it's too much. That much data would overtax the user's data plan and burden the phone trying to deal with it all.

Other APIs, like the NextBus and WMATA APIs, work differently. For those, an app sends it only the very specific question it wants answered, like asking for next arrivals at a particular bus stop.

Twitter, as an analogy, has both types of APIs. For most uses, you use a more transactional API. You ask Twitter for a list of recent tweets matching a hashtag, or ask it to post a specific tweet. But Twitter also offers a "firehose" API where certain users, who have to be approved ahead of time, can get the entire stream of all tweets, everywhere.

We need GTFS-realtime AND a transactional API

Ultimately, for transit, there needs to be both. If you're building a smartphone app, it's too hard to get the firehose of all bus positions, and easier to ask one simple question. But if you're designing a real-time screen, it's a burden to ask for each possible bus and bus stop every minute; you'd rather just get all the data at once.

WMATA's API also goes through another service, called Mashery, which limits how many of these API questions you can ask in a set period of time. The intent is to keep someone from overwhelming WMATA's systems and crashing them. But when Eric Fidler was building the real-time screen demos, he found that just asking for a few bus lines at nearby bus stops every minute, his system quickly hit the limit.

Plus, since one server was running many screens at once, the more screens, the quicker you hit the limit. We kept asking WMATA to increase the limit, and they did, but for many applications these limits will quickly become untenable.

Every transit agency ought to provide GTFS-realtime feeds for those that need them. ART's vendor, Connexionz, now also offers it, making 2 area agencies that do. Others should join Ride On and ART and offer this feed as well. Often it will be the agency's API contractor that offers it; agencies that pay NextBus for bus tracking services should require NextBus to offer a GTFS-realtime feed.

What's the common transactional API?

At the same time, we need a transactional API, ideally a common one. If everyone used the same API, it would be really easy for app developers to support all of the region's (or the nation's or world's) bus systems.

Unfortunately, there is no consensus here, unlike with GTFS. Most APIs are nonstandard ones an agency's IT staff or its contractor devised. New York uses the European standard SIRI, but had to make some changes of its own, and few US agencies use that. NextBus's is pretty widespread since that company serves a lot of agencies.

What to do? There are a few solutions.

First of all, everyone could get together and try to coalesce around an existing standard. It doesn't really matter which. It doesn't have to be the best one. Most standards are pretty imperfect; we type on QWERTY keyboards, which are one of the least efficient keyboard layouts you could devise, but any effort to come up with something else has failed. There's a strong lock-in, but to some extent, it doesn't really matter; we manage to type fine.

We could use SIRI; Europe does. Or NextBus could make their API a standard. Google did this with GTFS. Google initially created GTFS, but then they stopped controlling it and let the community of developers and agencies take control. They changed the "G" to stand for "General" instead of "Google." Many standards in computing started out as some company's property, but they transferred it to some national or international committee to shepherd.

If NextBus wanted to do this, they would probably want to give it a different trademark, so an agency offering the API wouldn't be saying they offer "NextBus" (we've had enough problems with NextBus trademark confusion already). And they would need to let other agencies and developers make changes, through some process, without the company having control.

Another approach would be to not worry about this at all. It's not all that hard to write some code to interact with multiple APIs, as long as they have a few features that you need to make them interoperable, like common identifiers. In the next part, we'll talk about this.

Some other company or entity could also set up an intermediary computer system that takes in all of the data on one end, and lets app developers connect to it. It would have to get the "firehose" style data from the agencies, and can then even offer 5, 10, or 50 different styles of APIs on the other.

What has to happen for that to be possible? For one, someone has to maintain it and pay for the bandwidth. An organization like COG, or a partnership of the DC, Maryland, and Virginia state DOTs, could do it. Or, to go national, a group like APTA or a federal agency could provide it. Or, perhaps some private entity would find it worthwhile, though the amount of revenue they could make is probably limited.

But for that to happen, the agencies have to offer the "firehose" of GTFS-realtime. For that reason, while there isn't consensus around all of the APIs, our region's transit agencies can and should take one step now, to offer GTFS-realtime, as Ride On and ART now do.


Post-turkey video: That's a lot of dots

Using GTFS data, STLTransit has created videos showing all of the transit vehicles in a city over one day. Here's Washington's.

Via PlanItMetro.

The video shows one dot for each schduled Metrorail, Metrobus and Circulator vehicle. View the video in full screen (click the rectangular icon in the lower right of the video) to more clearly see the trains, which the video shows in a color corresponding to their line.


Apple dropping Google Transit is actually good for transit

To the disappointment of many, Apple's new iOS 6 won't include the Google Maps app, which means iPhone and iPad users won't have access to Google's transit directions. Future iOS users will need to install third-party apps for transit information.

Image by ebatty on Flickr.

The feature shake up with iOS 6 is part of a larger battle around Google Maps. Like many other Google Maps users, Apple is dropping support in favor of an in-house solution. Google's decision to start charging large users for maps, combined with an explosion of competitive alternatives, has enabled users of all sizes to chart their own course.

Apple's move is also part of a larger conflict between the two companies over Android and iPhone. Transit users are now counting themselves as collateral damage.

There's been a lot of hand wringing over this change. But in the long run, this is actually great news for the transit community. Here's why:

There's tremendous opportunity for innovation in how we design and communicate information about personal mobility. Unfortunately the tools have not kept pace, in part due to a lack of proper incentives for new services. With iOS 6 Apple is building a market for new tools rather than offering a default solution.

In DC, where I live, we're redefining urban transportation. On any given day I might combine trains, buses, a shared bike and even reservationless one-way car rentals. This array of new options fundamentally changes how I move through the city.

How I best use these options also changes. Traffic accidents and road closures happen, Metro trains fail and Capital Bikeshare inventory ebbs and flows throughout the day. Depending on the actual bus arrival time or the distance to the nearest available Car2Go my best route may change radically from day to day.

Integrated and accurate information tools are key to navigating the incredible and complex multimodal future we're building.

While Google Maps helps, it doesn't solve every problem. Want to find the best combined bike/transit journey (important for my daily commute)? No dice. New to the city and want to plan a safe trip using Capital Bikeshare? You'll need to shuttle info between two sites. Unless you're on an iPhone which doesn't offer Google bike routing at all. Want to use a car share? Go elsewhere.

Software developers have a real opportunity to contribute solutions for these new challenges. New user interface designs and improved computational techniques can make huge impacts. However, we've seen relatively little innovation.

One possible reason is that Google's free tools de-incentivize others from entering the market. iPhone and Android users have had little reason to download alternate apps, especially paid ones, when the pre-installed features solve much of the need. Unlike many other Google technologies, there's no current option to extend the functionality for transit or other directions, or incorporate this data into non-Google apps.

Google offers a free utility to any transit operator willing to share their data (and legally indemnify Google). However, it's very unlikely this service generates a profit for the company on its own. Instead, it's a loss leader, cross-subsidized by revenue from other services. Transit in turn contributes to the overall value of Google Maps as a comprehensive platform.

Unless Google makes a big change in how it derives revenue from its consumer applications, transit isn't going to be a major source of income. And without revenue and some healthy competition, there's less incentive for them to innovate.

It's pretty hard for a startup to compete with something that one of the world's largest tech companies decides to offer up as free service. Even transit operators, faced with shrinking budgets and limited staff, find themselves wondering if they should leave the responsibility of customer communication to Google—some do.

But Google's biggest contribution to transit is not the end-user functionality in their map application. It is the way they catalyzed an open standard for transit information. Before Google Transit, it was nearly impossible to get timetables and stop locations from agencies in electronic formats—a requirement if you're building an information service.

To solve the data problem, Google, in collaboration with TriMet, created a data standard known as GTFS (the General Transit Feed Specification) that allows agencies to share their schedules in a common format. Now, anyone building an app in one city (including Google) can make that app work in another location as long as the transit operator shares their schedule in the GTFS format.

GTFS, along with other open data platforms like OpenStreetMap, have made the development of universal, global transit services a real possibility. Similarly, open source technologies, like OpenTripPlanner (built by OpenPlans, also in partnership with TriMet), have served as the foundation for companies and transit operators around the world to start offering new and innovative information services.

TriMet championed the creation of OpenTripPlanner to provide combined bike and transit functionality to users (an especially important feature if you're in Portland, Oregon). At the time, no other solution could do that. And Google's still doesn't.

This spring, OpenPlans wanted to create a bike share trip planer ahead of New York's Citibike launch. It turned out that this feature had already been implemented by another OpenTripPlanner developer, Laurent Grégoire, who wanted similar support for Paris' Vélib bike share system.

Because OpenTripPlanner is open source and built on open data standards, we were able to work together, rather than reinventing the same features. A few weeks later, we launched for New York—a trip planning first—and are working to bring it other cities in the US. Now, bike share operators, software companies and individual developers around the globe can reuse this work to offer the same functionality in their communities.

Thanks to open data feeds like GTFS and OpenStreetMap, and open source tools like OpenTripPlanner, anyone can start a company that offers transit information services that are competitive with or even surpass Google's existing offering. The more that do, the more we'll see competition on features and opportunities for innovation with new modes of transport.

In the short term, there will be some pain as new tools are developed. But in the long run, iOS 6 might do as much for transit innovation as GTFS and Google Maps already have.

Cross-posted at OpenPlans blog.


WMATA directions now on Google Transit!

Metrorail and Metrobus data are now a part of Google Transit!

If you now go to Google Maps, you can get directions from one place to another not just by car and bike, but also by transit using WMATA services, as well as a few other area transit systems. Here's a sample trip from Arlington to H Street:

WMATA just put out a press release that they and Google "will make a joint announcement about technology improvements that will benefit Metrorail and Metrobus riders" tomorrow. They don't say what the announcement will be, but there happens to be one service WMATA and Google have been working on for a long time now, for which they signed a contract in July and said was "very close" in March... and which has suddenly become available for riders.

The service has also been available on Bing Maps for some time.

Since the MTA Maryland buses and light rail and MARC are all on Google already, you can now also use Google Maps to plan a trip from Baltimore to DC using multiple transit services. Here's a somewhat complex example, with 6 different options; one even uses the Amtrak Northeast Regional.

The DC Circulator and Montgomery County's Ride On are also on Google already.

We're just two days shy of 29 months since Michael Perkins launched our first email petition to ask WMATA to take this step for riders. Michael kept doggedly on the story, FOIAing other cities' agreements to rebut arguments about legal obstacles. There was a lot of resistance from within the agency and from some members of the Board, but other sympathetic staff and Board members pushed this forward, and now it's a reality.

Thanks, WMATA!

Disclosure: I used to work for Google, but had no involvement with Google Maps. I no longer own any Google stock and have no other financial interest in Google.


Google Transit still "very close"

WMATA CEO Richard Sarles said a launch of trip planning on Google Maps is "very close," but declined to give a specific timeline. This project has frustratingly remained "very close" for more than a year.

Photo by Cavan Moon on Flickr.

A December email from Victor Grimes in WMATA IT said they "anticipate going live by mid-January." That deadline has long passed. the agency was saying the exact same thing a year ago.

Integrating with Google Maps will provide big benefits to Metro. Many people already have Google Maps on iPhones and Android phones, and visitors to DC or infrequent riders use it to navigate. Putting bus stops on the maps and providing trip planning right from that interface will make riding transit easier and advertise its existence to many wouldn't otherwise know about options or find them too daunting.

I know that technological projects sometimes take longer than expected and problems can crop up. However, WMATA management has continuously remained very tight-lipped about their lack of progress, and did not respond to a request yesterday.

It certainly seems as though this is simply not getting much attention at all. If it is, and there are just unforeseen issues, or if Google is the one being difficult, it would certainly behoove WMATA to explain these facts.

The agreement was signed in July, and the data made high-quality enough to release publicly in the fall. Now, according to Sarles, they are working to prepare the data to upload to Google.

In response to a question from Councilmember Tommy Wells at the oversight hearing this morning, Sarles noted that Metro routes and schedules do appear on Microsoft's Bing. What's so much harder about getting on Google?

A comment from Chris Zimmerman last July continues to seem most prescient. He said this project seemed to be "asymptotically approaching" completion. So far, that's still as true now as it was then.

Disclosure: I used to work for Google, but had no involvement with Google Maps. I no longer own any Google stock and have no other financial interest in Google.


Google and WMATA signed Google Transit agreement in July

Google and WMATA signed an agreement on July 22, 2010, to provide the Google Transit service, according to documents obtained via public information request.

Photo by buckofive on Flickr.

WMATA had previously stated that Google Transit was expected to go live in mid-January 2011, more than two years after Greater Greater Washington started a petition campaign to encourage WMATA to allow Google to display transit routing and schedule information.

The agreement appears to be based on the typical Google boilerplate agreement. Google is not paying WMATA for the use of the data, which was one of WMATA's early sticking points.

The indemnification paragraph from the boilerplate agreement appears to be missing, which means that WMATA would not be held liable for any mistakes caused by Google and did not agree to legally defend Google if they were sued. This was one of WMATA's biggest objections to signing the boilerplate agreement. We first reported that Chicago was able to remove this indemnification from their agreement.

Either party may terminate the agreement, unlike the boilerplate agreement, which only gives that option to Google. The agreement provides rights to both WMATA and Google where the boilerplate agreement only provides them to Google.

The agreement WMATA got looks like the best they could hope for. It's balanced and removed the features WMATA found most objectionable. WMATA's status as one of the largest transit agencies in the country allowed them to negotiate from a better position than most agencies.

I haven't been able to get an update on the actual Google Transit release date.


Google Street View now shows bus stops

Google Maps recently rolled out a new feature that will help potential transit customers use bus services in cities that participate in Google Transit. You can now see the location of a bus stop in the Street View mode of Google Maps, as well as upcoming departures according to the published schedules.

This should reach the Washington area soon: WMATA says they are checking all of their data in advance of a January roll-out on Google Transit.

Image from Google Street View showing a bus stop in Portland, Oregon.

This feature is especially useful to people that are unfamiliar with the bus system, because it will let people see where their bus stop is while they are planning their trip.

Much of the time, the only information available is a line on a small map in a PDF file, and with Metro reducing the number of stops, it's more important to know where the stops are. For some routes like the DC Circulator, the distance between stops can be a half mile, so walking to the route and then walking along the route until you get to the stop is not ideal.

WMATA has a feature right now which helps in this regard. Under the "Rider Tools" section of the website, you can find the "Service Nearby" link, which allows you to find bus stops near an address or landmark.

This produces a text description the stops in the database, which works for some riders, but it doesn't give the same contextual clues that seeing the stop in a photo would. Some riders are good at navigating according to landmarks, so telling them their stop is at the corner of "NW P and NW 18TH ST" is less helpful than "in front of the gray and tan building next to the CVS."

The Street View page will also show the next transit trips scheduled at the stop, which can give customers a sense of how frequently the service runs. If you click on a bus line shown, it opens a page specifically for that bus line, pointing out the schedule at that time of day, what other services run from that stop and some nearby stops that might have other service you're interested in.

This is the kind of customer information that WMATA wouldn't be able to afford to develop by themselves. Google has taken the information made available to them by transit agencies in a standard, open format, and has improved its usability for customers. Hopefully, WMATA will be able to keep their scheduled January 2011 launch of the Google Transit service.

Support Us
DC Maryland Virginia Arlington Alexandria Montgomery Prince George's Fairfax Charles Prince William Loudoun Howard Anne Arundel Frederick Tysons Corner Baltimore Falls Church Fairfax City