Bus on-time performance: It matters what you measure

Transit

Bus on-time performance: It matters what you measure

by Michael Perkins • June 17, 2009 12:58 pm

Every month, Metro's customer service committee looks at a presentation on operating statistics, including a chart showing the latest bus "on-time performance" percentage. That percentage reflects the number of buses that arrive within a certain time before or after the published schedule—usually around 73-75%. However, on its own, this number doesn't give management enough information to effectively maintain bus quality. In addition, it doesn't capture important elements of bus performance, like bus bunching.

Photo by aussiegall.

If the on-time percentage improves or degrades, without looking any further, could Metro say why? If the percentage degrades drastically, could the Board do anything other than ask management to do a better job? Management needs to be able to identify trends, detect problems with individual routes or trips, and focus their attention on the areas that might need more resources or oversight.

The single on-time performance percentage also does not identify problems with bus bunching, especially on frequent routes. Imagine a bus line that is supposed to have service every ten minutes, but experiences bunching. Take five buses in a row starting at 8am (see top line in the figure), bunch the middle three together and spread the other two out (see bottom line in the figure). This is a worst case, but under Metro's on-time percentage rules, all of these buses are considered "on-time," because each bus is no more than two minutes ahead of or seven minutes behind schedule.

The green bar shows the range of times the 8:20 bus could arrive and still be considered on-time. A passenger arriving just on time for the 8 am bus will have just missed it, as it left two minutes early, and will have to wait until 8:17 for the next bus. That's a wait of just over seventeen minutes for a bus that's supposed to come every ten!

London looks at bus on-time measuring differently. Because the bus lines in London are operated by private contractors, it's very important for the local transit authorities to accurately measure on-time performance because there are real financial incentives or penalties involved. They measure how often buses pass by certain points on the network and track the "excess waiting time." All that time you have to wait for a bus that's running late or is bunched with others is added up and averaged over the route, and the excess waiting is compared to how much you'd normally have to wait assuming you come to the bus stop randomly. The reports for bus performance are published on the web.

In our example above, the average scheduled wait time is five minutes, and there are two buses you'd have to wait on average eight minutes, so the excess waiting is six minutes total, about 1.5 minutes per bus, or about 30% extra. (Under the London system, buses don't get credit for making you wait less than average.) This makes it easy to see when high-frequency buses are not meeting the required headways, and London applies this calculation to all buses that are supposed to come every 12 minutes or better. They even post the information on the web quarterly.

Metro should adopt London's system to measure on-time performance. The current measurement does not work for frequent buses. The London model focuses on the rider experience, compares various bus routes' performance, and gives a sense of the magnitude of the problem. In addition to this change for high-frequency bus routes, Metro should start regularly reporting on-time performance figures for all bus routes, as part of the monthly ridership report. They should also highlight the worst performing lines for each jurisdiction. If the problem is somehow Metro's fault, the route can receive the appropriate management attention. More likely, traffic congestion or other factors are at fault. In that case, data in hand, Metro Board members could make their case with state and local transportation officials to make transit operation a higher priority on corridors that are experiencing poor performance.

By identifying and improving poor performing bus lines, we can get people moving to their destinations more quickly, and reduce operating costs. Faster travel speeds and more regular schedules would drive up ridership, improving Metro's bottom line and allowing more service with the same local subsidy. Metro should revise the way they present performance metrics to make it possible.

18 comments

Tags: buses, London, on-time performance, WMATA

How do you evaluate Metrobus performance? (Dec 10, 2008)
Optimizing trunk bus lines could improve service (Feb 17, 2010)
Could a "12 Minute" bus map increase bus ridership? (Apr 22, 2009)
WMATA creates 15-minute bus map (Nov 21, 2010)
London's spider maps (Sep 25, 2009)

Michael Perkins blogs here and at Infosnack about Metro operations and fares, performance parking, and any other government and economics information he finds on the Web. He lives with his wife and two children in Arlington, Virginia.

Comments
Add a comment »

Metro allows buses to be 2 minutes early! As a general rule, that's insane! Buses should almost never leave before a published schedule says so. Every bus has a GPS receiver that displays absolutely accurate time--there is no excuse for leaving early. Early buses are the bane of my route (A4/A5).

Since all buses have GPS, and can also transmit their location back for NextBus, WMATA ought to be able to automatically generate performance charts for each run of each bus.

by thm on Jun 17, 2009 1:29 pm • link • report

How would you suggest to fix bunching? If you suggest the lagging buses stop and wait, you'll have even more irate passengers that possible riders.

by Wayan on Jun 17, 2009 1:30 pm • link • report

I've been on buses that had to stop to get back on schedule. It was rush hour, we were just a block from a Metro where most people depart, and nobody seemed to care. It helped that the driver explained it clearly to the bus and told us exactly how long we were going to wait.

by Reid on Jun 17, 2009 1:38 pm • link • report

This measurement is all great for management but I think most riders would be happy enough with real time arrival information through an effective NextBus system.

I don't care whether the bus is on schedule if I know exactly when its going to show up.

by whiskeybacon on Jun 17, 2009 1:43 pm • link • report

Fixing bunching would be nice, but what about the buses that just disappear? I've waited 45 minutes for the X2 in Chinatown... it's supposed to come every 7.

by retired gamer on Jun 17, 2009 1:51 pm • link • report

Is the raw data available, so someone else could run different calculations with the same data? This post just talks WMATA's "on-time" metric. Could we run numbers on, e.g., how many buses are more than 2 minutes early? How many are less than 3 minutes late (big difference between 3 minutes late and 7 minutes late)? Chart the performance of various lines (what % arrive x minutes early or late)?

by Gavin Baker on Jun 17, 2009 2:14 pm • link • report

@Gavin: The most I've been able to get from them is the on-time percentages by line, and that took two months and a FOIA request. (It was the subject of this article) You're welcome to try. Ask boardofdirectors@wmata.com, I recommend asking for a dump of all bus arrival times for the past week. The problem will be comparing with the schedule.

by Michael Perkins on Jun 17, 2009 2:20 pm • link • report

I think that WMATA is now concentrating on getting Next Bus to work properly. The implementation date is the same date as a number of schedule changes which is challenging. Every bus stop has to be programed and bus operators need to remember to log on every time they board a bus. The equipment has to be tested and work properly. I expect there will be some initial problems (as there are with any new schedule implementation although WMATA reports the testing going well.

Currently, putting buses back on schedule mainly involves re-directing them once they reach the end of the line.

If there was the ability (currently a lack of sufficient controllers to do this task) to identify and communicate with operators (WMATA radio communications have been notoriously poor, although they are in the process of upgrading them) on a bus line, it would be possible to give permission for a given bus to "help" a late bus and run slightly ahead of schedule for a period of time or re-route buses around a fire or other major events in a timely way (instead of waiting for a street supervisor to arrive at the scene and establish a detour).

Unfortunately, more personnel and improved equipment would be necessary to perform these "daily" problems.

The larger problem is that increasing traffic continues to put pressure on bus schedules. As time is added to schedules (and service is thereby slower), a bus ultimately needs to be added to a line (at an increased cost) just to maintain existing levels of service (current headway). This means worse service (slower speed, abeit more reliable) at an increased cost. Ultimately, to solve the problem of bus schedules, bus priority and BRT measures will need to be implemented to improve reliability, increase speed and control operating costs.

by kreeggo on Jun 17, 2009 3:29 pm • link • report

I think the 73-75% number is fairly meaningless unless we also weight the buses by number of passengers. Running all the off-peak buses with three passengers on time does not make up for a full articulated bus that ran late during peak.

Instead of % of buses that travel on time it should be % of passengers delivered to their destinations on time. That's a meaningful number, since the whole point of running buses isn't to drive them on streets but to transport people to places. So let's measure that.

by Steve O on Jun 17, 2009 3:47 pm • link • report

@SteveO: Even if we re-weighted the bus numbers as you suggest and start reporting the newly adjusted number (let's say for sake of argument it would be 60% instead of 75%). What would Metro do with the new number? It's just as useless in my opinion.

by Michael Perkins on Jun 17, 2009 4:07 pm • link • report

The biggest problem is known bottlenecks and traffic congestion at rush hour. WMATA could split some bus lines into smaller segments to cut down on bunching and improve performance. If lines were split into two segments, for example, people waiting for a 70s bus going north on Georgia Avenue from U Street wouldn't be affected by the inevitable delays on 7th Street in Chinatown. And riders of the 90s buses headed east won't be held up by delays through Adams Morgan and U Street. At the very least, buses could take alternate routes that are known to be less congested during rush hour, etc. It doesn't work for every line, but splitting/diverting buses could certainly help navigate around areas that experience very frequent congestion.

by Adam L on Jun 17, 2009 4:22 pm • link • report

@Michael: As to getting more out of this data: The Post has professional data-monkeys. Surely this would be a good story, since it's a service that affects a lot of readers and is supported with public money. Maybe you (since you wrote this) could punt this their way and encourage them to dig into it.

by Gavin Baker on Jun 17, 2009 4:38 pm • link • report

Here's a fun one: Let's say the bus comes at 8:00 or 8:10. Then the next one comes at 8:50, for some crazy reason. Does that count as the 8:20 one being 30 minutes late? Or do they fiddle the numbers even more?

As far as bunching buses goes, it sounds silly, but it does strike me that the best way to "fix" the issue is to have buses lag sometimes, to space them out. It might inconvenience a select few, but it reduces worst-case scenario far, far more than it raises the average time spent on the bus. I got very, very tired of waiting 30 minutes for a bus that was supposed to come every 10 when I lived downtown. I should have had to wait, on average, five minutes. Perhaps I'll show up and have to sit on a bus that isn't going anywhere for five minutes, but I'd much rather do that than stand 30 minutes waiting in the rain...

by Justin on Jun 17, 2009 4:58 pm • link • report

As a budding cartographer...

Here's a presentation that would be useful to periodically present, in the form of a computer map:

"Average Number of Minutes Someone Showing up on-time has to wait" at a given stop, for each of the 24 hours of the day and each of the 24,000ish stops of the system. The data format is a tiny 24-hour clock, with radial spokes sticking outwards for each hour whose length and color is determined by the the number of minutes.

This would have to be paired with some system for seeing what lines run to a given stop & highlighting all the stops on a given line. Day-of-the-week filtering would be nice as well.

That's the minimum level of detail that would be necessary to form the types of optimizations you're looking for. Any idea where one could get the data?

by Squalish on Jun 17, 2009 5:03 pm • link • report

Just getting through this entry now - someone did an FOIA? That's great! Thank you! Kreego, second your posting. Bus drivers have got to be able to push lights. It's one of the must haves to improve the system. I also think that while the drivers are good, they could definitely be better.

by Jazzy on Jun 17, 2009 6:11 pm • link • report

@Jazzy, up until recently, FOIA requests and board presentations were the only way I could get information out of WMATA. Since then, I've been put in touch with a staff contact who has been very helpful in finding me useful information.

I've done a number of FOIA (actually PARP) requests, some of which have turned up interesting information like which bus lines are more reliable, and some of which have not (my request for the incident reports for the double-derailment earlier this year did not turn up anything interesting).

by Michael Perkins on Jun 17, 2009 11:21 pm • link • report

Nice piece. I do think, however, your argument against these metrics on the grounds of their lack of usefulness in directing a response or to improve performance is too general. That critique will apply to pretty much any high level metric. The changes to the London metrics you suggest have the same issue--the metrics might be better or more informative, but if average wait time is high, that number still doesn't suggest a solution.

The point of these high level metrics is to give some rough measure of system performance, so that managers can see how things are doing and whether they're getting better or worse. If you want to improve performance, you really need to dig down several levels below these metrics into specific details of bus routes, scheduling, SOPs, traffic/environment, etc.

by Doug T on Jun 18, 2009 7:53 am • link • report

Great post. See my response to it here:

http://www.humantransit.org/2009/06/mundane-things-that-really-matter-defining-on-time.html

by Jarrett at HumanTransit.org on Jun 20, 2009 8:37 am • link • report