Greater Greater Washington

Transit


NextBus accuracy slips, "ghost buses" explained

WMATA released NextBus accuracy statistics last week, and frequent users will not be surprised to find out that accuracy is not where it could be.


Photo from {Guerrilla Futures | Jason Tester}.

NextBus only has predictions for 78% of buses, far below the 92% accuracy target. The buses themselves are also not keeping to schedules particularly well, being "on time" only about 75% of the time even with a generous on time standard of up to 2 minutes early (which buses exceed almost 7% of the time) and 7 minutes late (which they exceed 18% of the time).

Bus manager Jack Requa provided some explanation for the "ghost buses" that riders widely reported: if a bus stands still for 2 minutes, the NextBus system stops showing that bus at all. It also removes the bus if it deviates more than 160 meters from the typical route, meaning that any bus reroutes generally result in disappearing buses.

The Examiner's Kytja Weir noted that this accuracy is lower than the 80% present when Metro "paused" the program for two years. That doesn't mean Metro should take NextBus down again (and they don't plan to), but does suggest some folly in taking it down in the first place, or keeping it under wraps until it was extremely accurate.

Still, when NextBus does work, it is very useful. When predictions do exist, they are usually correct (but not always), and make it possible to leave home or work just in time to catch a bus. Weir quotes Arlington Board member Chris Zimmerman saying, "NextBus has been one of the best things ... It's greatly improved the quality of my life. ... That said, it has to work with a fair amount of accuracy."

Overall, all riders are better off having an imperfect NextBus than no NextBus at all. However, the accuracy needs to improve. It would help for WMATA to detail its specific plans for what can be improved and at what cost.


Chart from WMATA.
The presentation misleads about customer satisfaction with its use of statistics. Requa's presentation shows a large pie chart of the total number of uses and a very tiny slice of the number of complaints. That makes it appear that every person not complaining is happy, which we know not to be necessarily the case. Requa said, "With 1.5 billion million uses [there were] only 148 complaints. The important thing is, there are very few complaints, and customers are happy."

But many customers don't complain because it's very difficult to complain. The online complaint form is very complex, and there aren't easy links to give feedback from the NextBus interface on the Web or on the phone. If Metro made it easier to complain, they would have more information about problems, but under the "customers are happy because there are few complaints" standard, it would make things seem worse. I hope that's not deterring the IT staff from making it easier to report problems.

It's terrific that Metro is tracking the accuracy for each bus garage and division, giving managers inside the bus system direct feedback about how well their groups are doing. The "dashboard" statistics we see do not reveal the causes of this inaccuracy, however. Are operators not signing on? Are the transponders breaking down? It would be helpful to see what percentage of the inaccuracies creep in at each stage. Zimmerman asked for this as well at the meeting. It looks like the dashboard listed on the Board presentation has the bottom cut off, so perhaps this information is on the full charts.

Accuracy seems to have slipped since January. The previous presentation showed log-on performance of 89.46% for the week ending January 23, 2010. Showing overall predictability instead of log-on performance is a better number, though it would be nice to show both. Bus performance also dropped, from almost 80% in that January week to 75% today.

Metro staff said that they are working on the problems, but gave few details beyond assurances that they were "doing everything that they can" and that "a lot of manpower and resources is needed." It'd be helpful to hear a roadmap for what Metro staff or the NextBus company itself is going to do to improve things. Ultimately, though, the proof will be in the numbers.

David Alpert is the Founder and Editor-in-Chief of Greater Greater Washington and Greater Greater Education. He worked as a Product Manager for Google for six years and has lived in the Boston, San Francisco, and New York metro areas in addition to Washington, DC. He loves the area which is, in many ways, greater than those others, and wants to see it become even greater. 

Comments

Add a comment »

Did Requa really say "billion" with a B? Because the number in the graph is million with an M. If every bus rider used NextBus, all 450,000 per day, it would take about 9 years to accumulate 1.5 billion rides. So I'm going to guess that that million with an M is the right number.

by thm on May 17, 2010 11:07 am • linkreport

Oops, no, that was a transcription error on my part. I actually caught that and was going to fix it and somehow it didn't get fixed.

by David Alpert on May 17, 2010 11:09 am • linkreport

Nextbus was useful for all of a few weeks.

I have gone back to using the printed schedule and trying to arrive 10-12 minutes before the bus is scheduled to arrive. And then I wait about 10-20 minutes after its stated time before I look for alternative arrangements.

Relying on nextbus has caused me to waste much more time -- whether waiting for buses that never appear or not waiting for a bus because its not listed -- only for it to show up.

Nextbus is not rocket science. The limitations put in place need to be reworked. Stopped for 120 seconds and then the bus disappears? That's outrageous. Whoever approved that needs to be fired.

by StuckinDC on May 17, 2010 11:17 am • linkreport

I used to use Next Bus but it was so unreliable I just stopped all together. It is just too frustrating when you rely on it only to be let down. I travel along the S2 S9 corridor between Adams Morgan and Silver Spring. I would check it to see if an S2 was near or if I should walk further to a stop that the S9 and S2 stopped at. It would tell me that there wasn't an S2 coming for 20 minutes, so I would start walking to the S9 stop only to have the S2 whiz by me a minute after I left the stop. And equally as annoying. Sometimes it would tell me an S2 was coming in 2 minutes. So I would wait instead of walking to the further S9 stop. After 10 minutes waiting for the S2 it wouldnt show and the S9 would fly by to the stop I could have been at had Nextbust told me an S2 WASNT coming. Now I just pretend the service doesn't exist.

by John on May 17, 2010 11:20 am • linkreport

Certainly, NextBus does improve bus riding in many cases, but it does look like it needs to be added to the burgeoning "over-promises and under-delivers" folder of bus enhancements. It should also be added to the "cautionary tales about unproven new transit technologies" folder.

Even with its flaws, though, the amount of new data that WMATA should have on its bus operations is staggering. Are they actually using it to analyze and improve the way things are done? I'd think they could identify trouble spots where delays and bunching occur and then adjust schedules as necessary. They also have verifiable data now to follow up on customer complaints of, say, a bus leaving 5 minutes before the schedule says it is supposed to (which is at times a chronic problem on the A4 and A5 buses that I ride most frequently). They should also be able to accumulate enough late/early/transponder off/deviates from route statistics to flag potential "bad apples" amongst the drivers, and ultimately, quantify the prevalence of bad-appleness amongst the drivers as a whole.

by thm on May 17, 2010 11:22 am • linkreport

That's about the correct ratio for me (78%) that NextBus has worked for me as well.
My biggest complaint about the interface, on a mobile phone "browser" is that you have to type in a full stop number (they are often not posted at terminal bus stops, such as the Pentagon or Silver Spring Metro), or scroll through every single route, alphabetically. It would be great if the data entry accepted typing in "C4" instead of "2000192" and then I could scroll through to find my stop. That should be a very easy change for the developers to make.

I can understand the detour (160meters) causing the bus to fall off the screen, but 2 minutes of idling - really? Have the NextBus people ever sat at a left turn light in Northern Virginia?!?

by Joe in SS on May 17, 2010 11:24 am • linkreport

I'm generally happy with Nextbus... but did anyone notice the gap in one graph that says "SNOW DAY!" Love it. At least somebody at WMATA tries to have fun with Metro's poor performance.

by Adam L on May 17, 2010 11:26 am • linkreport

@Joe in SS

Not sure what phone you have, but the Metro DC app on Android allows you to search by stop, bus name and/or number, and save favorites.

Re: Users v. complaints

I agree completely. If Nextbus was only 80% accurate during that selected two-month period, then Metro should expect to receive 200k complaints. I think WMATA's IT staff should work at making it easier to complain until they reach that goal ratio.

by Adam L on May 17, 2010 11:30 am • linkreport

On the topic of under reported complaints about NextBus, its worth noting that "NextBus" isn't even a "Topic" or "Incident Type" that you can select on WMATA's Contact Us form.

http://www.wmata.com/about_metro/contact_us/ridercomment.cfm

by bryandc on May 17, 2010 11:31 am • linkreport

and I'll add, of course I don't bother to call up and wait on hold forever to register a complaint each time it happens. I did this only once over the winter when I was waiting for a Bus. It was before one of the snow storms but at the time the roads were perfectly clear. I waited for a full HOUR at an S9/S2 stop. The whole time I called nextbus every 5 minutes. It always assured me that there were S9s and S2s on the way in a matter of Minutes. After an hour I called to register a complaint. As I was on hold a bus came. We cramed in like sardines and it had to pass all the people waiting at the next 20 stops because there was no room. I was on hold all the way from Silver Spring to Adams Morgan. When someone finally came on the phone. listened to my complaint. And told me that a bus would be there soon. She had no answer as to why one of the heaviest commuter bus routes in DC has no service for an hour at rush hour. Where were the buses?

by John on May 17, 2010 11:32 am • linkreport

By the way, I also think you want this link to the report:
http://www.wmata.com/about_metro/board_of_directors/board_docs/051310_NextBus.pdf

(The linked one is from earlier in the year.)

by thm on May 17, 2010 11:33 am • linkreport

I also use the Android app and it has been very helpful. Living on Columbia Pike with its frequent bus service, I might be a bit spoiled, but the app (which uses NextBus data) has generally been right on.

Also, I once called to complain about a bus being late, got a follow-up call and spoke with a manager, and I was assured the problem would be resolved. Overall, my only experience with Metro's customer service was positive.

by Teo on May 17, 2010 12:04 pm • linkreport

Even accounting for the 'gridlock disappearances' and 'detour disappearances', something is up with Nextbus. Probably 2 out of the last 5 times I've tried to catch a 30's bus on Wisconsin during the afternoon rush, I've seen multiple buses with arrival times of less than 10 minutes just disappear after 15 minutes of waiting, only to be replaced by 'new' arrival times of 50-60 minutes, so I walked down the street about 3/4 a mile and caught a circulator -- which I would have done in the first place if I knew the wait was going to be an hour for a bus in rush hour.

by Jay on May 17, 2010 12:12 pm • linkreport

This part made me laugh: "Still, when NextBus does work, it is very useful."

Well, yes. And airplanes, when they don't crash, are also very useful.

John Catoe should have been fired long ago.

by Fritz on May 17, 2010 12:18 pm • linkreport

First, thanks for this post. Improving bus service and getting more middle class riders on the bus is very important and often ignored.

Second, +1 on the comments re: the mobile web page by Joe. Far far too difficult to use with a modern phone browser. Should be much easier to just input your bus number rather than scroll through 100 choices. I don't use the app version, so maybe that fixes it. Seems as if it was designed for the SMS interface rather than mobile web. Needs to be both.

the idea that stopping for two minutes makes a ghost bus is insane.

I'm not sure using this as a metric for driver performance is a good idea. Plenty of things will slow down a bus that don't have to do with the driver. Gives us some baseline data but how many times is there a parked car blocking traffic, unruly passenger who won't pay, etc, that impacts performance?

What would also be interesting to see what are the most checked lines, and whether there in any increase in ridership on them.

by charlie on May 17, 2010 12:25 pm • linkreport

I have a few thoughts on this.

First, misleading/wrong information is very harmful. If the system can't be aware of the accuracy of its predictions, it shouldn't give any predictions at all. I think I'll disagree with the statement that it's useful even if it doesn't work all the time. It should be all or nothing. The story about the S2/S9 above is evidence of the harm it can do.

Secondly, I want to note that if you go to nextbus.com on your mobile device, you'll be brought to a screen that asks if you want to add it to your desktop. Do it. It loads a link on your home screen that takes you right to your closest bus stops. I've stopped using the NextBus DC app and the other webpage and have been using this new service on my iPhone.

Third, in response to "charlie", the issue with driver performance isn't actually the performance of their bus operation, but forgetting to log in, or logging in improperly to the onboard bus systems. If the driver doesn't log in, the "system" knows there's a bus out there moving around but doesn't know which route to link it to, therefore, not giving predictions for it. Therefore, a bus shows up even when nextbus says there isn't one coming.

Fourth, I find it surprising that the conclusion slide from the board presentation claims that NextBus was very useful during the snow storms, yet doesn't show its performance during that time, blocking out the data and replacing it with "Snow Day!!!". Its times like these when we need reliable arrival predictions more than ever, and if Metro is going to take credit for being there for us, they should also let us know how well they did the job.

Finally, I agree that the predictions have been really bad lately. Just yesterday waiting for the 42 at Columbia and Ontario, Nextbus predicted a bus in 2 minutes, then switched the prediction to Arriving, but there was no bus to be seen. Except one in the opposite direction that pulled away from the stop at the moment the "arriving" prediction left the screen. Hrm.

All in all, Metro could do a much better job of getting accurate predictions on all their bus services.

by michael on May 17, 2010 12:38 pm • linkreport

I don't agree that the current system is better than no system at all. It's bad enough that I don't bother to use it, because I can't trust it. Maybe some day it will be accurate enough to be useful, but how will I know that and start using it again? The answer is, I won't and won't.

We often say "the perfect is the enemy of the good" but Nextbus isn't currently anywhere near good enough for that to apply. It shouldn't have been released in this kind of shape.

by jcm on May 17, 2010 1:01 pm • linkreport

Only 142 complaints. That is so laughable. Please add me to the list to make it 143.
Have they checked the comment streams here and on unsuckdcmetro and the other blogs to add to the complaint totals? Perhaps they should.

Yes, it is too hard and a pain to make a complaint, so we can assume that those who did were really, really, really, pissed. I've just been somewhat pissed (bad prediction; missed bus by 20 seconds), so I haven't bothered to file a complaint.

I use NextBus as a curiosity tool. I use the schedule for planning, and then I check NextBus just to see how it does. I think the 78% is in the ballpark based on my experience, but I'm not willing to take a 22% chance.

by Steve O on May 17, 2010 1:32 pm • linkreport

The voice interface is also horrible. Takes 2+ minutes to make a request.

Ideally, they should have a service where you send a text message with your stop number, and it returns a list of the next buses arriving.

by andrew on May 17, 2010 1:35 pm • linkreport

@andrew

http://www.nextbus.com/wirelessConfig/sms.jsp

by Adam L on May 17, 2010 1:46 pm • linkreport

The fact that buses disappear if they don't move for 2 minutes explains a lot for me. My problem is the "ghost bus" which can vanish as it approaches or suddenly materialize where there was none before. I catch a bus soon after it leaves the Bethesda Metro. I know that buses sometimes stop there for a long time; maybe to correct a spacing issue, maybe to change drivers, whatever. It sounds like Nextbus loses buses that sit too long in one spot, so this "should" be an easy fix, right?

by Joe on May 17, 2010 1:46 pm • linkreport

Complaints to twitter don't count.

by Michael Perkins on May 17, 2010 1:48 pm • linkreport

Why does the bus operator need to login/out anyway.

Couldn't GPS and triangulation be used to figure out where the bus is or isn't.

Lets say the bus leaves the garage it sends a message to whatever saying that it is leaving the garage going to XX destination to start its route

Lets says its 32 when it arrives at Southern Ave couldn't GPS tell that it is at Southern Ave and then continue like it normally does until Friendship Hgts where the machines resets itself.

Why does a human need to be involved in that at all.

by kk on May 17, 2010 2:37 pm • linkreport

@kk

I stated in a related topic that Nextbus is tied into the radio system that Metro has and the operators have to log onto their radios via their badge number and assignment number. It's probably something with taking the data from the radio system into Nextbus that might be causing some of these ghost bus problems. Even if they were to use GPS triangulation, it still couldn't tell if a bus was doing a 32 36 31 or even heading downtown to start another route. That's all dependent on the AVL system. On a side note, the AVL system that Metro uses is the same as Los Angeles, Chicago, Baltimore, and Montgomery County.

by K Conaway on May 17, 2010 2:54 pm • linkreport

But what about during Snowpocalpyse when there were absolutely no buses running in my neighborhood for a week+ (the D2, D1 routes). Nextbus claimed buses were coming when in fact they literally were not running at all.

by Lauren on May 17, 2010 3:51 pm • linkreport

Nextbus is all right for what it is. It has helped me more than not, but I don't complain when it is inaccurate which it can be, very. I've seen an approaching time change from 2 minutes to 47 when it refreshes. I think the problems are more with the drivers, however. When I lived in England, if a bus was early, the driver waited at the next stop until their scheduled arrival time and then continued. Here in DC a G2 bus maybe scheduled to arrive at 7:47 but as I am walking down the street to the the bus stop it goes whizzing by at 7:30. To make matters worse, I'll sometimes see two or even three buses in a sort of convoy all following the same route at the same time, leapfrogging each over the other.

by Xopher on May 17, 2010 8:00 pm • linkreport

Perhaps nextbus may not be a better way for riders to plan their rides. The old paper method maybe better and ultimately cheaper. As a performance tool for management, nextbus maybe useful if they can effectively manage the data. That takes staff and a willingness to use the information. Thats always a big question mark. I doubt that nextbus has created any new bus riders.

by Interested on May 18, 2010 8:14 am • linkreport

@Interested

I dunno about everyone else's experience, but for me personally NextBus is a huge leap over looking at paper schedules. This is especially true if you take an infrequently-running bus to work. Before NextBus, you had to try to plan and hope that the bus wasn't 10 minutes early or 10 minutes late or not coming at all. With NextBus, you can see when the bus is coming, or if it's not on there you can make other plans.

I ride multiple buses every day, and I haven't experienced many of the problems that people are always complaining about with NextBus. I've never had a bus that was listed as being 6 or 7 minutes away just not show up (a friend experienced this right after the snowstorm due to a detour). I've never had a bus listed as a couple minutes away suddenly jump to 15 or 20+ minutes away. I have had buses that show up 5 minutes late but I'd attribute that to specific traffic problems the software can't anticipate.

Is NextBus perfect? No it isn't - and the WMATA data backs that up. But is it better than not having it at all? Absolutely.

by MLD on May 18, 2010 9:07 am • linkreport

I wonder if we are all complaining about NextBus, when the real problem is that the buses aren't running on time. Or all bunched up. Or not running at all.

Of course what's the purpose of NextBus if the bus has to follow the schedule to the second in order for NextBus to be right?

by b.o. on May 18, 2010 9:55 am • linkreport

78% accuracy is obviously not good, but I hope Metro doesn't take it down completely. I agree NextBus has problems (and it was completely useless during the blizzards and aftermath). But overall for me it has still been a huge improvement from relying on paper schedules, which are basically fiction. I may take 3 or 4 different bus lines a day, so I've bookmarked my most frequently used stops on my mobile device. But even if I have to look up a new stop, it takes maybe 15-30 seconds at most.

For the most part I've had buses show up pretty much as predicted - usually a few minutes later, rarely early or not at all. I've been taking Metrobus since I moved to DC in '93, so I'm already well-versed in changing transportation plans on the fly if there's no bus in sight.

by Serena on May 18, 2010 9:58 am • linkreport

I've found that the accuracy seems to depend on the line. I've used NextBus pretty frequently for the 90/92/93 lines and for the X2 line. I've had a few problems with the 90/92/93, but, other than during the snow storms, nothing major. However, I consistently have problems with the X2. I can either take the bus and have a shorter walk or take the metro and have a longer walk, so I generally take the bus when I know one is coming and take the metro otherwise. However, since NextBus is so inaccurate for me, there are often times when I decide to take the bus because I think it will be slightly faster and I wind up having to wait 15 minutes, only to have two or three X2's show up at the same time. It's so frustrating, but no more frustrating than paper schedules, which are also inaccurate for the X2.

by lilkimbo on May 18, 2010 10:54 am • linkreport

I still use Nextbus for the morning commute and it is ok. There are ghost buses but that's tolerable.
CAPTCHA- cobbing vehicle

by Mari on May 18, 2010 11:55 am • linkreport

The iPhone NextBus app (and its GPS locator of nearby bus stops) is the only way the MetroBus system is even possible to use. However unreliable the information is, without it I'd be spending a lot more on cabs and Zipcar...

by RS on May 18, 2010 4:04 pm • linkreport

I'll often look up on wmata.com when the bus will come then go to the bus stop 5 minutes before that. Then I call next bus and it tells me the bus will be coming 10 minutes after wmata.com told me it would come, and it comes at that time. I find a lot more inaccuracy with the wmata online scheudle than next bus.

by danielle on May 18, 2010 6:05 pm • linkreport

I find the NextBus telephone system pointless. It never has info for the bus I want, abnd constantly restarts if a car goes by. I used to bookmark the web page on my phone, but it was cumbersome. The Iphone app is much easier to use and gleans its info from NextBus. It does make my comute much less stressfull knowing I have missed yet another bus (or that I had better hurry)

The fact thatMetro makes is almost impossible to comment, much less complain - about their service is why they believe we are all happy little campers, gleefully riding the bus. The TV investigation of Metro Board members and their parking perks showed many of us riders that they were unfamiliar with Metro Rail... I would love to see them ride the X2 regularly and discover what real bus passengers endure.

by Greggndc on May 19, 2010 12:06 am • linkreport

NextBus has been getting spottier. I don't use it as frequently now as I did during the winter, but when it's raining and I want to use, half the time it doesn't list a bus at all. (X8, D6)

by lou on May 21, 2010 2:20 pm • linkreport

Today's experience just confirms that NextBus creates more harm than good. Trip planner, too.
I should just learn my lesson and check the schedules.

Trip planner told me 10:23 AM, then 10:53 AM for the next morning (today).

Missed one bus walking to the stop about 9:59. Checked Nextbus at the stop. It said 10:29 then 10:59. It's now 10:04.
Starting walking to local shops to kill time. Bus zooms by at 10:08 or so. Huh!??! F*^&! I could have been on that bus.

Decide to bag the shops and just wait. Bus comes at 10:22--nine minutes earlier than predicted just 20 minutes ago. If I'd gone in a shop, I would have missed this one, too.

So let's see. First NextBus fails to tell me of a bus coming in 5-8 minutes from now--a bus that Trip Planner never noted. Then it misses by almost 50% how long it will be before the following bus. Fail. All three buses were relatively close to the printed schedule, which I got once I was on the bus. Guess I'll stick to that.

So NextBus is worse than worthless. It's just like 20 years ago: go to the bus stop and wait for the next one.

(FYI, this was for the 1A/1B bus EB on Arlington Blvd.)

by Steve O on Jul 27, 2010 4:19 pm • linkreport

Add a Comment

Name: (will be displayed on the comments page)

Email: (must be your real address, but will be kept private)

URL: (optional, will be displayed)

Your comment:

By submitting a comment, you agree to abide by our comment policy.
Notify me of followup comments via email. (You can also subscribe without commenting.)
Save my name and email address on this computer so I don't have to enter it next time, and so I don't have to answer the anti-spam map challenge question in the future.

or