Posts about Data Openness
Those aren't bridges—
The darker dots are entries, the gray ones exits. You can see the stations with the greatest traffic are the ones with the dark peak in the afternoon (stations generally downtown). What else do you notice?
Planners in Boston, and eventually, regular residents will be able to analyze patterns of where and when people take Uber. The ride-hailing service has announced that it will give Boston data files listing all of the trips people have taken, with the locations anonymized to only show the ZIP code where they start and stop.
Cities already collect this kind of information from taxis, and it's available for services like Capital Bikeshare. But Uber doesn't provide it. In September, I suggested that as cities legalize such services and essentially deregulate the taxi market, they demand this kind of transparency in return.
However, Uber fought the idea. In New York, company representatives fiercely opposed efforts by the city's Taxi and Limousine Commission to collect the information. In DC, they more privately lobbied councilmembers not to require Uber to disclose this information, and enough didn't want to pick a fight with Uber that they didn't make it an issue.
Uber has had a run of bad press lately, and as it has grown, has encountered more criticism from the public. Emily Badger writes in Wonkblog that by making this concession, Uber may be hoping to win over some suspicious city officials and also set the terms of what data it will and won't share.
The data could be very valuable to planners, who will be able to understand where people are and want to go at various times of the day and week. This could help cities think about where transit service should go, where there is demand for new housing and retail, what happens during special events, and much more.
On the other hand, Uber is keeping secret much of the data that cities might need for consumer protection. While it's possible to compute the regular fare based on distance and time, which are part of the data set, it says nothing about surge prices or other special pricing.
Uber's data will also not reveal how long people have to wait for Ubers or whether in certain areas or certain times of day people can't get a car at all. This is something cities will want to know if, sometime in the future, Uber drivers are avoiding certain low-income or minority areas, for instance. Even if Uber itself doesn't do that, another ride-hailing company might. If Uber's data becomes an industry standard, regulators won't know that about the other company, either.
Finally, in Boston Uber is only giving the data to officials, not the public, but Badger says it will be subject to open records requests. If so, we can hope that Uber would start simply releasing the data file more publicly to save the step of making the request.
Uber representatives say the company will eventually start offering the data to other cities. Given all the facts, videos, maps, and graphs people have been able to generate from Capital Bikeshare data, we can look forward to learning fascinating things about how people travel once Uber provides the same for DC.
There's a new tool to show real-time information about Metro, Circulator, Capital Bikeshare, car sharing and more: Ride DC, which launched yesterday afternoon.
The design focuses on businesses that might want to put up a large screen in a store, building lobby, or other location. You pick an address and a set of transportation services, and it creates a web page at a specific address where you could point a browser displaying on a large screen.
To set one up, you go to ridedc.ddot.dc.gov, register for an account, then set up a screen. It assumes you're a business, but you should be able to just make up a business name if you want to try it out.
Right now, it doesn't work on mobile devices or (some report) iPads, but the @DDOTDC Twitter account said that an app version is coming in January.
Screens build visually upon earlier efforts
These screens share a lot in common with the screen project Eric Fidler built as part of a fellowship with Arlington County in early 2012. (Disclosure: I helped project manage that effort.)
Designer Kerry Mitchell created the first designs, and you can see many of the same elements in this screen, like the concept of showing the full Capital Bikeshare docks in red and empty ones in gray, an element I don't remember seeing on any earlier screen.
Most previous efforts were much busier visually, and the team working with Fidler and Mitchell, which also included Matt Caywood and Kevin Webb, wanted to cut away the clutter and make a design which focused on the key information about routes and times, so that you could see what you need to know at a glance even if the screen were farther away. This new screen, fortunately, maintains much of that simplicity while also adding a map.
Make the screens open source
It's not illegal or unethical or even rude for this DDOT screen to emulate many of the designs: the project was open source, with the explicit goal of providing a foundation for others to build upon. Caywood built on it most directly by creating his own company, TransitScreen, to offer screens for locations around the nation; it's headquartered in DC's 1776 incubator and offers more customization options than the Ride DC screens (but no maps).
The best way for DDOT to also give back is to make its own code open source. Taxpayers paid for the work, after all, and there's no reason to need DDOT to keep a lot of control. DDOT is not going to have the resources to add every feature anyone could want.
Therefore, it should keep running and improving its own service to give many people a quick, turn-key way to create a screen, and at the same time, open up its code so that others could build upon it in other ways.
Even better would be if DDOT had built upon the open source code already in existence. It might have saved money, and could have made it easier for people in other cities to benefit from what DDOT has done. That's what Caywood did when he organized a hack day to get civic hackers building their own screens and adding to the code base that everyone could use.
The best way to create valuable, reusable public technology is to foster a robust open source project where multiple individuals and organizations contribute. Nobody has to pay to reinvent the wheel, and everyone's efforts benefit others.
How the screens can be better
After all, there are plenty of little ways to continue improving this screen, as there is with the first version of any piece of software. You can't scroll or zoom the map right now; that would be useful.
When the data updates, the lines "flip" like an old-style train arrival board, which is cool, but they flip a lot; it looks like about every five seconds. That's distracting. It may make more sense to only "flip" when a number actually changes, or at least flip everything less frequently (an exception is when there are more bus lines than there's room, in which case it should flip more).
The screen creator selects whether to include services within a two, four, or six-minute walk, but often someone will want to include services that are farther away, especially something like a Metro station. It would make sense to offer larger distances, or decouple the distance for something like buses with Metro; you very well may want to only show buses six minutes away but also include a Metro station ten minutes away.
You can't show two Metro stations at once; instead, if there are two, it toggles. This is the same as what it does for buses, scrolling between all of the different lines. However, you could certainly imagine the screen owner wanting to show multiple Metro stations at once, especially if they aren't showing as many bus services at the same time.
This broad question, about how to pick among data, is one of the most complex questions we dealt with in the 2012 project. This project chose one reasonable option: basically, for each service you select, it blocks off some space and then rotates between everything inside your zone. That's a simple way to do things that will work for most people. The 2012 project and TransitScreen instead offer more customization, where you can pick which and how many bus lines, Metro stations, etc. to include.
It'll be worth watching this DDOT project closely to see what else the team adds.
The latest neat mashup using Capital Bikeshare data tells you how often a station has a bike or a free dock available:
Hackers from Code for DC created this tool using publicly available data on CaBi availability.
As you might expect, it becomes hard to get a bike at stations in residential areas outside the core toward the end of rush hour, and hard to get a dock downtown at the same time. The pattern reverses itself in the evening.
It looks like some of the data might need cleanup in some way. For example, the station at Pennsylvania Avenue and Branch Avenue SE, in Ward 7, is almost always red (unavailable) but lime green if you select just "summer." I suspect that means the station was just offline for a while during this period.
At the edges of the system, like in Alexandria, there are always bikes and docks.
What do you notice?
Do you know how the proposed changes in school boundaries and feeder patterns will affect your family? Thanks to Code for DC and DC agencies' willingness to provide data, there's now an app for that.
After 6 months of analysis, discussion, and concern about proposed changes in the way students are assigned to DC schools, the Office of the Deputy Mayor for Education (DME) has released 3 possible scenarios. The DME's team has also released a lot of background data, creating the opportunity for an informed conversation between the government and the public.
But it can be hard for ordinary citizens to wade through all the data and make sense of it. To make that easier, one tech-savvy DC resident has come up with an app that shows how each individual proposal would play out for every DC family.
The app, called Our DC Schools, allows you to enter your address and see how the proposals would affect your education options. Chris Given, a member of the volunteer civic hacking group called Code for DC, created the app, which is being released today.
"I attended a public working group meeting at Dunbar High School," said Given, "And while I was impressed by the dedication of DME and DCPS staff, I was just bowled over by the scale of the challenge of getting meaningful feedback from everyone these policies affect. I wanted to create an on-ramp to engaging with a really complex issue."
The app also enables you to rate and comment on each proposal and provides links to relevant background information, resources, and additional data-driven tools created by Code for DC and others. Given was able to create the app because the DME's office has embraced the open data movement, publishing on its website the information it used to create the proposals.
"It feels like we're at a real tipping point for open education data here in DC." Given said. "This app might have been impossible to create just 12 months ago."
In addition to data provided by the DME, the app incorporates contributions from the Office of the State Superintendent of Education, DCPS, the Washington Post, and the 21st Century School Fund. You can access most of the data itself through the Open Data DC website, a project of Code for DC.
Code for DC hopes to use the app to solicit feedback not only from parents and teachers but also from DC residents in general, since all citizens have a stake in improving the District's schools. They're urging those who use the app to share it with others in their networks.
The organization will funnel all feedback collected through the app to the Student Assignment Advisory Committee and also make it public, with safeguards in place to protect privacy.
By publicly displaying the data, the bikeometer helps illustrate that a lot of people really do use bikes to get around.
Arlington bikeometer. The numbers aren't visible in the photo due to the camera scanning frequency. Photo by the author.
The bikeometer is on the Custis Trail in Rosslyn, near the Key Bridge. It's a busy crossroads for cycling traffic headed into DC from Virginia. Older bike counts have shown thousands of cyclists per day at the location.
As of about 11:30 am yesterday, after only a couple of hours running, the display already showed 768 cyclists.
The device is technically called an Eco-TOTEM. It reads an underground wire, which counts bikes rolling over the trail above and sends the data to a digital display.
Arlington's bikeometer is the first such device in the eastern US, although they're common on the west coast and in Europe.
Cross-posted at BeyondDC.
Though many citizens understand the basic political process, it is often difficult to figure out how specific political decisions are made. Open data can make it easier to participate in local government simply by providing information.
Open data gives citizens access to information that can be used to improve government services and provide greater transparency. The underlying confusion and skepticism many people feel about government has given way to a nationwide open data movement, particularly here in DC. The variety of data tools available in the DC area demonstrates what is possible through open data.
There are a number of key websites that help local residents learn about and get engaged in their communities. But there are a few ways that local governments could further improve access to data.
Track the performance of DC agencies
At track.dc.gov, anyone can go online to track the performance of various agencies and access information about budgets, spending, news, and performance indicators. The website covers a number of agencies including the Commission on Arts & Humanities, the Board of Ethics & Government Accountability, Child and Family Services Agency, and the Department of Behavioral Health.
The site can be used as a tool for any citizen who wants to find out how the government is using public money. It serves as an additional point of financial oversight and allows each resident to become a government watchdog. Easy access to this plethora of information helps citizens to be informed of the inner workings of government.
Many agencies try to measure their performance internally. With this site, everyday people will not only gain a better understanding of where tax money is allocated, but do their own performance review by looking at how it is being spent, and where the gaps are.
See DC by the numbers
Those interested in city operational data can visit data.dc.gov to see figures for crime incidents, purchase orders, building permits, and housing code enforcement. In order to increase transparency, the city has published 493 data sets from various agencies that use city finances.
This website has a massive amount of information and makes it easily accessible. Without online access, anyone looking for this information would have to search through government archives, but with this tool, it's right at your fingertips.
Leveraging this information in a useful way requires some research and a bit of creativity, but the possibilities are endless for involved citizens. It becomes a matter of education so the public knows this information is available to them. Many people have used this data to create interesting visualizations of trends around the city, such as this map of every building permit in DC.
Through open data tools like this one, people can search through records and recognize problem areas that are growing worse or areas that have improved. But the most important step is encouraging everyday citizens and empowering them to take control of the information available to them, and in doing so, improve their neighborhoods and the community at large.
See how the public interacts with DC agencies
Though DC has made great strides towards making information available, there is still room for improvement. Sites like data.dc.gov and track.dc.gov are both tools that use information supplied by the government. As a result, it's a very one-sided solution to the open data dilemma.
The other important piece to open data is allowing citizens to communicate feedback to government agencies and evaluate their performance. In this vein, sites like grade.dc.gov collect feedback about particular agencies from social media sites like Twitter and Facebook.
The information is then analyzed and used to assign the agency a monthly letter grade based on how good or bad the reviews were. The results are two-fold: citizens have an avenue to report frustrating behavior, and agencies are made aware of problems.
Pushing open data even further
While the open data movement has made progress in DC, there is much more to be done. The government needs to make information more accessible and easier to sort through.
There also needs to be a greater emphasis on collaboration. The agencies must engage the people they are working for in order to figure out what changes need to be made. After that, it's up to citizens to educate themselves and use open data as a tool to hold the government more accountable.
WMATA planners have created a new ridership data visualization, a video that shows the volume at each station across the day:
This has a lot in common with Kenton Ngo's animated GIF that works basically the same way, but with less fine-grained time resolution:
WMATA planners created this before they saw Ngo's, planner Michael Eichler noted in an email. In each one, the circles are larger at times when more people are entering or exiting the station. The color shifts based on whether the traffic is people entering (pink), exiting (blue), or a mix (shades of purple in between).
The WMATA animation uses April 10, 2013, which was Metro's 4th highest ridership day ever. The PlanItMetro post says:
A combination of cherry blossom peak bloom and two sporting events ratcheted ridership up to 871,000 for the day, compared to an average weekday ridership of around 750,000. Note the high level of activity at the Smithsonian station all day long, and big dots that grow and shrink as the sports games begin and then end near Gallery Place and Navy Yard-Ballpark stations.You can access the data yourself to create your own visualizations here. If you make some, let us know at firstname.lastname@example.org and we'll post some of the best.
Kenton Ngo made an animation showing how many people are entering or exiting Metro stations at each hour across the day.
Green circles show where people enter, and red where they exit. As you'd expect, green circles swell and then shrink at end-of-line and other busy suburban stations in the morning, while even larger red circles appear at the stations at major job centers. In the evening, the pattern reverses.
This is another way of visualizing the Metro station data which WMATA released last year. Matt Johnson used it to compute the busiest stations and the balance between stations. In 2009, Matt diagrammed the flows in each direction.
- Metro proposes ending late-night service PERMANENTLY. That's a terrible idea.
- For Metro's plans to cut late-night service, big questions remain unanswered
- Find out your personal Metro on-time stats with this tool
- What do you think of these bike plans for Columbia Pike?
- DC's 43,766 acres: 25% "roads," 2% high-rises
- This may be DC's most ridiculous missing crosswalk
- Think you know Metro? It's whichWMATA week 88