Data Openness - Greater Greater Washington

Visualize Metro ridership

Wed, 05 Jun 2013 10:03:00 EDT

A recent Data Visualization Hack Day not only drew coders from around the region, but also some WMATA planning staff. They used a Javascript library to build a calendar of Metrorail ridership, showing each day's ridership since 2004 as a square of a different color.

Image by WMATA. Click to enlarge.

If you go to the large and interactive version on PlanItMetro, you can mouse over individual squares to see the date as a tooltip.

The darkest red days have the lowest ridership, the darkest green the highest. You can see high ridership events like President Obama's January ~~2005~~ 2009 inauguration, the Stewart/Colbert rally in October 2010, Snowmageddon/Snowpocalypse in February 2010, and more.

Stepping back, it's clear how ridership is highest in April, June, and July, and the number of very high ridership days jumped significantly in 2008 but then has stayed flat or a bit down since. Weekend ridership has gotten lower in recent years, probably because of all the trackwork.

What do you notice?

42 comments

OSSE releases more school data on students' neighborhoods

Thu, 30 May 2013 10:06:00 EDT

by Sandra Moscoso

Photo by Daquella manera on Flickr.

June 1 is National Day of Civic Hacking and a series of DC education projects are planned for DC's Hack for Change event. In anticipation of this event, the Office of the State Superintendent of Education (OSSE) released some very exciting data about school populations that can potentially help families negotiate school choice.

Which candidates did your neighbors donate to?

Wed, 17 Apr 2013 12:58:00 EDT

by David Alpert

The Sunlight Foundation has put together a great interactive map of contributions for the April 23 DC Council at-large special election.

Map by the Sunlight Foundation. Contribution data from the April 15 release
by the DC Office of Campaign Finance.

Their article by Ryan Sibley also shows many other interesting statistics, such as who got money from outside the region, the balance of corporate and individual contributions (Anita Bonds and Michael Brown got only about half individual contributions, while it's nearly 100% for Silverman), and more.

Sibley also notes that while DC's Office of Campaign Finance releases computer-readable data files with contribution information, some data is not in those files, like which candidate goes with a campaign committee. That's in PDFs, but PDF data isn't usable in mash-ups without human work.

What do you notice?

7 comments

A copy of DC's laws is now free and open

Wed, 10 Apr 2013 09:50:00 EDT

by David Alpert

After open data advocates pointed out how ridiculous it is that private companies have a copyright on the only publicly-available versions of DC's laws, DC Council General Counsel David Zvenyach helped make a public domain version and posted it online.

Photo from BoingBoing.

Tom MacWright explained the problem last month. DC, like many governments, contracts with a company (in this case LexisNexis) to compile all of the laws and keep them updated as they change. They post the laws online, but with licenses that restrict your rights to reuse the information, even though it's the public law.

Rather than ignoring the problem or issuing silly legal threats against people who were digitizing the code without permission, Zvenyach worked with the advocates to create a version of the code free of these restrictions.

Mike Masnick writes at TechDirt:

Part of the issue was that the only digital copy of the code that they had was the one given to them by West, and it contained a variety of extraneous information that was West's IP, including West logos on each section of the law (representing many thousands of copies). Zvenyach had Joshua Tauberer come by and spend a day removing every bit of West IP from the document and quickly releasing a downloadable copy of the DC Code with a CC0 public domain license.

Tom MacWright notes that this is just one step:

There are a few things that this isn't: it isn't the official copy of the code, and lawyers would be ill-advised to cite it alone. It isn't up-to-date—the council is fast-moving and this is just a snapshot. In time we'll fix these problems too.

What can people do with an open source set of DC laws? We can think of a lot of things, but the best part is when people do things we don't think of. Some commenters on MacWright's post wondered why this matters; can't you just find the code on the existing website? Yes, you can't link directly to a part of the code, and can only download pieces in Microsoft Word, but so what?

So what is all the ways someone could build better tools to make it easier to find the laws. Someone already made a tool that's for some purposes better than the official site. Or people could write automated programs to compare the laws on some topics, like yielding to pedestrians, to those in other states. (Hey, that would be a great idea! Has someone done that yet?)

Do you have ideas or want to implement some? MacWright is organizing a hackathon on Sunday. If you build something neat with the code, let us know and we'll show it off here.

6 comments

MPD not releasing marijuana data

Wed, 10 Apr 2013 08:29:00 EDT

by Nick Casey

MPD's data problems are continuing. The crime map came back online last week after a long hiatus but they can't pull data on 2012 or 2013 marijuana arrests. (DCist)

Comment

ART to release unencumbered schedule data (Aug 11, 2009)

Congratulations Sandra Moscoso, OpenGov champion!

Tue, 09 Apr 2013 13:54:00 EDT

by David Alpert

The Sunlight Foundation has named Greater Greater Education contributor Sandra Moscoso as an "OpenGov Champion" for her efforts to get more open education data in DC for parents and policymakers. Sandra also talks about her quest to ensure funding for school librarians and the effect librarians have on children, which she has blogged about here.

Transit, real estate mash-up helps you live near transit

Fri, 29 Mar 2013 13:49:00 EDT

by David Alpert

Say you're moving to the area, have a job, and want to find places with good transit to work. How do you figure it out? A lot of people just look at the Metro map and don't consider other modes, but a new service called AutNo is trying to help people locate near transit.

Image from AutNo.

This is actually a problem I hear often. A family friend moved to DC a couple of years ago, for a job at PriceWaterhouseCoopers in Tysons. The Silver Line was still a few years off, but he wanted to live in a vibrant, urban neighborhood. Where should he go?

The bus maps are daunting to decipher. It took me a couple of hours to really puzzle through the combinations and cross-reference it with my general knowledge of housing prices in various neighborhoods.

Boston-based AutNo tries to help by putting rental listings and trip planning together in one interface. You can view available rentals (it doesn't have places for sale, yet), click on one, and see transit directions to your office or another location you specify.

The about page reads:

AutNo is the first apartment search designed and developed specifically for people without cars. For the first time since the automobile was invented, the percentage of Americans who drive to school or work is on the decline. Gas prices are skyrocketing and automobile carbon emissions are contributing to global warming. Commuting and living without an automobile is the way of the future for many people. AutNo is dedicated to helping these people find apartments.

It will also show driving routes to work, too, if you want them.

You can narrow down results by price and number of bedrooms. A future feature that would be helpful is to also let people restrict the searches by travel time. That way, you could say that you want a place under $2,000 a month that's no more than a 45 minute trip to work, or whatever.

Basically, combine this with Mapnificent:

Places within a 1 hour transit ride of PWC in Tysons. Image from Mapnificent.

And, at the risk of sounding like a broken record: this is why open data is valuable. A transit agency might build a great app, but they're never going to build a mash-up of real estate data and transit data. When it's easy to put transit routing into an app, you not only can build apps that give people transit routing, but tools and apps that combine transit routing with almost anything else.

Update: I hadn't know it, but WalkScore actually has this exact Mapnificent-style feature. You can filter apartment listings by transit distance to a point:

Apartments within a 1 hour transit ride of PWC in Tysons. Image from WalkScore.

However, when you click on an apartment, WalkScore does not show you the transit routing with trains and buses you would take, while AutNo does. Without that information, people won't as easily learn which buses might work best for them or be able to judge whether a location is really likely as acessible from transit as the system says.

It would be best to have both at once on the same site; as it is now, I'd recommend that people use a combination of both tools for their search.

9 comments

And...

Tue, 26 Mar 2013 08:28:00 EDT

by Steven Yates

Some cool stuff people are doing with open data. (Atlantic Cities) ... As Arlington has grown enormously in 15 years, traffic hasn't gotten worse. (WAMU) ... The Washington housing market is not in another bubble. (Post)

Comment

DC's laws aren't yours

Tue, 19 Mar 2013 14:55:00 EDT

by Tom MacWright

There's a deep, persistent, and crippling problem with the laws of DC: you can't download a copy.

Photo by PublicResource.org on Flickr.

Due to a weak contract and a variety of legal techniques, it's not possible to create better ways to read the law or download it for offline access, or even to try to do better than the crummy online portal that serves as its official source.

It also means that it's hard to discuss legal matters online, since you can't link to specific laws—this Salon.com article about David Gregory has had a broken link to the law in question since 40 minutes after it was posted, months ago.

How the law became scarce

How did this happen? It's a tricky answer of access, ownership, and contracts.

The DC Council writes and publishes bills, which are additions and subtractions to the law itself. The law is compiled by a contractor—previously WestLaw, now LexisNexis. So the contractor holds a complete copy of the law.

The contractor publishes a few different versions of the "compiled law," each of which with restrictions:

The online portal has a "browsewrap" restriction against copying in full.
The CD they publish has a "clickwrap" restriction against copying at all.
Even the printed version has a registered copyright by the Council itself.

Unfortunately, courts have upheld these types of restrictions in the CD and website Terms of Service. They get further support from the wire fraud statute, which prosecutors used in the Aaron Swartz case to escalate charges to felonies. And in all of these versions, the contractor tries to claim copyright through compilation copyright and additional content like citations and prefaces.

In the face of these strong guards against freeing the law, the most reasonable avenue for creating a freely-accessible copy is buying and scanning the printed copies, which is exactly what some citizens are starting to do.

Why this matters

This has effects in many places. Advocacy organizations pushing for changes can't reference laws by linking to them, so they have to copy & paste relevant sections and hope that people trust their versions. Of course, when laws go out of date, these copy and pasted guides stop working.

The goal of better educating the police about laws (like the rules of the road for bicyclists) is harder. Police can't have an offline copy of the law for quick access in the field, and the online version is near-useless on smartphones.

It's also locking the DC Council into using a contractor for this purpose. DC's contracts with WestLaw and LexisNexis aren't strong enough to force the contractors to provide them with a copyright-cleaned version, so the council itself doesn't have a compiled copy of the law that they can publish by themselves if they want to take this in-house.

What's Next

This is a hard problem to unwrap and fix, and there are multiple efforts afoot.

Waldo Jaquith is building The State Decoded, an open-source system for storing and displaying state codes. It's already deployed with Virginia's laws. Public Resource.org is working on the long task of scanning and digitizing the print edition. And a group of residents are encouraging the council to write a better contract than the current one with LexisNexis, which doesn't provide for copyright-free copies.

Meanwhile, it'll be months or years until it's possible to download DC's laws onto your iPhone and clarify whether it is, indeed, legal to bike on a sidewalk (sometimes) or drink in public space (never).

33 comments

DCPS releases more helpful budget information

Thu, 14 Mar 2013 14:15:00 EDT

by Sandra Moscoso

On Friday DCPS released its initial budget allocations for the 2014 school year. This year's budget includes more information to help average parents and residents better understand the budget.

How school tiers match up with Walk Score

Wed, 13 Mar 2013 15:11:00 EDT

by David Alpert

One of the best effects of open data is when people correlate data sets from very different places to generate interesting information. This graph cleverly combines DC's school quality tiers (known as "accountability categories") with Walk Score:

Sandra Moscoso wrote yesterday about how Code for DC's School Decisions Project has been gathering coders who want to use open data to help parents, students, and policymakers. This is one of the graphs they created at the recent Open Data Day using data from the Office of State Superintendent of Eduaction (OSSE).

I've asked to get access to the raw spreadsheet for this graph so we can look at, for example, which schools each dot represents. Here are the accountability categories by school. I will add the spreadsheet with WalkScore matched up with category when it's available. Update: here's the data as a CSV file.

A few things immediately jump out. The most successful DCPS schools have high Walk Scores, while the least successful ones mostly (but not entirely) cluster in the lower range. This may reflect the fact that a public school's success has a lot to do with the socioeconomic status of the neighborhood, and the local retail that is a big part of Walk Score locates in areas with higher incomes.

That income effect is also very pronounced in the graph Sandra posted yesterday:

That's not the case with charter schools. 3 of the 5 "reward" charters are in low-Walk Score areas (which could mean something, or just be a consequence of little data), while the "Rising" charters are basically all over the place. This may have a lot to do with the simple fact that since charters have to find and pay for their own space, they're in all manner of locations.

An interesting future step might be to correlate the school tiers with some data set about land prices or rents, or resident incomes. That could help illuminate whether charters end up locating in less-expensive areas, because they want to serve poorer residents and/or because they need cheaper land.

What do you see from looking at this data?

10 comments

Is pre-K in DC working?

Wed, 13 Mar 2013 10:45:00 EDT

by Joe Weedon

Photo by jeannechristine on Flickr.

DC has made a major commitment to pre-kindergarten education. Are these programs improving kids' performance in the rest of their education? Based on information available so far, we don't know for sure. We do know that a pre-K program has to be high quality to make a difference, and some do better than others.

Community of civic hackers for education takes shape

Tue, 12 Mar 2013 11:50:00 EDT

by Sandra Moscoso and Harlan Harris

Photo from video by the World Bank.

With so many school options, applications, and lotteries, there is a dire need for information that will help parents make the best choices for their children. Code for DC's DC School Decisions project aims to use data and develop code that helps DC parents and students better navigate school lotteries and decisions.

Another great Capital Bikeshare visualization

Mon, 11 Mar 2013 12:05:00 EDT

by David Alpert

Starting at 12:06, Greater Greater Washington contributor Veronica Davis, WABA head Shane Farthing, and Arlington bike planner Chris Eatough will talk about bicycling in DC on the Kojo Nnamdi Show. Listen live or catch the archived audio once it's posted this afternoon.

They also posted this video which visualizes a few days of Capital Bikeshare trips:

This is yet another consequence of Capital Bikeshare's excellent decision to provide anonymous trip data. People have done all kinds of useful things with the data, like MV Jantzen's similar video and interactive visualization tool.

6 comments

Visualize the DC budget

Fri, 08 Mar 2013 10:13:00 EDT

by David Alpert

At the recent International Open Data Hackathon, Justin Grimes put the DC budget into a "treemap," a chart that shows a lot of items as rectangles of different sizes. This makes it very easy to understand how much money is going to different functions.

View larger chart.

Since Justin's spreadsheet was public, I was able to make a copy to tweak a few things. I modified some of the titles to get the agency's abbreviation to the start, so that you can understand more of them in the top-level chart, and revised the color scale to one that should be more perceptible to color-blind readers.

The colors represent which categories increased or decreased in FY2013, the budget approved last year for the fiscal year we're in now. Green boxes increased more, while purple boxes decreased. Though sometimes categories in the DC budget grow and shrink because functions get shifted from one to another, so it can be tricky to really understand increase and decrease numbers without delving into the budget deeply.

What do you notice in the budget?

And if you make a better treemap using a tool without some of the limitations of the Google one, or make a treemap for another area jurisdiction's budget, let us know at info@ggwash.org.

Thanks to Sandra Moscoso for the tip.

23 comments

Open data can help families with confusing school choices

Mon, 04 Mar 2013 12:14:00 EDT

by Sandra Moscoso

Photo by Lori Greig on Flickr.

Educating children in DC's publicly-funded schools can feel like a roller coaster ride for parents who have to sort through myriad educational options. DC can ease this task by making more information freely available and encouraging people to develop more useful tools that help parents make sense of it all.

3 neat apps

Fri, 22 Feb 2013 09:05:00 EDT

by David Alpert

Walkanomics tries to better estimate a street's walkability, but needs more and better open data. ... SpotAgent uses Baltimore's open data to estimate your chance of getting a ticket if you park illegally ... StreetMix lets you draw your own cross-section of a street, like the ones in transportation plans. (Atlantic Cities)

Comment

New site lists open data apps, agencies (Dec 10, 2009)
Cheh would limit regulation for Uber and taxi apps (Nov 5, 2012)
WalkScore now includes Transit Score (Aug 16, 2010)
On the calendar: Happy hour tonight on H Street, plus Alexandria tour, open data, BRAC, Connecticut Ave (Jul 14, 2010)
ART to release unencumbered schedule data (Aug 11, 2009)