nyc Archives

December 27, 2006

Origin and Destination Estimation In New York City with Automated Fare System Data

[Syndicated from CiteULike: fruminator's library]

Transportation Research Record, Vol. 1817 (2002), pp. 183-187.

New York City Transit's automated fare collection system, known as MetroCard, is an entry-only system that records the serial number of the MetroCard and the time and location (subway turnstile or bus number) of each use. A methodology that estimates station-to-station origin and destination (O-D) trip tables by using this MetroCard information is described. The key is to determine the sequence of trips made throughout a day on each MetroCard. This is accomplished by sorting the MetroCard information by serial number and time and then extracting, for each MetroCard, the sequence of the trips and the station used at the origin of each trip. A set of straightforward algorithms is applied to each set of MetroCard trips to infer a destination station for each origin station. The algorithms are based on two primary assumptions. First, a high percentage of riders return to the destination station of their previous trip to begin their next trip. Second, a high percentage of riders end their last trip of the day at the station where they began their first trip of the day. These assumptions were tested by using travel diary information collected by the New York Metropolitan Transportation Council. This diary information confirmed that both assumptions are correct for a high percentage (90%) of subway users. The output was further validated by comparing inferred destination totals to station exit counts by time of day and by estimating peak load point passenger volumes by using a trip assignment model. The major applications of this project are to describe travel patterns for service planning and to create O-D trip tables as input to a trip assignment model. The trip assignment model is used to determine passenger volumes on trains at peak load points and other locations by using a subway network coded with existing or modified service. These passenger volumes are used for service planning and scheduling and to quantify travel patterns. This methodology eliminates the need for periodic systemwide O-D surveys that are costly and time-consuming. The new method requires no surveying and eliminates sources of response bias, such as low response rates for certain demographic groups. The MetroCard market share is currently 80% and increasing. MetroCard data are available continuously 365 days a year, which allows O-D data estimation to be repeated for multiple days to improve accuracy or to account for seasonality.

January 18, 2007

Lessons From The Number 7 Train Extension

[Syndicated from]

Bruce Schaller in effect. Most important is the use of Tax Increment Financing to fund a new subway line/extension:
One key to the No. 7 extension’s fast progress involved the financing. Mayor Michael Bloomberg and Deputy Mayor Dan Doctoroff, champions of the project, recognized early on that the 7 extension would go nowhere if it had to compete for funding with the Second Avenue subway, connection of the Long Island Rail Road to Grand Central Terminal, airport rail access projects, a new tunnel from New Jersey, replacement of the Tappan Zee Bridge and other mega projects. Thus, they developed a novel funding method. The bonds issued last month, and another set of bonds to be issued in several years, will be paid off using payments made to City Hall in lieu of taxes by new residential, retail and office development on the West Side, payments for development rights in the eastern part of Hudson rail yards themselves, and from other taxes collected from the area. Having its own funding source was immensely important to bringing the 7 extension along.
This isn't really that new a concept for transportation funding, just in America! It is generally referred to (eg in Cervero) as Value Capture. And what, putting the Gowanus Expressway underground?

February 11, 2007

Feelin' the Flow (NYC Traffic Data, and more)

Who knew that there was such a wealth of statistical data published about traffic flows in NYC? You can't yet get the count for every corner, but if you're interested in people coming in and out of Manhattan, you're in luck.

The first place to look is the The New York Metropolitan Transportation Council (NYMTC) Data and Model page. Most detailed is probably the Hub Bround Report which goes into ridiculous detail about, unsurprisingly, people entering the Hub (Manhattan below 60th street). On the average business day in 2003, how many people entered Manhattan on the N train between 9 and 10AM? 10,031 of course. On the L train? 21,336 (which, if you recall from this chart, is about a third more than a 7 lane freeway).

The NYC Department of Transportation Publications Page is also a good source -- New York City Bridge Traffic Volumes, Bicyclist Fatalities and Serious Injuries, and more.

Gee, all this would be really useful if you were, for example, trying to build a rough model of the effects of a London-like Congestion Pricing scheme for the NYC core...

February 28, 2007

Congestion Pricing, Positioning, and Meshed Wireless Networks

As part of my internship at the Regional Plan Association I was asked to research the applicability of mesh networks to congestion pricing for New York City. What follows is the result of several days of reading, surfing the web, talking on the phone, and stroking my chin. It assumes some knowledge on the topic, most of which can be found in descriptions of London's Congestion Charge, upon which any scheme in New York is likely to be based.

Primary Questions

  • What about London's CC scheme do we not like?
  1. Pricing is not very flexible. No variability of charge over time or space (i.e. path)
  2. For the most part, only charged for crossing the boundary into the zone
  3. Post-payment (i.e. account-based billing) is impossible
  • What would we do the same in a first implementation?
  1. Charge people for driving within a certain area during a certain time period
  2. Use cameras to charge people who opt out of any other system, i.e. cheaters and tourists
  • What would we want to do differently, ?
  1. Charge people with accounts, like EZ Pass
  2. Charge people who do not cross the zone-charging boundary (i.e. remain entirely within the zone)
  • What would it take, from a technologic perspective, to do it as we prefer?
  1. Substantially higher accuracy of detection
  2. Detection within the zone, not just at its edges
  • What technologies and approaches are likely candidates to be considered?
  1. Automatic Number Plate Recognition (ANPR, cameras reading license plates, like london) for enforcement
  2. Dedicated Short Range Communications (DSRC, like EZ Pass) for point detection for account-holders
  3. GPS positioning for area-wide detection
  4. Wireless Positioning System (WPS -- TV/WiFi/GSM) for area-wide detection
  5. Wifi/Mesh Networks for communications
  • What sort of physical footprint or envelope, both in the vehicle and on the streets, would we expect for each different solution?

Continue reading "Congestion Pricing, Positioning, and Meshed Wireless Networks" »

June 28, 2007

Le Triboro RX

In its 1996 Third Regional Plan, the Regional Plan Association describes a rapid transit line in Brooklyn, Queens, and the Bronx that could be built almost entirely on pre-existing rail rights of way and would connect with at least twenty existing subway lines. The so-called ''Triboro RX'' (''TRX'' for short) presents a unique opportunity to provide mobility and accessibility to New Yorkers living or working within these three boroughs, at a fraction of the cost of most transit projects of similar size. In my part-time internship at the RPA, which ends today, the lion's share of the work I have done has focused on fleshing out the idea of this line.

Working with the singular Jeff Zupan and his former sidekick Alexis Perrotta, I helped to develop a possible alignment for the Triboro RX, and a crude estimate of what levels of initial commuter ridership one could expect to see if it were built. The fruits of this labor can be seen on the web at (including sections on the alignment, our data sources, the demand model, and detailed results). There I describe in detail how the line and its stations are laid out and how we made our estimates. At the end of the day, we can comfortably say that at least 76,000 New Yorkers (including 32,000 diverting from other modes of transportation) would use the Triboro RX to get to and from their jobs every day. This number that is quite competitive with many existing lines, and without ever touching the island of Manhattan.

At the heart of our ability to make this estimate is the Journey-to-Work data published by the census -- counts of commuters between every census tract and every other census tract in the city. Given these flow data, the shape of the subway network with and without the Triboro RX, and a rough model of how people make travel decisions on public transportation, it's not so hard to guess which subway riders would use a new transit line if it were built. Estimating new transit riders is more nuanced, but we did our best with limited resources.

This study of the Triboro RX has, for me, been much more than a semi-traditional transportation modeling exercise. I took it as an opportunity to get intimately familiar with the state of the art in Open Source mapping and GIS software, including PostGIS, GeoServer, and OpenLayers. These pieces represent a full network-enabled stack for, respectively, storing and manipulating, mapping and presenting, and client-side interfacing of spatial data. I don't think they are quite yet usable by the non-hacker, but I wouldn't be doing this work if I didn't think that my computing skills brought something special to the table. That said, I encourage you to check out the following:

Now, it wouldn't be a perfect project to do, for free, when I should be saving money for school, if it didn't also involve getting my hands dirtier than they already do from all the crumbs in my keyboard. It seems absurd to talk about planning a transit line without actually having visited the areas it would connect. Having synced the clocks on my GPS device and digital camera, I twice explored the Triboro RX right-of-way and its environs from Flatbush to Bay Ridge, in Brooklyn. The results are viewable either in Google Earth or directly on the web. What really struck me was the diversity of neighborhoods -- Flatbush, Ocean Parkway, Borough Park, Sunset Park, Bay Ridge -- traversed by the Triboro RX in less than a third of its length. Continuing on, it runs through East New York, Brownsville, Cypress Hills, Middle Village, Jackson Heights, Astoria, and Mott Haven. You could eat your heart out while getting from Brooklyn to the Bronx, skipping "the city" entirely.

Finally, no contemporary New York transportation project is complete if it doesn't some how tie into Congestion Pricing. In terms of providing mass transit to unserved communities in the outer boroughs, methinks this graphic speaks for itself:

PS Please forgive me, I know these maps need legends for the quantitative parts. It's all a big hack, trust me!

PPS The interactive web maps work much better in FireFox than Internet Explorer. Save your soul and get a real browser.

November 20, 2007

Mailer for Mayor (In Memorium)

In 1969 Norman Mailer ran for Mayor. In 2002 my grandfather (probably the most Mailer-like person I know) gave me an original copy of the campaign poster that he had squirreled away for 30+ years. A week or so ago, Mr. Mailer passed away, so it seems like the appropriate time to put this poster on the web, since I have never been able to find a copy online before. I'm about as far from a knowledgeable design critic as you can get, but this thing is an undeniable work of art, especially in the eye of any native New Yorker.

What a platform:

The boroughs, in order of time I've spent in them:

Some lovely embellishments on the Hudson River:

And my favorite little twist:

Not a bad running mate:

But not a good day either (they came in 4th):

For more info on the campaign itself, check out a recent NYTimes podcast and an interview on WNYC with Jimmy Breslin, Mailer's running mate and another icon of New York realness.

December 3, 2007

Congestion Pricing is a Technology, Remember?

In many ways London's system for Congestion Pricing should be model for New York, but in other ways it really isn't. The most obvious way that it isn't is in the actual technology proposed to do the job. Yes there are cameras and computers involved, that's sort of where the similarity ends.

Specifically, in the UK tradition, all of the cameras relay a full video feed to some central processing location. Not only is this absurdly costly (think fibre!!) but it allows for plenty of privacy invasion by anyone who has access to the cameras' feeds. The proposition for New York is very different. The proposition is much cheaper and seems to all but eliminate the possibility of using the cameras for anything but looking at license plates. That's because the cameras would be equipped with enough smarts to know when to snap a photo, and only that still image would be sent to be processed. If you don't believe me, read this excerpt from IBM's recently released proposal:

A worst case analysis shows that for a very busy lane, with one thousand vehicles passing the detection equipment every hour and forced to send two 100kB images for each vehicle, the bandwidth requirement is a mere 57kB/s. This is within the capacity of wireless networks today, but is not the optimal solution approach.

A more realistic case, in which 50% of vehicles are equipped with an E-ZPass tag, 90% of the remaining license plates are read with a sufficient confidence at roadside and 80% of charges are paid in a timely manner, leads to a bandwidth requirement of 8kB/s. A very busy, six-lane detection point would thus be well within the capacity of NYCWiN, even without local reinforcement of the wireless network.

We estimate that, with our proposed solution approach to vehicle detection at the edge of the network and given the estimated amount of traffic in the city, the average local bandwidth requirement across the system will be on the order of less than 1kB/s per lane, and the overall load on the backbone of the wireless network will be small.

More generally speaking, I think it suffices to say that Congestion Pricing uses technology, and as we know technology only gets better and cheaper over time, so we can be sure that NYC's Congestion Pricing technology will be much better and cheaper than London's.

Now, if only somebody would only explain this to all the privacy freaks and civil libertarians that are making this process so painful...

July 16, 2008

It's the distribution, stupid

Never thought I'd see this in print, but the MTA let NY Times publish a distribution of Metrocard usage for monthly passes (see below).

While the caption of image points out that "some riders use the $81 passes for 40 or fewer rides," it fails to point out that anyone making 46* or fewer trips is losing money on their pass. The calculus of "losing" or "making" money on a monthly pass is of course fraught with nuance (e.g. how much is it worth to me to not have to think about paying on each trip? are any of these passes subsidized?) but the article doesn't touch on it at all.

It's no secret that I love NYC Transit and transit in general, but that doesn't mean people should be buying passes when it's far from beneficial. Don't even ask about the London case...

* 46 trips is the breakeven point when the monthly pass cost $81 and individual trips cost $1.74 (after the bonus)

May 7, 2009

Spark it Up

This post is literally 2 years in the making. In the Spring of 2007, Jeff "You Don't Mess With the" Zupan gave me a spreadsheet with the annual 'registrations' (i.e. recorded entries) at each station in the NYC subway system going back to the beginning (1905). At the time, I was heavy into the new open source geo stack, as is reflected in the main piece of work I did at RPA. Hammer in hand, I of course saw this spreadsheet as a bucket of nails.

The result, after much whacking, is, I think, compelling, but you'll have to see for yourself. The general idea it that the history of subway ridership tells a story about the history of a neighborhood that is much richer than the overall trend. An example, below, shows the wild comeback of inner Williamsburg, and how the growth decays at each successive stop away from Manhattan on the L train:

This is somewhat in contrast to the South Bronx, which is yet to see the resurgence in ridership, other than at Yankee Stadium and the Grand Concourse:

The stations around Wall Street tell a totally different story, in which the ups and downs of each dep/recession have more immediate but temporary effects:

My first stab at visualizing this data was a traditional cartographic approach, showing the overall growth from 1977 to 2006 at each station. This told an approximate story at the level of the whole the city, but did not leave much room for detailed exploration. Thanks to geoserver's awesome new(ish) dynamic symbolizers functionality, it was trivial to plot the station-by-station time series sparklines (generated in R of course) onto the interactive online map. (Originally I the plots were produced in Perl and placed onto the map with a Javascript WFS layer, but that is so 2005.)

For all this, I never really felt like this little experiment was ready for an audience. That all changed when OpenGeo put up its Open StreetMap base layer for the web, giving fancy overlays like this one the context they need.


At least 2 people have taken the data I put out there and used it to make some zippier interactive flash apps:

  • The first is very polished, but I think the designer is quite misled in his desire to not plot dots on a map, and thus to plot what looks like a network flow diagram but with totally bogus data
  • The second is a little rougher around the edges, but I'd say is much more honest, and thus useful

Not sure if anyone knows, but I also have GIS files for the subway here:

March 7, 2012

Time for MTA Bus Time

In mid-January, MTA launched MTA Bus Time, its real-time bus tracking and customer information system. It's currently live on Staten Island and the B63 route in Brooklyn, is expanding to the Bronx this year, and will cover the whole city in 2014

It has web map, mobile web, and SMS text interfaces, along with a fully featured, standards-based developer API. Smartphone users can scan the QR code at each stop to be automatically directed to the Bus Time page for that stop.

Particularly compelling to the author is the approach we've taken to delivering this project quickly and inexpensively (for the short and long term). With MTA as the overall systems integrator, we've ensured that the system is constructed using open standards for all interfaces, and open source software licensing when appropriate. And, oh yeah, it runs on Amazon's cloud.

Please, let us know what you think.

Better late than never right? (this post, not the project...)

About nyc

This page contains an archive of all entries posted to Frumination in the nyc category. They are listed from oldest to newest.

networks is the previous category.

papers is the next category.

Many more can be found on the main index page or by looking through the archives.

Creative Commons License
This weblog is licensed under a Creative Commons License.