Your City's Problem Map Is Really a Map of Who Complains

Open any city's operations dashboard and the map looks like an X-ray of where things are broken: bright clusters of potholes, dead streetlights, missed collections, heating failures. It reads as need. It is not. Every dot on that map is the product of two things multiplied together, a problem and somebody's ability, time and willingness to report it. Change the second term and the map moves even though the streets do not. This article is about what happens when cities forget the second term, and about the growing body of evidence, plus a shelf of entries in this atlas, that lets us measure it.

The map that measures itself

A fixed-time traffic light knows the clock and nothing else. A civic-reporting map has the opposite problem: it knows a great deal, but about the wrong quantity. When residents file to New York's 311, Boston's BOS:311 or FixMyStreet Brussels, the record that lands in the open dataset is a report, not an inspection. Between the pothole and the dot sit a chain of filters: did anyone notice it, did they read it as the city's job, did they know which app or number to use, did they have the phone, the language and the minutes to spare, and did they believe the report would change anything. Each filter has a socioeconomic gradient. So the finished map is a faithful picture of reported demand and a distorted picture of need, and the two are routinely confused because the dashboard does not label which one it is showing.

This is not a hunch. In Kansas City, a study of more than 500,000 service requests, checked against a 21,046-response resident survey and pavement assessments of 29,884 street segments, found lower-income and minority neighbourhoods reported several street and nuisance problems less often even where the independent inspections and the survey said the need was as high or higher. The complaint layer and the condition layer disagreed, systematically, in the same direction.

Hold the problem constant

The cleanest evidence comes from studies that pin the physical problem down and then watch who reports it. Snow is the natural experiment: a storm dumps roughly the same depth across nearby census tracts, so reporting differences cannot be blamed on differences in need. Analysing over 500,000 Boston service requests, Feigenbaum and Hall found that a 10% higher tract income predicted about 3% more snow-removal requests, and that the richer requests arrived disproportionately through the city's smartphone app.

A 2024 study went further and held rubbish constant with a camera. Running a computer-vision model over millions of Google Street View images in Austin, Boston, Detroit, Los Angeles and Philadelphia, the authors measured how much visible trash was actually on each block, then compared it with 311 reports. Conditional on the same measured mess, a one-standard-deviation rise in neighbourhood income was associated with 96.6% more reporting, and a one-standard-deviation influx of white residents with 35.5% more, well above the 8.88% associated with a comparable influx of high-income residents. The same trash, very different maps.

Even resolution speed carries the fingerprint. A team at Cornell used a clever trick to avoid needing an omniscient list of every real incident: when several residents independently report the same fallen tree, the gap between the reports reveals how fast that neighbourhood notices and files. Across 223,416 New York parks reports and more than 900,000 Chicago ones, some neighbourhoods reported up to three times as fast as others. For the highest-risk tree incidents that translated into a resolved-within time of about two days in Manhattan against fourteen in Queens, a gap the model attributes to reporting, not to the crews. The validation is elegant: during Tropical Storm Isaias the parks department received 15,266 tree requests in a single day against a normal 200, and the model's neighbourhood delay estimates correlated with the storm's actual backlog at r = 0.88.

Queens peer-reviewed · Nature Comp. Sci. ≈14 days

Manhattan peer-reviewed · Nature Comp. Sci. ≈2 days

Same hazard class, same agency, same crews. The difference the model isolates is how quickly each neighbourhood turns a fallen tree into a report. Source below.

The subtlest case is the one you cannot photograph. A 2025 model in the Annals of Applied Statistics estimated latent underreporting of heating and hot-water failures, the single largest 311 category in New York, across 1.6 million records. Buildings in the neighbourhoods it flagged as likely to underreport had more elderly residents, more households with children, and more limited-English speakers of Asian languages. Its sharpest illustration is a Staten Island block of three buildings with more than eighty units each, 265 homes, two of them reserved for low-income elderly tenants, that between them placed just 12 heating calls across seven winters. Silence, not because the radiators worked, but because those residents were the least likely to call.

The people who fall off the map

Why they do not call is not one story but several, and this atlas and the literature agree on the channels:

• Device and channel. Reporting has migrated onto apps that not everyone holds on equal terms. San Francisco's 311 now takes 61% of its requests through a smartphone app; Boston's app carried 6% of requests in 2010 and about 28% by 2014. Each migration lets some residents in and prices others out.
• Tenure. Using Boston's data, O'Brien found homeowners were about three times as likely as renters to report a public problem. A renter with the same phone may feel it is not their pavement to police.
• Trust and immigration climate. In Baltimore, immigrant concentration was linked to fewer 311 requests in Latino and Black neighbourhoods but not white or Asian ones, and the Latino effect emerged mainly after 2017, consistent with a chilling effect from the federal policy climate rather than any change on the ground.
• Concentration. The platform can be the work of a tiny minority. In Brussels, an analysis of 30,041 FixMyStreet reports found 67% of users reported just once while a single hyperactive user filed roughly 1,560, and North African, Sub-Saharan and Turkish districts, the city's poorer "croissant," were underrepresented. Boston's archive says the same at the individual level: about 76% of studied reporters filed a single case, and roughly 80% of reports came from within two blocks of the reporter's home.

The self-reinforcing city

These gaps do not stay still; they compound. Analysing 399,364 FixMyStreet users, Sjoberg, Mellon and Peixoto found that a successful first report, a problem actually marked fixed, raised the probability of a second report by about 57%. Responsive neighbourhoods learn to report and generate ever richer data; neglected ones learn that reporting is pointless and go administratively quiet, which then reads on the dashboard as an absence of problems. The map trains the city that made it.

And the map is disturbingly easy to move without touching the city at all. When Boston redesigned its reporting app in 2015, one neighbourhood's requests rose 234.5% against a citywide 82.3%, a Harvard Data-Smart analysis found; the streets did not deteriorate by 234.5%, the interface got easier. The same analysis matched fourteen years of that neighbourhood's resident reports against a computer-vision survey of pavement defects and found that only 7.9% of a 1,000-report sample sat within fifty metres of a machine-detected defect. Two partial sensors, looking at the same streets, largely disagreeing about what is wrong.

Don't throw the map away

Here is where honesty cuts the other way, because the tempting conclusion, that complaint data is junk, is also wrong. Three correctives keep the thesis from tipping into a takedown:

1. For visible, salient problems the map can be accurate. Across 369,581 Chicago rat complaints checked against trapping, complaint volume alone explained most of the variation in where rats actually were, an adjusted R² of 0.67, rising to 0.84 with housing tenure and vacant land added. A rat, unlike indoor heat loss, is salient to everyone on the block.
2. The response stage can be fairer than the reporting stage. Examining 311 systems in fifteen US cities, Clark and colleagues found no systematic pattern of slower responses to poorer or minority neighbourhoods once a request was in the queue. Unequal reporting does not automatically become unequal service; institutions sit in between, and a broader four-case study by Mellon and colleagues makes the same point, that inequality at one stage need not propagate to outcomes.
3. Easier reporting can enfranchise the previously silent. Not every surge from a wealthier-looking app base is harmful noise; some is suppressed legitimate demand finally becoming visible.

But the counter-examples also mark the limits of "just collect more." In Kampala, a field experiment that recruited fifty reporters in each of a hundred neighbourhoods generated 23,856 reports over nine months and produced no measurable improvement in waste collection. Voice without the crews, budgets and authority to act is just a better-documented failure. And "resolved" is its own trap: in Mumbai, a study of about 20,000 digital water complaints found the bureaucracy formally closed almost all of them, yet a supervised classifier judged only 44% of the official replies to contain a substantive response.

What this atlas already shows

That last trap runs straight through our own corpus, and it is worth saying plainly because these are live, fact-checked entries, not hypotheticals. Not one of the 311 systems we have catalogued in Boston, New York, Toronto, Montreal or Washington DC has an independent audit confirming that a case marked "closed" was actually fixed, and several of the cities' own dashboards concede that vague resolution text makes it impossible to tell. New York's City Council dashboard says so outright. Toronto's own glossary warns that meeting a service standard can mean an initial investigation rather than a repair.

Bangkok's Traffy Fondue is the same lesson from the other side of the world. The platform is a genuine success at routing, over a million reports handled, but at the April 2024 council session the split was 79% resolved, 17% forwarded to agencies outside the city administration where they vanish, and 2% still pending, and councillors have warned that tickets get closed to protect departmental scores. As the atlas entry puts it: a closed ticket records that an office responded, not that the problem stayed fixed. The academic write-up of the platform reports 91% citizen and 84% staff satisfaction, both operator-measured (operator-reported).

The map is also only as durable as the platform beneath it, and here the atlas has caught things a single-city view would miss. The Open311 standard that Washington DC authored now points at an endpoint that went dark, timing out in July 2026, while San Francisco, which merely adopted the standard later, still serves live records. Miami's unusually complete record, with a target and an overdue flag on every case, describes a system frozen at an August 2024 platform migration, and 86% of its requests came by phone against 6.5% by app, the mirror image of San Francisco. Montevideo's FixMyStreet-based portal simply closed in 2022. And who is on the map differs so much by channel that the channel deserves its own chart.

San Francisco, 2025 city open data 61%

Boston, 2014 city figure 28%

Miami, to 2024 city open data 6.5%

The channel is not a detail; it selects who ends up on the map. A city read mostly through an app hears from different residents than one read mostly through a phone line.

The honest fix

None of this argues for deleting the dashboard. It argues for reading it correctly and for three specific disciplines. First, triangulate: treat complaints as one sensor with a known bias and fuse them with rotating inspections, representative surveys and physical sensors that have different error structures. The newest research points the way. A 2026 method fused 9.6 million crowdsourced reports with 1.04 million government inspection ratings across 139 incident types and beat either source alone, most of all where inspections were sparse. The caution is that a fusion model inherits the biases in how those inspections were chosen, since many inspections are themselves triggered by complaints, so the "independent" benchmark can quietly re-import the original skew. A related warning sits in this atlas: a Washington DC rat-risk model trained on 311 data predicted future complaints well but flunked a field test of where rats actually were. A model trained on reports learns who complains.

Second, label the map honestly: a dashboard should say "reported demand," not "need," and budget papers should distinguish the two. Third, keep the non-digital doors open, the counter, the phone line, the SMS route, because a map assembled only from the digitally fluent is a self-portrait of the fluent. Brussels has gone furthest here, requiring since 2024 that new digital public procedures keep at least one non-digital alternative and pass a digital-inclusion review before launch.

The uncomfortable part is that correcting the bias is not free. When the Cornell team simulated equalising reporting rates in New York, service in under-reporting Queens sped up by about 2.5 times, but Manhattan slowed by about 2. Debiasing does not conjure extra crews; it reallocates the ones you have. That is a political choice, and it should be made in daylight rather than hidden inside a data-cleaning step.

Verdict

Crowdsourced reporting is one of the genuinely useful things a city can build, and this atlas is full of platforms that route a real complaint to the right desk faster than any hotline of the past. But a complaint map is a measurement, with a bias as predictable as a miscalibrated sensor, and the bias runs toward the residents who already have the most access, tenure, trust and time. Treat it as an X-ray of need and you will keep sending the crews to the neighbourhoods best equipped to ask, then point at the resulting data as proof you were right to. The fix is not more dots. It is knowing what the dots are made of.

Is this a map of where the city is broken, or a map of who believed it was worth telling us?

— the question every civic dashboard should carry as a caption

The related pillars in this atlas ask versions of the same question from other angles: how cities are ranked when the yardstick is chosen by the ranked, showcase or use case when a pilot is dressed as a system, and why smart cities fail when the technology works and the institution does not.

Frequently asked

Does 311 data show where a city's problems actually are?

Not directly. It shows where problems were reported, which is the underlying problem multiplied by the probability someone noticed it, knew how to report it and expected the report to help. Studies find that probability varies with income, language, tenure and trust, so raw counts measure reported demand, not need. A 2025 model in the Annals of Applied Statistics found neighbourhood socioeconomic characteristics predict which New York buildings underreport heating failures.

Are 311 apps biased toward wealthier neighbourhoods?

Often, but not universally. Holding visible trash constant with computer vision across five US cities, a 2024 Socius study found a one-standard-deviation rise in income was associated with 96.6% more reporting. In Boston, where snow makes need roughly equal, a 10% higher tract income predicted about 3% more snow-removal requests. But direction depends on the issue: Chicago rat complaints track measured rat abundance well, because a rat is visible to everyone.

Does a 311 case marked "closed" mean the problem was fixed?

No, and cities rarely check. Closure can mean fixed, duplicated, transferred, out of jurisdiction, or simply that an office responded. Boston, New York, Toronto, Montreal and Washington DC all publish closure timestamps, but none has an independent audit confirming a closed case was repaired, and in Mumbai a classifier judged only 44% of formally closed water complaints to contain a substantive reply.

How should cities use crowdsourced reporting data fairly?

Treat it as one sensor among several. Fuse complaints with rotating inspections, representative surveys and physical sensors; keep non-digital channels (counter, phone, SMS) open so the map is not just the digitally fluent; and label dashboards as reported demand, not need. Be honest that correcting the bias reallocates scarcity: a New York study showed equalising reporting rates would speed service in Queens but slow it in Manhattan.

Sources

Evidence policy: every figure above links to its source via the dotted ⓘ marks; operator- and self-measured numbers are labelled as such, and simulation results are kept out of the charts. Spotted an error? Tell us.