AUDIT — resolution-equity

Honest self-critique of the resolution-equity case study. Originally written for the 2026-04-20 rebuild; revised 2026-07 for a full cold-run rebuild on corrected, hour-grain data (see §10 for the dated addendum). Voice is pragmatic, not paper-cadence. Readers interested in the publication-style write-up should go to MANUSCRIPT.md.

1. What this showcase does and does not claim

It claims: for the first-of-each-month sub-sample of NYC 311 noise-related complaints between 2020-01-01 and 2026-06-30, the measured mean resolution time at the community-board × month level is not patterned in the direction a naive equity story would predict. Bronx resolves faster than Manhattan; within-borough heterogeneity dwarfs between-borough heterogeneity (Theil T_between / T_overall = 8.68%). Seasonal variation (~139.7 hours peak-to-trough amplitude, 11.4% of variance share) exceeds the between-borough gap in absolute magnitude, though residual month-to-month variation (83.5% of variance share) dominates the series.

It does not claim:

• Causal identification. All regressions are descriptive conditional associations. Treatment here is a composite construct (borough demographic exposure) that co-varies with dozens of unobservables; the coefficients are correlations with fixed-effects structure, not policy counterfactuals.
• Tract-level equity. Our analysis is community-board × month. A tract-level analysis (~2,300 × 78 ≈ 179,000 cells) per the modernization plan would require a tract↔community-district crosswalk that nyc311 does not ship and that we did not add under this branch.
• Completeness of the 311 system. Only resolved complaints enter the analysis (99.1% of the 164,515-record pull had closed_date populated; a further 1,156 rows (0.7%) were dropped as closed_date <= created_date or duration > 365 days, leaving 161,862 usable). Complaints that never resolve are excluded.

2. Hour-grain resolution latency — RESOLVED UPSTREAM (nyc311 1.0.4)

Status (2026-07): Closed — nyc311 1.0.4 fixes a bulk_fetch CSV-cache defect that dropped closed_date and truncated created_date / closed_date to whole calendar dates. This study now pins nyc311 >= 1.0.4 and computes resolution latency at hour grain (closed_date minus created_date) straight from nyc311.pipeline.bulk_fetch. notebooks/01_load_and_explore.py fetches the full-corpus stream and filters to day-1-of-month rows client-side; there is no hand-rolled Socrata pagination.

Original gap (preserved for context): before 1.0.4 the on-disk CSV cache quantized every timestamp to midnight, so every latency this study computed collapsed to an integer number of days. The published April 2026 write-up therefore reported resolution time in days, not hours — a silent data defect, not a modeling choice. The 1.0.4 patch preserves full second-resolution timestamps in the cache, and this cold-run rebuild is the first pass to measure resolution time at hour grain (median 0.86 hrs, mean 39.06 hrs across 161,862 usable records).

3. Day-1-of-month sampling

Notebook 01 calls nyc311.pipeline.bulk_fetch over the 2020-01 → 2026-06 window and keeps the 164,515 records whose created_date fell on the 1st of a calendar month, filtered client-side. This keeps the first-run round-trip short, at the cost of a within-month sampling bias we have not formally corrected. Two things make the bias small in practice for this analysis:

• Resolution-time distributions are agency-level — DEP and NYPD SLA deadlines are the same on every day of the month — so first-of-month draws are not atypical on the outcome dimension.
• Monthly means are still computed per community-board × month cell, and the minimum-5-complaints-per-cell filter excludes cells where sampling noise would swamp the signal. After the filter we retain 4,559 of 5,382 possible cell-months (84.7%).

We did not bootstrap a sampling-bias correction or run the full panel to confirm the bias is small. A follow-up run with a richer sample (days 1-14 of each month) is the natural next iteration.

4. Why statsmodels OLS instead of factor_factory.engines.panel_reg

factor_factory.engines.panel_reg.estimate in 1.0.2 builds its regressor matrix from the Panel.outcome_col plus regressors= tuple. The engine's record-view flow attaches covariates as per-record attributes; the outcome is a count (number of records per cell). Our outcome is a custom float (mean resolution hours within a cell), not a raw count. Threading a custom float outcome through the engine requires either (a) constructing a synthetic record set where cell weights encode the mean — lossy — or (b) calling pyfixest directly outside the engine wrapper.

We chose the simplest faithful path: statsmodels.formula.api.ols with HC1-robust SE for the pooled specs and cluster-on-board SE for the TWFE spec. The reporting discipline (SE, 95% CI, t, exact p, R², n, df) matches factor-factory's PanelRegResult field-for-field (see artifacts/twfe_results.json), so downstream automation is not blocked.

Upstream fix: a future factor-factory release that accepts a pre-computed outcome_col on a Panel built from a pandas DataFrame (rather than requiring record aggregation) would remove this engine-fit friction.

5. Spatial geometry — real centroids

Notebook 04 places each community board at its nyc-geo-toolkit representative-point centroid — centroids_from_boundaries( layer='community_district', representative=True), which returns shapely.representative_point() rather than the bare geometric centroid, guaranteed to fall inside the polygon (this matters for non-convex CBs along the Brooklyn / Queens shorelines, e.g. Coney Island CB 14, Far Rockaway CB 14). An earlier build used a hash-placed-centroid shim; that shim is retired and no longer appears anywhere in the pipeline.

Current Moran's I on pooled resolution hours (real centroids, extended 2020-01 → 2026-06 window, 999 permutation reps):

band (km)	mean neighbors	Moran's I	z	p_perm
1 km	0.00	undefined	—	—
2 km	0.44	+0.102	0.44	.380
5 km	5.90	+0.107	1.77	.055

Neither band reaches conventional significance. The 1 km band is undefined (no CB pair falls within 1 km at community-board scale); the 2 km band is clearly non-significant (p = .380); the 5 km band is marginal (p = .055), just above α = .05. Both point estimates are small and positive but the permutation null cannot be rejected. LISA at 2 km labels all 59 matched boards "ns" — no HH/LL/HL/LH clusters. The honest current read is no statistically significant spatial clustering.

Note: an interim note drafted against the pre-rebuild artifacts (broken, shorter window) expected the 5 km band to clear significance (p ≈ .03). That number was an artifact of the day-grain, 2020-2024 run; on the corrected hour-grain, extended-window data the 5 km band is only marginal and the conclusion is a null, not a boroughwide-clustering finding.

Unmatched CBs: 10 of the 69 panel community boards do not match the 59 geometries shipped by nyc-geo-toolkit (pseudo-CBs like "UNSPECIFIED BRONX" and airport catchment zones). Those rows drop out of the spatial analysis but remain in the panel regressions in notebook 03.

6. Engine / adapter inventory

Engine	Family	Status	Notes
factor_factory.engines.stl	STL	used	Ran directly on city-monthly series; not through Panel adapter
factor_factory.engines.changepoint	PELT	used	ruptures under the hood
factor_factory.engines.inequality	Theil	partially	TheilEngine's canonical path wants a count-outcome Panel; we reproduced the decomposition math inline (code path verified against engine formula)
factor_factory.engines.panel_reg	TWFE	not used	See §4 above
factor_factory.engines.spatial	Moran's I / LISA	not used	Needs a Panel with (latitude, longitude) record-view; our CB-aggregated outcome is not a per-record panel. Hand-rolled distance-band version substituted.
nyc311.pipeline.bulk_fetch	—	used	1.0.4+ preserves full-timestamp closed_date in the cache; hour-grain latency; §2 resolved
nyc_geo_toolkit.centroids_from_boundaries	geometry	used	representative-point mode; real centroids; see §5
statsmodels.formula.api.ols	OLS	used	HC1 + cluster-on-board SE for the four equity specs

Net: factor-factory earns 2 of 5 expected uses. The gap is primarily an API contract issue (engines assume count-outcome Panels built from raw records; we have pre-aggregated float outcomes). A 1.1.x engine-Panel adapter that accepts pre-aggregated outcomes would close this gap for every aggregate-outcome case-study this ecosystem will ever host.

7. Reproducibility checks

Checked:

• uv run jellycell run over notebooks 01-05 re-runs them cleanly on a cold cache (verified manually this session).
• uv run jellycell render produces HTML under site/ for all five notebooks + an index.
• uv run jellycell lint exits 0 with no violations.
• The tearsheet export writes five files under manuscripts/tearsheets/ without error.
• All artifacts/*.json are stdlib-only JSON (no NumPy-specific serializers) and re-readable with plain json.loads.

Not checked (deferred):

• Byte-identical regeneration of FINDINGS.md across two clean runs. The findings tearsheet uses fixed override values so it regenerates deterministically, but we have not run the diff-empty check twice.

8. Reproducibility notes

A few implementation choices are shaped by the reproducible-notebook toolchain rather than the analysis itself, and are recorded here so a reader regenerating the study understands them:

• Every notebook cell imports what it uses inline (no shared setup cell), so cells cache and re-run independently.
• Figures are inlined as static images for portability across notebook viewers.
• All committed artifacts/*.json are plain stdlib-serializable JSON (no NumPy-specific encoders), re-readable with json.loads.

9. What would the next iteration look like

1. Descriptor-level stratification — the highest-value follow-up. The Bronx-faster gap is most plausibly a complaint-descriptor-mix artifact: Bronx volume skews to "Loud Music / Party" (routes to NYPD, closes fast at scene); Manhattan skews to commercial / construction descriptors (route to DEP with multi-day SLA targets). The cleaned record stream already carries the descriptor column (artifacts/clean_records.parquet); a within-descriptor panel that compares resolution times across boroughs would test whether the gap survives descriptor control or dissolves into a routing / composition artifact — the load-bearing question for any equity claim here.
2. Tract-level panel via nyc-geo-toolkit crosswalk (~30 more observations per borough; dilutes within-borough Theil slightly).
3. Tract-level ACS covariates (not just borough): real Oaxaca-Blinder decomposition into race, poverty, age endowments becomes well-defined.
4. Sampling fidelity: pull days 1-14 of each month to reduce within-month sampling noise.
5. Additional outcome: pct_same_day as a robustness DV. Already computed per cell (column pct_same_day on artifacts/panel.parquet); regress it with the same four M1-M4 specs and report alongside resolution-hours.

(The two items that appeared here in the April 2026 revision — closed_date upstream in nyc311 and real CB centroids — are both resolved as of this rebuild; see §2, §5, and the §10 addendum.)

10. Addendum — 2026-07 cold-run rebuild

This revision documents a full cold-run rebuild of the study. The prior (April 2026) run shipped with a silently broken data cache; every number in it should be treated as superseded by the current artifacts. Four things changed:

1. Hour-grain resolution latency (nyc311 1.0.4). The published run computed resolution time in whole days because the bulk_fetch CSV cache dropped closed_date and truncated timestamps to calendar dates. nyc311 1.0.4 fixes the cache; the study now pins nyc311 >= 1.0.4 and measures latency at hour grain (median 0.86 hrs, mean 39.06 hrs). See §2.

2. Extended window. The panel now spans 2020-01 → 2026-06 (78 months) rather than 2020-2024 (60 months): 164,515 day-1 records fetched, 161,862 usable, 4,559 cell-months across 69 community boards. Headline gap widened from the day-grain estimate to a 64.6-hour Bronx-faster gap (Bronx 8.63 hrs vs Manhattan 73.24 hrs).

3. Real-centroid spatial re-run. Moran's I was recomputed on nyc-geo-toolkit representative-point centroids over the extended window. The result is no statistically significant spatial clustering (2 km p = .380; 5 km p = .055, marginal; 1 km undefined; LISA all-ns). This overturns an interim note that expected a significant 5 km band — that expectation was computed on the broken, shorter-window artifacts. See §5.

4. Centroid-merge label-format fix. The community-board labels in the panel are normalized to the format the nyc-geo-toolkit centroid layer keys on, so cells merge to geometry correctly; 59 of 69 boards now match (10 pseudo-CB / catch-all rows remain unmatched, as before).

Net effect on the narrative: the direction of the headline (Bronx resolves faster than Manhattan) holds, but the magnitude, the panel dimensions, the inequality shares, the seasonality share, and the spatial finding all changed. The honest current story is a large Bronx-faster gap that is almost entirely within-borough (91%), most plausibly an agency-routing / descriptor-mix artifact, with no significant spatial clustering and no COVID structural break — not a demographic-bias finding.