Survey Setup

Data for this study come from an original survey of 2,508 respondents across three U.S. cities: Miami ( $ n=473 $ ), New York City ( $ n=450 $ ), and Phoenix ( $ n=1,585 $ ). These cities were chosen to provide a variety of political and regional contexts, with the aim to collect large enough samples for each city to conduct within-city analysis.

Survey respondents were recruited via e-mail using a list of email addresses attached to registered voter records.Footnote ⁴ The list was provided to the researchers by the vendor TargetSmart. Those who did not respond to the initial invitation were sent up to 3 weekly reminder e-mails. No compensation was offered or provided to respondents and no deception was employed in this study. Section S1 of the Supplementary Material contains more information on the sampling process.

Among voters who received a survey invitation, response rate to the survey invitations was 0.8% (0.5% among total voters sampled). This is only slightly lower than previous survey recruitments using another email list a similar vendor (see, e.g., Brown and Enos Reference Brown and Enos2021). In total, we collected 7,691 responses, split evenly between five experimental conditions (see the Additional Supplementary Material for the details of the experimental conditions), but limit analysis to the 2,508 that drew their neighborhood on our mapping tool. We use the data from the control group to introduce and illustrate the statistical model while we discuss the details of the experiment and present experimental results in Additional Supplementary Material.

Respondents who accepted the invitation to take the survey were presented with a consent form informing them they were taking part in a research study. The consent form was followed by demographic questions including partisanship, race, age, income, employment, homeowner status, whether they had children, marital status, and how long they had lived at their current residence.

Section S1 of the Supplementary Material presents the summary statistics of the survey sample (control group). Across cities, the sample is approximately evenly divided between self-identified Democrats and Republicans, and a majority of respondents reported voting for President Biden in the 2020 general election. Tables S1 and S3 in the Supplementary Material compare the summary statistics of the sample with those of the overall adult population of the three metropolitan regions under study. We find that our sample is more predominantly white, wealthier, educated, and more likely to be a homeowner than the population of each of the cities in our sample.

Mapping Tool

Next, respondents were presented with an embedded mapping application where they enter their residential address, at which point the maps zooms to a centered view of their address. Then, the underlying census block grid was shown on the map over the road base map, and respondents used the brush tool to select the census blocks that they considered a part of their “local community.”Footnote ⁵ Our first survey used this terminology, mirroring previous surveys that asked respondents to draw their own neighborhoods (Wong et al. Reference Wong, Bowers, Rubenson, Fredrickson and Rundlett2020). These authors have shown that the phrase “local community” is a tangible concept in people’s minds, and further demonstrate the consistency of drawn neighborhoods when re-contacting survey respondents.

Our mapping application is comparable to these previous surveys in functionality. One difference is that in this case, rather than having respondents use a drawing tool to draw a circle around their residence that constitutes their neighborhood, the application offers a brush tool to shade in the census blocks around the residence that are included in the neighborhood. Respondents could zoom in or out on the map, and were able to make edits to their neighborhood after the initial shading. The only constraint was that neighborhoods had to be contiguous. Figure 1 shows a screen shot of the map drawing tool. We make our map drawing tool publicly available so that other researchers can use it for their own surveys (https://github.com/CoryMcCartan/neighborhood-survey). In particular, the tool can be embedded into a popular survey platform such as Qualtrics as done in our survey.

Figure 1.

Map with Brush Tool Used to Draw Neighborhoods

Descriptive Statistics of Drawn Neighborhoods

Figure 2 shows the central tendencies and range of the population, area, proportion Democrat, proportion Republican, proportion white, and proportion Black of all drawn neighborhoods, broken out by city as well as pooled. Partisanship is measured using TargetSmart voter records of every registered voter in the three cities, using information on the residential location of each voter to create aggregate counts by census block. The population, area, and racial demographics are measured using the 2010 census.

Figure 2.

Descriptive Statistics for Respondent Neighborhoods

We find a wide range of neighborhood sizes, both in terms of population and land area. The median number of residents contained in a drawn neighborhood is 3,020 residents, while its range extends from single digits to over 340,000 residents. Similarly, the median area is 0.78 square miles, but the entire distribution ranges from 0.01 to 344.74 square miles. This variation in size of neighborhoods is indicative of the variation in how different individuals experience their local geography and define their neighborhood. Accounting for this heterogeneity is critical in our statistical model we introduce in the next section.

We also find substantial variation in the demographics of the drawn neighborhoods (see Section S2 of the Supplementary Material). Figure S1 in the Supplementary Material shows the distribution of proportion white and proportion Black in drawn neighborhoods separately for white and non-white survey respondents. Drawn neighborhoods from white respondents are on average 21.1 percentage points whiter than those from non-white respondents. Figure S2 in the Supplementary Material contains the distribution of proportion Democratic and Republican separately by respondent party registration, with Democrats drawing consistently more Democratic neighborhoods and Republicans drawing more Republican neighborhoods.

These differences likely reflect objective differences in racial and partisan exposure across race and party, but may also be influenced by conscious or subconscious motivations for respondents to construct their subjective neighborhoods to include more members of their own racial or partisan in-group. Our statistical model can quantify the extent to which, net of other variables that may determine whether someone includes an area in their neighborhood, the added predictive effect of racial or partisan demographics on neighborhood inclusion. We now turn to our proposed statistical model of subjective neighborhoods.

Notation and Setup

For ease of notation, we begin by describing the model for a single neighborhood drawn by one respondent. We start with an undirected graph $ G=(B,E) $ representing the layout of the city or town, with each vertex $ {B}_i\in B $ corresponding to a census block and the edges E corresponding to block adjacency. We write $ {B}_i\sim {B}_j $ if $ {B}_i $ and $ {B}_j $ are adjacent, that is, the edge $ (i,j)\in E $ . We use $ K:=|B| $ to denote the total number of blocks. In Figure 3, for example, the block with respondent’s residence is adjacent to four blocks labeled as “1a,” “1b,” “1c,” and “1d.” We do not consider two blocks that are touching one another with a single point as adjacent blocks (i.e., no point contiguity). Thus, the block with respondent’s residence is not adjacent to blocks “2a,” “2b,” and “2c.”

Figure 3.

Model Schematic

Note: The respondent’s location is indicated by the black house in the center. Blocks are labeled in the order in which they are considered for inclusion in the neighborhood, with the number indicating the graph-theoretical distance and the letter the spatial distance tiebreaker. Blocks shaded purple have been included in the neighborhood, whereas blocks shaded light orange have been excluded. Parentheses around a block label indicate that it will not be considered for inclusion in the neighborhood because none of its neighboring blocks which are closer to the respondent belong to the neighborhood.

Without loss of generality, we number the blocks in order of their distance from the block where the survey respondent resides, according to some distance function $ d:B\to \mathbb{R} $ , so that $ {B}_0 $ is this block, $ {B}_1 $ its closest neighbor, and so on. Thus, if $ i<j $ then $ d({B}_i)<d({B}_j) $ . In this application, we take d to be the graph-theoretical distance (i.e., the minimum number of edges between two nodes), with ties broken by spatial distance (i.e., the distance between centroids of census blocks). Figure 3 illustrates this ordering scheme. For example, block “1d” in the figure is numbered “1” because it is one step away from the respondent’s block, and “d” because among all the one-step-away blocks, it is the fourth closest to the respondent spatially. We note that the use of graph-theoretical distance introduces some dependency between the distance measure and population density: areas with higher population density generally contain smaller blocks. Graph-theoretical distance will increase faster in these areas over a given spatial distance compared to areas with lower population density.

Let $ {Y}_i $ be an indicator variable for the inclusion of $ {B}_i $ in the respondent’s neighborhood, so that the neighborhood itself may be defined as the set of blocks with $ {Y}_i=1 $ , that is, $ Y=\{{B}_j:{Y}_j=1\} $ . Since we require any neighborhood to contain the block where respondent’s residence is located, we always have $ {Y}_0=1 $ . We define a following connectivity indicator $ {C}_i $ to be 1 if block $ {B}_i $ is connected to the respondent’s neighborhood by way of any closer blocks. Formally,

$$ \begin{array}{rll}{C}_i=1\{\mathrm{there}\ \mathrm{exists}\ \mathrm{a}\hskip0.3em j<i:{B}_i\sim {B}_j\hskip0.3em \mathrm{and}\hskip0.3em {Y}_j=1\}.& & \end{array} $$

This indicator checks whether, among the blocks which are closer to $ {B}_0 $ than $ {B}_i $ is, if any are in the respondent’s neighborhood. In Figure 3, blocks with parenthetic labels have $ {C}_i=0 $ . For instance, block “3d” would be expected to be considered before “3e,” “3f,” etc. based on its location, but since none of “2d,” “3a,” and “3c” are in the neighborhood, “3d” is never considered.

The Model

Under the proposed model, a neighborhood is generated sequentially, starting with $ {B}_0 $ and adding blocks in order of increasing distance from $ {B}_0 $ according to a probability. This probability is heavily influenced by the graph-theoretic distance between the block under consideration $ {B}_i $ and $ {B}_0 $ , and its connectivity. In particular, we assume that the neighborhood is connected and our survey tool does not allow respondents to draw disconnected neighborhoods.

The core of the model is

$$ \begin{array}{rll}{Y}_i\mid {Y}_0,\dots {Y}_{i-1}\sim \mathrm{Bernoulli}({\pi}_i\cdot {C}_i),& & \end{array} $$

where $ {\pi}_i $ is the inclusion probability of block $ {B}_i $ into one’s neighborhood provided that it is connected. As long as $ {\pi}_i\to 0 $ as $ d({B}_i)\to \infty $ , the generated neighborhoods will be bounded around $ {B}_0 $ almost surely. Figure 3 illustrates the state of the neighborhood partway through the generation process, when block “3i” (shaded gold) is under consideration. The process concludes once the neighborhood is surrounded by light orange blocks, since then there are no blocks left which could be added while keeping the neighborhood contiguous.

The specification of $ {\pi}_i $ determines the type of neighborhoods that are generated. Let $ \mathbf{X} $ be a $ m\times K $ matrix of predictors, not including an intercept, with $ {\mathbf{x}}_i $ the column vector of m predictors for block $ i=1,2,\dots, K $ . These may include the characteristics of the respondent, those of the graph or map (e.g., the demographics of blocks, locations of landmarks and roads), and their interactions. The inclusion probability can also depend on the inclusion of blocks whose distance to $ {B}_0 $ is less than that of the block under consideration $ {B}_i $ . This means that the predictors can include the information about the partially drawn neighborhood. The factorization formulation above, however, precludes the possibility that $ {\mathbf{x}}_i $ depends on $ \{{Y}_j:j\ge i\} $ , that is, the inclusion of farther-out blocks.

We model the inclusion probability using a kernel function that smoothly decays as the distance between $ {B}_i $ and $ {B}_0 $ grows:

$$ {\displaystyle \begin{array}{l}{\pi}_i=\pi \left({\mathbf{x}}_i,\boldsymbol{\beta}, \alpha, L,\sigma \right)=\mathrm{\exp}\left(-,|,\frac{d_{\mathrm{sp}}\left({B}_i\right)}{L}\mathrm{\exp}\left({\mathbf{x}}_i^{\top}\boldsymbol{\beta} +\varepsilon \right){|}^{\alpha}\right)\\ {}\hskip1em \mathrm{and}\hskip0.35em \varepsilon \hskip0.3em \sim \hskip0.3em \mathcal{N}\left(0,{\sigma}^2\right),\end{array}} $$

where $ \varepsilon $ is the respondent random effect, $ {d}_{\mathrm{sp}}({B}_i) $ represents spatial distance between $ {B}_0 $ and $ {B}_i $ , L controls the scale of decay, and $ \alpha $ controls its rate. In particular, $ \alpha $ represents the sharpness of the neighborhood boundary. Along with L, the random effect $ \varepsilon $ plays an important role in addressing the heterogeneity of neighborhood size across individual respondents. Figure 4 visualizes the kernel function we use where we choose an arbitrary length scale (horizontal axis) for illustration. As the value of $ \alpha $ increases, the inclusion probability decays faster as a function of distance.

Figure 4.

Illustration of Kernel Function across a Range of Values of the $ \alpha $ Parameter, Indicated by Different Colors

Note: The length scale shown here is arbitrary; in the model, it is estimated as the L parameter.

Conveniently, the model reduces to a Bernoulli GLMM with complementary log-log (cloglog) link function for the exclusion probability

(1) $$ 1-{\pi}_i={\displaystyle \begin{array}{l}1\hskip-1pt -\hskip-1pt \mathrm{\exp}\Big\{\hskip-2pt -\hskip-1pt \mathrm{\exp}\hskip-2pt \Big(\alpha \log {d}_{\mathrm{sp}}\left({B}_i\right)-\alpha \log L\hskip-2pt +\hskip-2pt \alpha {\mathbf{x}}_i^{\top}\boldsymbol{\beta} +\alpha \varepsilon \Big)\hskip-2pt \Big\}.\end{array}} $$

That is, we can fit the model by regressing the non-inclusion indicators on the log distance, any covariates, and an individual random effect, using the cloglog link function. As discussed previously, we do not include every census block in the model—just those that are included in the neighborhood, and those which are not but border the neighborhood (i.e., those for which $ {C}_i=1 $ ). This reflects the sequential generation process that underlies the model.

Since the link function is nonlinear, the marginal effect of each covariate varies by its underlying value, and is not simply equal to the value of the coefficient. We choose to focus our interpretation on the effect of each covariate on the “margin” of the neighborhood, where the probability of a block’s inclusion is 50%. We interpret the coefficient estimates by calculating

$$ \begin{array}{rll}\mathrm{exp}(-\mathrm{exp}({\mu}_{0.5}+{\beta}_j))-\mathrm{exp}(-\mathrm{exp}({\mu}_{0.5}))& & \end{array} $$

for each coefficient $ {\beta}_j $ , where $ {\mu}_{0.5}:=\log \log 2 $ is the value of the linear predictor which corresponds to an inclusion probability of $ 0.5 $ . This quantity reflects the percentage point change in the probability of inclusion for a one-unit change in the covariate $ {\mathbf{X}}_j $ , at the margin.

The model is completed with the following prior distributions:

$$ \begin{array}{rll}\alpha \log L\sim {\mathrm{t}}_3(0,2.5)\hskip1em \mathrm{and}\hskip1em \sigma \sim {\mathrm{t}}_3(0,2.5).& & \end{array} $$

Prior distributions for the coefficients are formed indirectly by taking a QR decomposition of the centered covariate matrix (including the log distance variable) (Goodall Reference Goodall1993). The coefficients on the QR-decomposed (centered) covariates are given a $ {\mathrm{t}}_2(0,2.5) $ prior. This setup implies priors for the actual coefficients of interest which are weakly informative and adapted to the scale and correlation of the covariates. We have used importance sampling to fit the model under alternative prior specifications and found no measurable changes in the posterior distribution of the parameters.

Because of the sequential generation and the indicator function, the posterior distribution simplifies to

$$ \begin{array}{lll}p(\theta \mid G,Y,\mathbf{X})\hskip0.35em \propto \hskip0.35em p(\theta ){\displaystyle \prod_{i=1}^K}p({Y}_i\mid {Y}_0,\dots, {Y}_{i-1},{\mathbf{x}}_i,\theta )& & \\ {}\hskip5.5em =p(\theta ){\displaystyle \prod_{i:{C}_i=1}}p({Y}_i\mid {Y}_0,\dots, {Y}_{i-1},{\mathbf{x}}_i,\theta )& & \\ {}\hskip5.5em =p(\theta ){\displaystyle \prod_{i:{C}_i=1}}\pi {({\mathbf{x}}_i,\theta )}^{Y_i}{\{1-\pi ({\mathbf{x}}_i,\theta )\}}^{1-{Y}_i},& & \end{array} $$

where $ \theta =(\boldsymbol{\beta}, \alpha, L,\sigma ) $ is shorthand for the parameters, and $ p(\theta ) $ is its prior distribution. This formulation only requires the computation of the likelihood for the blocks in the drawn neighborhood and all of their adjacent blocks.

We assume individual responses are exchangeable, allowing us to simply multiply their likelihoods to create a joint model for all responses. The $ \beta $ , L, $ \sigma $ , and $ \alpha $ are shared across responses, but each respondent has its own error term $ \varepsilon $ common to all of its blocks. As appropriate, $ \beta $ may also contain hierarchical terms that vary by demographic categories, or metropolitan area or subdivision. Computational details for fitting models are described in Section S3 of the Supplementary Material.

Model Specification

To illustrate the proposed model, we apply it to our survey data using the control group alone. We limit our analysis to the 468 respondents in the control group who drew a map that consisted of more than one census block. Because the census block that contained the residential address is highlighted by default, we cannot distinguish between respondents who selected single block neighborhoods and those who entered a residential address but decided not to draw a neighborhood. In Table S4 in the Supplementary Material, we show that neither respondent characteristics nor the experimental conditions are powerful predictors of who draw usable maps.

We fit a full model, which includes demographic information, as well as a baseline model, which includes only geographic information. We fit this model to a random sample of four hundred respondents from the 468 in the control group. The unsampled 68 will be the test set for our prediction analysis in the “Predicting Neighborhoods” section. Comparing the predictions of these two models will allow us to quantify the extent to which demographics contribute to the prediction of subjective neighborhoods. In the full model, we include as predictors individual characteristics consisting of voter race, political party, homeowner status, educational attainment, income, age, retirement status, and length of residence in current home. Individuals that differ along these characteristics may view their local area differently, and we quantify the predictive power of these factors about drawn neighborhoods.

We also include geographic characteristics of census blocks including race, party, and education demographics, whether the block contains a school, park, or church, and the distance to the closest of each of these features, whether the block is in the same block group as the voter’s residence, the same census tract, whether the block is bounded by the same major roads and railroads as the respondent’s residence, block population, and block land area. These aggregate characteristics account for features of place that may influence whether respondents include census blocks in their neighborhood. In particular, indicators for same block group, same census tract, and same road/rail regions should help disentangle demographic effects from the effects of physical boundaries, which can often align with sharp transitions in demographic composition. Block groups and tracts generally group blocks together following natural boundaries like existing neighborhood designations, highways, or bodies of water. The custom indicator for road/rail regions is designed to have the same effect.

In our analysis, we limited these infrastructure variables to those which could be computed from national data such as Census TIGER shapefiles, but researcher could also incorporate more specific geographic data, often available from municipalities, such as by using road speed or width data, or the locations of community centers and city facilities.

The model specification also includes interactions between respondent race and racial demographics, respondent party and party demographics, and respondent educational attainment and education demographics. Table S5 in the Supplementary Material contains detailed model specifications, including transformations and interactions of our covariates.

Our main coefficients of interest correspond to the three variables that measure the fraction of people in each block who belong to the same racial, partisan, and educational category as the respondent, respectively. We allow these coefficients to vary by the categories of each variable as well, to understand differences between groups. For example, the coefficient for the same race category variable can differ between white and minority respondents.

We fit a separate model to each city’s data. This decision is in part based on the fact that there are a sufficient number of respondents for each city, leading to relatively precise parameter estimates. Linking all three cities through a single hierarchical model is also possible, but fitting such a model to the entire data would substantially increase computational cost.

Empirical Findings

To interpret the fitted models, we use the method described earlier. We compute the posterior estimate of the percentage point change in the predicted probability that a respondent will include a census block in their neighborhood when increasing the value of the corresponding covariate by one unit, over a baseline probability of 50%, while holding other variables in the model constant. Figure 5 presents these posterior means and credible intervals (90% and 50%) for selected coefficients from the full model (see Section S5 of the Supplementary Material for the posterior summaries of all coefficients from the full and baseline models). Posterior summaries are plotted separately by city.

Figure 5.

Selected Full Model Coefficient Posteriors, Scaled to Show the Percentage Point Change in Probability of a Block’s Inclusion for a Baseline Probability of 50%

Note: Plotted are 90% and 50% credible intervals, with posterior medians displayed to the right of each interval. Section S5 of the Supplementary Material contains the full results table for the other variables specified the “Model Specification” section.

Holding other variables in the model constant, a white respondent is 6.1 to 16.9 percentage points more likely to include a census block composed entirely of white residents compared to one with no white residents. We cannot be confident that this preference for racial homogeneity occurs for neighborhoods drawn by minority (non-white) respondents, as the credible intervals for each city sample overlap with zero. Partisan similarity exerts analogous predictive power. Democrats are more likely to include Democratic neighbors in their neighborhoods and Republicans are more likely to include Republican neighbors, holding other variables in the model constant.Footnote ⁶

We do not observe consistent results for educational similarity. College educated and non-college educated respondents include areas with more college educated residents, although the credible intervals for some of the city samples overlap with zero and the medians vary considerably across cities. Different results across cities could be due to contextual differences between cities but could also be due to sampling noise. Additionally, it is difficult to determine with the data what specific characteristics of cities might produce differential results.

Size and population of local areas also influence inclusion probability. In the New York and Phoenix, respondents are more likely to include populous census blocks in their neighborhoods. This interval is less precise and smaller in magnitude in Miami. In all three city samples, larger census blocks are more likely to be included in neighborhoods.

The presence of a church in a census block is negatively associated with inclusion of the census block in a neighborhood in the New York and Phoenix samples, with no predictive effect in the Miami sample. Distance to a church is also negatively associated with inclusion in each of the samples, meaning that census blocks that are closer to churches are more likely to be included. These results may speak to respondents opting to include residential census blocks over ones with churches in them, but their neighborhoods still being shaped by proximity to churches.

In Table S6 in the Supplementary Material, we report all the estimates from the full model described earlier (see Table S7 in the Supplementary Material for the estimates from the baseline model). These include administrative variables such as roadways, census block groups, and census tracts. We find that these administrative definitions and physical characteristics, net of other factors in the model, influence whether people include areas in their neighborhoods. For example, respondents are more likely to include areas in their neighborhood that fall on the same side of major roadways as their residence. Similarly, they are more likely to include areas that fall in the same census tract. These estimates demonstrate how objective features of neighborhoods influence subjective definitions.

Out-of-Sample Predictive Ability

First, we examine the quality of model fit as measured by its out-of-sample predictive ability. There is significant heterogeneity in respondents’ neighborhoods, as reflected in the wide range of neighborhood areas and demographics shown in Figure 2. Neighborhoods range in size from less than 0.01 to over 100 square miles, and much of this variation in size is not captured by demographic variables. Any model will consequently struggle to make accurate predictions, especially for respondents not included in the data to which the model was fitted.

Despite these challenges, both the full and baseline models are more effective in predicting respondents’ neighborhoods than a naive approach based on circles centered around their residence locations. We measure predictive accuracy by first generating one hundred posterior predictions for each respondent’s neighborhood. This is accomplished by taking a random sample of parameter values from the posterior, and then sequentially sampling census block inclusions according to the model’s data generating process.

For each neighborhood prediction, we compute the precision and recall for the constituent census blocks, and then take their median values over predictions. Precision measures the fraction of the predicted neighborhood that is in the original neighborhood, while recall measures the fraction of the original neighborhood that is in the prediction. The baseline and full models have in-sample median precision of 0.32 and 0.34, respectively, and recall of 0.75 and 0.71. Out of sample, precision increases moderately to 0.38 and 0.48 for the baseline and full models, respectively. The out-of-sample recall falls to 0.69 and 0.64.

However, due to the contiguity requirement and the sequential nature of the neighborhood model, precision and recall in this context are driven largely by the size of the predicted neighborhood. As the neighborhood grows larger, the recall will increase at the cost of precision. In addition, by shifting the intercept of our model, we can grow or shrink the predicted neighborhood while maintaining the same discrimination with regards to predictive covariates.

Thus, to better contextualize this performance, we compare each posterior prediction (before averaging) to a circular neighborhood of the same radius as the prediction. We find this radius by taking the smallest circle centered on the respondent’s home which covers their drawn neighborhood. Using the same fixed-radius circle for comparison with all model predictions would not be appropriate given the wide variation in neighborhood sizes, but allowing the circular neighborhood to exactly match the modeled radius gives this naive approach a significant leg up—it can leverage all the information learned in the model about the neighborhood’s radius. We might therefore expect the baseline and full models to only minimally improve upon the circular neighborhoods, especially for out-of-sample predictions, where individual random effects have not been fit.

For the one hundred predictions for each respondent, we take the median of the difference in the F1 score, which is the harmonic mean of precision and recall, between the prediction and the matching circle. We also calculate this difference between the prediction and a census tract, the most common unit at which researchers measure local context (see Figure S8 in the Supplementary Material for comparison to ZIP Code Tabulation Areas). Figure 6 shows the results of this comparison, broken out by city. Both models outperform the circular-neighborhood approach by around 0.04 in-sample, and 0.03 out-of-sample on average. Importantly, both in and out of sample, only for a few respondents do the naive approaches meet or outperform the full model, as indicated by the bulk of each boxplot lying above the x-axis. And even in these cases, as shown above, the model is able to provide uncertainty quantification and coefficient estimates, which a naive approach cannot. Compared to census tracts, the predictive performance is similar, and only in New York City do we see consistent out-performance of the predictions compared to tracts.

Figure 6.

Posterior Median of the Difference in F1 Scores between a Neighborhood Predicted by the Model and a Circular Neighborhood of the Same Radius (Top) or a Census Tract (Bottom)

Note: The boxplot shows the variation in this median difference across the respondents included in the model fitting (left plot) and excluded from the model fitting (right plot). Positive values indicate the model outperforming the circular baseline, on average, for a particular respondent. The baseline model includes geographic information only while the full model also includes demographic information. Section S5 of the Supplementary Material contains the full results tables for the full and baseline models.

The substantial heterogeneity in neighborhood sizes makes accurate neighborhood predictions difficult in general. However, the model’s use of local and individual covariate information allows it to improve on purely distance-based measures, even when these are well calibrated by matching the radius of a circular neighborhood to that of the model-based prediction.

In-Sample Respondent-Level Prediction

Model predictions can be useful in-sample as well. Here, we demonstrate the predictive influence of race on census block inclusion probability using a single respondent in Miami. This voter is white, female, and is not registered to a major political party. Figure 7 maps the racial demographics surrounding the respondent’s residential address in the left panel (each census block shaded based on the percent white of its population), and the change in the posterior probability of inclusion for each census block comparing the full model to the baseline model in the right plot. This respondent lives in a mixed but majority white area (indicated by light orange color in the left plot) that is just to the north of areas comprised largely of minority residents (dark purple color). Her drawn neighborhood (represented by a black solid line) adheres sharply to this stark southern boundary. The posterior probability map shows how these majority non-white areas are less likely to be included when demographics are accounted for in the model (indicated by dark brown color in the right plot).

Figure 7.

The Left Plot Shows the Racial Demographics of Area Surrounding the Example Respondent

Note: The subjective neighborhood drawn by this respondent is indicated by the solid black line and each census block is shaded based on the percent white of its population. The right plot shows the difference in the posterior probability of a block being included in the respondent’s neighborhood between the full and baseline models. The baseline model includes geographic information only while the full model also includes demographic information. Blue areas are relatively more likely to be included under the full model, while orange areas are relatively less likely to be included.

Depending on one’s substantive questions of interest, other quantities may be of interest and can be directly computed from the fitted model though it may require additional causal and other assumptions. Examples include the probability of including one block, given that another block is or is not included; the change in an individual respondent’s posterior predictive neighborhood if their demographics were different; or how a change in the demographics of one block (say, by a new housing development) could influence the shape and size of a respondent’s neighborhood.

City Council Survey of New York City Residents

The survey was administered in two ways. First, similar to the previous survey, we contacted New York City residents via email, randomly drawing residents off of registered voter lists. Those who did not respond were sent a reminder email each week for 3 weeks. Of the 277,641 registered voters who were successfully contacted, we received 1,102 responses, for a response rate of 0.40%. Section S1 of the Supplementary Material contains more information on the sampling process.

The second method of survey administration was through targeted advertisements on Meta. From December 6, 2022 to February 21, 2023, we ran advertisements targeting New York City residents inviting them to draw their neighborhood on a map. Facebook users who clicked on the advertisement were led to the Qualtrics survey instrument. Based on statistics from Meta, 25,767 Facebook users clicked on our advertisements during this time period, of which 1,086 chose to take the survey.Footnote ⁷

In our analysis, we focus on the 627 respondents who drew maps consisting of more than one census block. Figure 8 shows the central tendencies of the population, area, proportion white and Black, and proportion Democratic and Republican for the drawn COI. These statistics are shown broken out by the email and Meta surveys, as well as the pooled sample. The median population of these drawn community of interests is 38,070 people, with the full distribution ranging from 319 to 455,384. The median area is 0.63 square miles (range: 0.01–12.41 square miles). The median percentage white is 56%, median percentage Black is 5%, and the median values for percent Democratic and Republican are 69% and 8% (out of registered voters). Figures S3 and S4 in the Supplementary Material show the breakdowns of map demographics by respondent race and partisan lean. The results show clear descriptive differences between whites and non-whites and between Democrats and Republicans in the racial and partisan demographics of their drawn COI.

Figure 8.

Descriptive Statistics for Respondent Communities of Interest

Determinants of Communities of Interest

Next, we fit the proposed model with the same specification as the one used for the analysis of the first survey. We examine the way in which respondent traits, aggregate characteristics, and their interactions influence the inclusion of different areas into their COI. Similar to the previous analysis, we first take a random sample of five hundred (approximately 80% of the sample) responses as our training data and fit the model to this training set. All coefficients reported below are on this sample, while predictive comparisons in “Quality of Model Fit” section conduct out-of-sample predictions on the 127 neighborhoods in the test set.

Figure 9 displays the coefficients of interest from the model fit to the city council data (see Table S8 in the Supplementary Material for the full results). Holding other factors constant, white respondents are 14.3 percentage points more likely to include a census block comprised entirely of white residents in their community of interest. Minority respondents are 7.9 percentage points more likely to include census blocks comprised entirely of minority residents. The analogous estimates from the New York City sample in the subjective neighborhoods survey were 6.1 percentage points for whites and −7.5 percentage points for minority respondents (but not statistically significant). Therefore, the preference for racial homophily is stronger when respondents define the areas that should be included in their city council district than when drawing subjective neighborhoods without explicit direction as to the political implications of these definitions.

Figure 9.

Selected Full Model Coefficient Posteriors, Scaled to Show the Percentage Point Change in Probability of a Block’s Inclusion for a Baseline Probability of 50%

Note: Plotted are 90% and 50% credible intervals, with posterior medians displayed to the right of each interval. Section S5 of the Supplementary Material contains the full results table.

We also find that Democratic respondents are 9.9 percentage points more likely to include Democratic census blocks, (slightly lower than that 11.1-percentage point estimate in the first survey), while Republicans are 28.1 percentage points more likely to include Republican census blocks (much higher than the 12.7-percentage point estimate in the subjective neighborhoods survey). The estimates for education are consistent in sign but smaller in magnitude as in the previous survey. Respondents, regardless of whether they graduated college or not, tend to include census blocks containing more residents who graduated college.

Looking at the other coefficients in the model, we find high levels of consistency between effect sizes and direction between the city council survey and the subjective neighborhood survey. For example, block population and block area are again positively associated with inclusion, while presence of a church and distance to the nearest church are both negatively associated with inclusion.

In sum, these results demonstrate that citizen conceptions of how they should be represented are shaped by local racial and partisan demographics, as well as by infrastructural and institutional characteristics of the places in which they live. Specifically, respondents are influenced by racial and partisan compositions of local areas when drawing subjective neighborhoods regardless of whether they are given specific definitions of neighborhoods. The magnitude of racial influence is particularly greater when drawing COI as they relate to legislative redistricting.

Quality of Model Fit

As before, we examine the quality of model fit using the city council survey. The top row of Figure 10 shows the distribution of the median difference in the F1 score between predicted COI and the matching circle, using the baseline and full model. The baseline model outperforms the circular-neighborhood approach by approximately 0.016 in-sample, and 0.049 out-of-sample on average. The full model outperforms the circular neighborhood by approximately 0.014 in-sample and 0.015 out-of-sample.

Figure 10.

Posterior Median of the Difference in F1 Scores between a Community of Interest Predicted by the Model Prediction and a Circular Neighborhood of the Same Radius (Top) and a Census Tract (Bottom)

Note: The boxplot shows the variation in this median difference across the respondents included in the model fitting (left plot) and excluded from the model fitting (right plot). Positive values indicate the model outperforming the circle (tract), on average, for a particular respondent. The baseline model includes geographic information only while the full model also includes demographic information.

The bottom row of the figure shows that compared to tracts, the model shows a much more notable improvement. The median difference in F1 scores between the baseline model and tracts is 0.27 in-sample and 0.30 out-of-sample. For the full model, the median difference is 0.25 in-sample and 0.29 out-of-sample. The performance advantage is much higher than that observed in the tract comparison from the first survey. This improvement in predictive performance suggests that drawn maps are easier to predict when respondents are provided with a more concrete prompt related to redistricting.

Building Consensus Neighborhoods

A major challenge for map drawers in redistricting city council boundaries is to incorporate many COI at the same time. As we have consistently found, different people living in the same location may define their local community or neighborhood differently. When it comes time to select a “community of interest” for redistricting purposes, these varying individual communities must be somehow aggregated. We can use individual predictions to explore options for this aggregation process, and to understand how our substantive findings on same-race preference affect the difficulty of building an aggregate or consensus neighborhood.

We begin by sampling a synthetic residential population for a particular census block. We generate a random race, party, homeownership status, and educational level for one hundred individuals according to the census-reported demographics for the block. Then for each synthetic resident, we estimate the posterior predictive distribution over census blocks by simulating 20 neighborhoods from the posterior predictive distribution of the city council model.Footnote ⁸ Since we are simulating neighborhoods for synthetic residents who did not take the survey, we draw new random effects for each resident. While all the synthetic residents live in the same census block, they differ in their covariates, and so their posterior predictive neighborhoods are different as well.

We can now aggregate the one hundred residents’ posterior predictive distributions by calculating, for each block, the fraction of the synthetic respondents who assign at least 50% posterior probability to that block. Blocks which belong, with high probability, to almost everyone’s predicted neighborhood will have high values, while blocks that generally belong to only one or two residents’ neighborhood will have low values. These values are plotted on a map in the left of Figure 11, for a block in a highly racially diverse area of the borough of Queens.

Figure 11.

On the Left, a Map Visualizing the Consensus Community of a Synthetic Residential Population of a Single Census Block, Which is Marked with a White Asterisk

Note: Darker blocks are those which are included in a higher proportion of synthetic resident’s predicted neighborhoods. On the right, the trade-off between the size of the community of interest and the degree of consensus is plotted for communities in two areas: one with high and one with low racial diversity.

Unsurprisingly, there is strong agreement for blocks close to the block where all the residents live, with the share of residents including a block falling with distance. This illustrates a fundamental trade-off in building consensus neighborhoods: all else being equal, a smaller neighborhood will have a higher level of consensus. We can visualize this trade-off directly by changing the threshold used to decide whether a block belongs to the consensus neighborhood in the left map of Figure 11. In other words, we consider modifying a minimum share of residents who include the block in their neighborhood. As we vary this threshold value (horizontal axis), the size of the consensus neighborhood changes as well.

This is visualized in the right plot of Figure 11 where the blue line represents the result for this highly racially diverse block in Queens. When we repeat this prediction exercise in an area of Brooklyn with low racial diversity, we obtain a different community consensus-size curve, which is represented by a yellow line in the plot. Specifically, in the low-diversity area, it is easier to build a consensus neighborhood: for any given neighborhood size, a higher fraction of block residents can agree on a neighborhood of that size. Conversely, for a given share of agreement, the neighborhoods in the low-diversity area are larger, on average.

These patterns reflect the individual-level findings from our fitted models: residents prefer racially homogeneous neighborhoods and communities. As a result, racially diverse areas will find it harder to agree on a common definition of a neighborhood.