2.1 The latent shrinkage position model

Network data typically take the form of an $n \times n$ adjacency matrix, $\mathbf{Y}$ , where $n$ is the number of nodes and entry $y_{i,j}$ denotes the relationship or edge between nodes $i$ and $j$ . Self-loops are not permitted and thus the diagonal elements of $\mathbf{Y}$ are zero. Here we consider binary edges but a variety of edge types can be considered.

Under the LPM (Hoff, et al., Reference Hoff, Raftery and Handcock2002), edges are assumed independent, conditional on the latent positions of the nodes. The sampling distribution is then

\begin{equation*} \mathbb{P}(\mathbf{Y}\mid \alpha , \mathbf{Z}) = \prod _{i \neq j} \mathbb{P}(y_{i,j}\mid \alpha , \mathbf{z}_i,\mathbf{z}_j) \end{equation*}

where $\mathbf{Z}$ is the matrix of latent positions, with $\mathbf{z}_i$ denoting the latent position of node $i$ ; $\alpha$ is a global parameter that captures the overall connectivity level in the network. Denoting by $q_{i,j}$ the probability of an edge between nodes $i$ and $j$ , i.e. $\mathbb{P}(y_{i,j} = 1\mid \alpha , \mathbf{z}_i,\mathbf{z}_j)$ , a logistic regression model formulation is used where the log odds of an edge between nodes $i$ and $j$ depends on the Euclidean distance between their respective positions $\mathbf{z}_i$ and $\mathbf{z}_j$ in the latent space,

(1) \begin{equation} \log \frac {q_{i,j}}{1-q_{i,j}} = \alpha - \Vert \mathbf{z}_i-\mathbf{z}_j \Vert ^2_2. \end{equation}

As in Gollini and Murphy (Reference Gollini and Murphy2016) and D’Angelo et al. (Reference D’Angelo, Murphy and Alfò2019), in (1) the distance is taken to be the squared Euclidean distance, giving higher weight to the latent space in the model than that induced by a linear Euclidean distance.

Gwee et al. (Reference Gwee, Gormley and Fop2024a) propose the LSPM, a Bayesian nonparametric extension of the LPM that allows automatic inference on the number of the infinite latent space dimensions that are necessary to fully describe the network i.e., the number that are effective since they have non-negligible latent position variance. Under the LSPM, the latent positions are assumed to have a zero-centered Gaussian distribution with diagonal precision matrix $\mathbf{\Omega }$ , whose entries $\omega _{\ell }$ denote the precision of the latent positions in dimension $\ell$ , for $\ell =1, \ldots , \infty$ . Spherical Gaussian distributions are assumed, as is standard in latent position models, since the likelihood is invariant to rotations of the latent space (Handcock, et al., Reference Handcock, Raftery and Tantrum2007). The LSPM employs a multiplicative truncated gamma process (MTGP) prior on the precision parameters: the latent dimension $h$ has an associated shrinkage strength parameter $\delta _h$ , where the cumulative product of $\delta _1$ to $\delta _{\ell }$ gives the precision $\omega _{\ell }$ . An unconstrained gamma prior is assumed for $\delta _1$ , while a truncated gamma distribution is assumed for the remaining dimensions to ensure shrinkage. Specifically, for $i = 1, \ldots , n$

\begin{equation*} \begin{aligned} \mathbf{z}_{i} \sim \text{MVN}(\mathbf{0}, \mathbf{\Omega }^{-1}) \end{aligned} \end{equation*}

\begin{equation*} \begin{aligned} \mathbf{\Omega } = \begin{bmatrix} \omega _{1}^{-1} & \dots & 0 \\ \vdots & \ddots & \\ 0 & & \omega _{\infty }^{-1} \end{bmatrix} \qquad \omega _{\ell } = \prod _{h=1}^{\ell } \delta _{h} \mbox{ for } \ell = 1, \ldots , \infty \\ \end{aligned} \end{equation*}

\begin{equation*} \begin{aligned} \delta _1 \sim \text{Gam}(a_1, b_1=1) \qquad \delta _{h} \sim \text{Gam}^{\text{T}}(a_2, b_2=1, t_2 = 1) \mbox{ for }h \gt 1. \end{aligned} \end{equation*}

Here $a_1$ and $b_1$ are the gamma prior’s shape and rate parameters on the first dimension’s shrinkage parameter, while $a_2$ is the shape parameter, $b_2$ is the rate parameter, and $t_2$ is the left truncation point (here set to 1) of the truncated gamma prior for dimensions $h \gt 1$ . This MTGP prior results in an increasing precision and therefore a shrinking variance of the positions in the higher dimensions of the latent space. Further details regarding these hyperparameters and their choices are given in Appendix A. Under this MTGP prior, the LSPM is nonparametric with infinitely many dimensions, where unnecessary higher dimensions’ variances are increasingly shrunk towards zero. Dimensions that have variance very close to zero will then have little meaningful information encoded in them as the distances between nodes will be close to zero. Thus, under the LSPM, the effective dimensions are those in which the variance is non-negligible (Gwee, et al., Reference Gwee, Gormley and Fop2024a).

2.2 The latent shrinkage position cluster model

To infer the unknown number of clusters of nodes, we propose the novel LSPCM which results from fusing the LSPM (Gwee, et al., Reference Gwee, Gormley and Fop2024a), the latent position cluster model (LPCM) of Handcock, et al. (Reference Handcock, Raftery and Tantrum2007) and the sparse finite mixture model framework of Malsiner-Walli et al. (Reference Malsiner-Walli, Frühwirth-Schnatter and Grün2014, Reference Malsiner-Walli, Frühwirth-Schnatter and Grün2017). Like the LPCM, the proposed LSPCM assumes that latent positions arise from a finite mixture of $G$ spherical multivariate normal distributions; however, unlike the LPCM, the LSPCM assumes an infinite-dimensional latent space as in the LSPM. Further, the LSPCM employs a sparsity-inducing Dirichlet prior on the mixture weights, facilitating automatic inference on the number of clusters.

Specifically, the LSPCM assumes, for $i = 1, \ldots , n$ ,

(2) \begin{align} \mathbf{z}_{i} \sim \sum _{g=1}^G \tau _g \text{MVN} (\boldsymbol{\mu }_g,\psi _g^{-1}\mathbf{\Omega }^{-1}) \end{align}

\begin{align*} \boldsymbol{\mu }_g |\mathbf{\Omega } \sim \text{MVN}(\mathbf{0}, \xi \mathbf{\Omega }^{-1}) \qquad \psi _g \sim \text{Gam}(a_{\psi }, b_{\psi }) \end{align*}

\begin{align*} \mathbf{\Omega } = \begin{bmatrix} \omega _{1}^{-1} & \dots & 0 \\ \vdots & \ddots & \\ 0 & & \omega _{\infty }^{-1} \end{bmatrix} \qquad \omega _{\ell } = \prod _{h=1}^{\ell } \delta _{h} \mbox{ for } \ell = 1, \ldots , \infty \end{align*}

\begin{align*} \delta _{1} \sim \text{Gam}(a_{1}, b_{1} = 1) \quad \delta _{h} \sim \text{Gam}^{\text{T}}(a_{2}, b_{2} = 1, t_2 = 1) \mbox{ for }h \gt 1. \end{align*}

Here $\tau _g$ denotes the probability that a node belongs to the $g$ -th component, so that $\tau _g \geq 0$ $(g = 1,\ldots , G)$ and $\sum _{g=1}^G \tau _g =1$ . For the mean latent position $\boldsymbol{\mu }_g$ of the $g$ -th component, a multivariate Gaussian prior with zero mean and a scaled version of the same precision matrix as the latent positions with its MTGP prior is assumed; this ensures the latent positions will all shrink towards zero at higher dimensions. Consequently, the component means are increasingly shrunk towards zero, resulting in higher dimensions characterized by increasingly overlapping mixture components, and, as a result, becoming less informative for cluster separation. Further, as in Ryan et al. (Reference Ryan and JasonWyse2017), the prior covariance on $\boldsymbol{\mu }_g$ is inflated by a scaling factor $\xi$ ; values of $\xi \gt 1$ result in cluster means that are more dispersed than cluster members. While a range of values of $\xi = \{1, 9, 100\}$ were considered, posterior predictive checks suggested that, similar to Ryan et al. (Reference Ryan and JasonWyse2017), $\xi = 9$ showed good model fit by encouraging both separate yet connected clusters. Additionally, a cluster-specific factor $\psi _g$ scales the precision matrix of the latent positions providing the flexibility to model clusters of different volumes.

While the LPCM assumes a finite mixture of spherical multivariate normal distributions and casts the problem of inferring the number of clusters as one of model selection, the LSPCM considers an overfitted (or sparse) finite mixture model which provides automatic inference on the number of clusters. Here, following Malsiner-Walli et al. (Reference Malsiner-Walli, Frühwirth-Schnatter and Grün2014); Fruhwirth-Schnatter, et al. (Reference Fruhwirth-Schnatter, Celeux and Robert2019), a symmetric Dirichlet prior is placed on the mixture weights $\boldsymbol{\tau }$ with a gamma prior placed on the Dirichlet’s influential sparsity-inducing hyperparameter $\nu$ , i.e.,

\begin{equation*}\boldsymbol{\tau } = (\tau _1, \ldots , \tau _G) \sim \text{Dir}(\nu , \ldots , \nu ) \qquad \nu \sim \text{Gam}(a_{\nu },G b_{\nu }).\end{equation*}

Under such an overfitted mixture, a distinction is made between the number of mixture components $G$ , the number of non-empty mixture components $G_+$ , and the number of clusters in the data $G^*$ . A large number $G$ of initial components is specified and under a gamma prior that favors small values of $\nu$ , unnecessary components are emptied out during the inferential process. The number of non-empty components $G_+$ then serves as an estimate of the number of clusters $G^*$ . Thus, the LSPCM allows for automatic, simultaneous inference of the number of clusters and of the number of effective latent space dimensions, while requiring only a single model to be fit to the network data.

While the LSPM adopts a nonparametric approach, an overfitted sparse finite mixture approach for the clustering aspect of LSPCM is employed here as the sampling algorithm is simpler than that required to fit an infinite mixture model. Further, as detailed in Frühwirth-Schnatter and Malsiner-Walli (Reference Frühwirth-Schnatter and Malsiner-Walli2018), under certain hyperparameter specifications, similar inference with respect to the number of clusters is obtained under the sparse finite and nonparametric approaches. Additionally, Frühwirth-Schnatter and Malsiner-Walli (Reference Frühwirth-Schnatter and Malsiner-Walli2018) and Murphy, et al. (Reference Murphy, Viroli and Gormley2020) highlight that overfitted mixtures serve well in situations where the data arise from a moderate number of clusters, even with growing sample size, whereas infinite mixtures are suited to cases where the number of clusters increases with sample size. As the networks considered here are anticipated to arise from a moderate number of clusters, an overfitted approach is preferred and taken.

To facilitate inference, we introduce $\boldsymbol{C} = (\boldsymbol{c}_1, \ldots , \boldsymbol{c}_n)$ which contains the latent component membership vectors $\boldsymbol{c}_i = (c_{i1}, \ldots , c_{iG})$ , for $i = 1, \ldots , n$ , where $c_{ig} = 1$ if node $i$ belongs to component $g$ and $c_{ig} = 0$ otherwise. Thus, $\boldsymbol{c}_i \sim \text{MultiNom}(1, \boldsymbol{\tau })$ . A summarizing graphical model representation of the LSPCM is shown in Figure 1.

Figure 1. The graphical model representation of the LSPCM.

3.1 An adaptive Metropolis-within-Gibbs sampler

Markov chain Monte Carlo (MCMC) is employed to draw samples from the joint posterior distribution. After defining necessary notation in Appendix B, derivations of the full conditional distributions of the latent positions, cluster membership vectors and model parameters are given in Appendix C. As the MTGP prior is nonparametric and assumes an infinite number of latent dimensions, in practice setting an initial finite truncation level, $p_0$ , on the number of dimensions fitted is required. Following Bhattacharya and Dunson (Reference Bhattacharya and Dunson2011), an adaptive Metropolis-within-Gibbs sampler is employed where the truncation dimension $p \leq p_0$ is dynamically shrunk or augmented as the sampler proceeds, with inferential validity ensured as detailed in Bhattacharya and Dunson (Reference Bhattacharya and Dunson2011). The number of effective dimensions, $\hat {p}$ , can then be subsequently inferred (see Section 3.2.3).

Denoting the current iteration by $s$ for $s = 1, \dots , S$ (but for clarity $s$ is omitted where it is unnecessary), a single pass of the sampler proceeds as follows:

1. 1. Sample the component mean $\boldsymbol{\mu }_g$ for $g = 1, \ldots , G$ from $\text{MVN}_p\left (\frac { \sum _{i=1}^n c_{ig} \mathbf{z}_{i}}{ \sum _{i=1}^n c_{ig} + \xi ^{-1} } ,\right.$ $\left.\left [ \boldsymbol{\Omega } \left (\sum _{i=1}^n c_{ig} + \xi ^{-1} \right ) \right ]^{-1} \right ).$

2. 2. Sample $\check {\nu }$ from a Gam $(\sigma _{\nu }, \frac {\sigma _{\nu }}{\nu ^{(s)}})$ where $\sigma _{\nu }$ is a step size factor. Accept $\check {\nu }$ as $\nu ^{(s+1)}$ with probability $ \frac {p(\nu = \check {\nu }\mid -)}{p(\nu = \nu ^{(s)} \mid -)} \frac {f(\check {\nu }; a_{\nu }, b_{\nu })}{f(\nu ^{(s)}; a_{\nu }, b_{\nu })}$ , where $p(\nu \mid -)$ is the full conditional of $\nu$ and $f(\cdot ; a_{\nu }, b_{\nu })$ is the gamma density. Otherwise, set $\nu ^{(s+1)} =\nu ^{(s)}$ .

3. 3. Sample the mixing weights $\boldsymbol{\tau }$ from Dir $(\sum _{i=1}^n c_{i1} + \nu , \ldots , \sum _{i=1}^n c_{iG} + \nu )$ .

4. 4. Sample the cluster-specific precision scaling parameter $\psi _g$ for $g = 1, \ldots , G$ from $ \text{Gam} \left ( a_{\psi }+\frac {p\sum _{i=1}^n c_{ig}}{2}, \quad \quad b_{\psi }+ \frac {1}{2} \sum _{i=1}^n (\mathbf{z}_{i} - \boldsymbol{\mu }_g)^T \mathbf{\Omega } (\mathbf{z}_{i} - \boldsymbol{\mu }_g)c_{ig} \right )$ .

5. 5. Sample the latent memberships $\boldsymbol{c}_i$ for each node $i = 1, \ldots , n$ from a Multinomial with $G$ categories by drawing the $g$ -th category with probability $ \frac {\tau _g \phi _p(\mathbf{z}_i ; \boldsymbol{\mu }_g, \psi _g^{-1}\boldsymbol{\Omega }^{-1})}{\sum _{r=1}^G \tau _r \phi _p(\mathbf{z}_i ; \boldsymbol{\mu }_r, \psi _g^{-1}\boldsymbol{\Omega }^{-1}) }$ where $\phi _p(\mathbf{z}_i; \boldsymbol{\mu }, \psi _g^{-1}\boldsymbol{\Omega }^{-1})$ is the $p$ -dimensional multivariate normal density.

6. 6. Sample $\check {\mathbf{Z}}$ from a MVN $_p(\mathbf{Z}^{(s)}, k\psi _g^{-1}\mathbf{\Omega }^{-1(s)})$ proposal distribution where $k$ is a step size factor. Accept $\check {\mathbf{Z}}$ as $\mathbf{Z}^{(s+1)}$ with probability $\frac {\mathbb{P}(\mathbf{Y}\mid \alpha ^{(s)}, \check {\mathbf{Z}})}{\mathbb{P}(\mathbf{Y}\mid \ \alpha ^{(s)}, \mathbf{Z}^{(s)})} \frac {\phi _p(\mathbf{z}_i ; \check {\mathbf{Z}}, k\psi _g^{-1}\mathbf{\Omega }^{-1(s)})}{\phi _p(\mathbf{z}_i ;\mathbf{Z}^{(s)}, k\psi _g^{-1}\mathbf{\Omega }^{-1(s)})}$ , otherwise set $\mathbf{Z}^{(s+1)} = \mathbf{Z}^{(s)}$ .

7. 7. Sample $\check {\alpha }$ from an informed Gaussian proposal distribution $N(\bar {\mu }_{\alpha }, \bar {\sigma }^2_{\alpha })$ (Gormley and Murphy, Reference Gormley and Murphy2010) where

   \begin{equation*} \begin{aligned} \bar {\mu }_{\alpha } = \alpha ^{(s)} + \bar {\sigma }^2_{\alpha }\times \left [ \sum _{i \neq j}y_{i,j} - \sum _{i \neq j} \hspace {2pt} \frac {\exp (\alpha ^{(s)} - \Vert \mathbf{z}_i-\mathbf{z}_j \Vert ^2_2)}{1+\exp (\alpha ^{(s)} - \Vert \mathbf{z}_i-\mathbf{z}_j \Vert ^2_2)} + \frac {1}{\sigma ^2_{\alpha }}(\mu _{\alpha }-\alpha ^{(s)}) \right ], \end{aligned} \end{equation*}
   and
   \begin{equation*} \begin{aligned} \bar {\sigma }^2_{\alpha } = \left \{\sum _{i \neq j} \hspace {2pt} \frac {\exp (\alpha ^{(s)} - \Vert \mathbf{z}_i-\mathbf{z}_j \Vert ^2_2)}{\left [1+\exp (\alpha ^{(s)} - \Vert \mathbf{z}_i-\mathbf{z}_j \Vert ^2_2)\right ]^2} + \frac {1}{\sigma ^2_{\alpha }}\right \}^{-1}. \end{aligned} \end{equation*}
   Then, accept $\check {\alpha }$ as $\alpha ^{(s+1)}$ following the Metropolis-Hastings acceptance ratio.

8. 8. Sample $\delta _{1}$ from

   \begin{align*} \text{Gam} \left ( \frac { \left ( n+G\right ) p}{2} + a_1, \right . \frac {1}{2} \sum _{i=1}^{n} \sum _{g=1}^{G} (\mathbf{z}_{i} - \boldsymbol{\mu }_g)^T \psi _g^{-1} \left (\prod _{m=2}^{\ell } \delta _{m}\right )\mathbf{I}_p (\mathbf{z}_{i} - \boldsymbol{\mu }_g) c_{ig} + &\\ \left . \qquad \qquad \qquad \qquad \qquad \qquad \quad \frac {1}{2}\sum _{g=1}^{G}\boldsymbol{\mu }_g^T (\xi ^{-1}) \left ( \prod _{m=2}^{\ell } \delta _{m}\right )\mathbf{I}_p \boldsymbol{\mu }_g +b_1 \right ).& \end{align*}

9. 9. Sample $\delta _{h}^{(s+1)}$ for $h=2, \ldots , p$ from

   \begin{align*} \text{Gam}^{\text{T}} \left ( \frac { \left ( n+G\right )(p-h+1)}{2} + a_2, \right . \frac {1}{2} \sum _{i=1}^{n} (\mathbf{z}_{i} - \boldsymbol{\mu }_g)^T \psi _g^{-1} \left (\prod _{m=1, m \neq h}^{\ell } \delta _{m}^{(s^*)}\right )\mathbf{I}_p (\mathbf{z}_{i} - \boldsymbol{\mu }_g) c_{ig} + &\\ \left . \qquad \qquad \qquad \qquad \qquad \qquad \quad \frac {1}{2}\sum _{g=1}^{G}\boldsymbol{\mu }_g^T (\xi ^{-1}) \left (\prod _{m=1, m \neq h}^{\ell } \delta _{m}^{(s^*)}\right )\mathbf{I}_p \boldsymbol{\mu }_g + b_2, \quad 1 \right )& \end{align*}
   where $s^* = s + 1$ for $m \lt h$ and $s^* = s$ for $m\gt h$ .

10. 10. Calculate $\omega _{\ell }$ by taking the cumulative product of $\delta _1$ to $\delta _{\ell }$ .

11. 11. At iteration $s$ , adapt the truncation dimension $p^{(s)}$ with probability $\exp (-\kappa _0 - \kappa _1s)$ as detailed in Section 3.2.2.

3.2 Practical implementation details

When implementing the adaptive Metropolis-within-Gibbs sampler, practical details such as initialization of latent variables and parameters, adapting the truncation dimension, and post-processing of the MCMC chain require attention.

4.1 Scenario 1: small networks

Networks with $n=50$ are generated with the true number of latent dimensions $p^*=2$ and true number of clusters $G^*=3$ , with shrinkage strength $\boldsymbol{\delta }=(1, 1.05)$ , $\psi _1=\psi _2=\psi _3=1$ , and cluster mean positions $\boldsymbol{\mu } =\{(0,0), (-4,0), (-4,4)\}$ giving well-separated clusters. Across the 30 simulated networks, the smallest cluster contained 6 nodes while the largest cluster contained 31, and fixing $\alpha = 6$ gave rise to network densities between 27% and 38%.

Figure 2a shows that the posterior modal number of clusters $G_m = 3$ and Figure 2b shows that the modal effective number of dimensions is $\hat {p}_m=2$ . Overall, the adjusted Rand index (ARI, Hubert and Arabie, Reference Hubert and Arabie1985), which measures agreement between the inferred cluster memberships and the true cluster labels, shows good correspondence, with mean value of 0.88 (standard deviation (sd) = 0.11). Procrustes correlations (PC, Peres-Neto and Jackson, Reference Peres-Neto and Jackson2001) between inferred and true latent positions also show that the latent positions are accurately estimated, with average value of $ 0.97$ (sd = 0.03). Additional visualization of the posterior distributions on the variances, shrinkage strengths, and the parameter $\alpha$ are available in Appendix D.

Figure 2. (a) Posterior distribution of the number of active clusters $G_+$ and (b) the distribution of the effective latent space dimension $\hat {p}$ across 30 small simulated networks. The true number of clusters and latent dimensions are highlighted in red.

Additionally, in this scenario sensitivity to the setting of $\epsilon _1$ was explored by also considering $\epsilon _1 = 0.9$ when deciding to add a truncation dimension but with the lower value of 0.8 employed when inferring the effective number of dimensions; there was no change in the modal effective dimension inferred, with only a small change in the posterior probability of $\hat {p} = 3$ to 0.9225.

For comparison, the LPCM is fitted on the same simulated data using the R package latentnet (Krivitsky and Handcock, Reference Krivitsky and Handcock2008). Upon fitting 25 LPCMs with different combinations of $p = \{1, \ldots , 5\}$ and $G = \{1, \ldots , 5\}$ to each of the 30 simulated networks, the BIC suggested 2 dimensions and 3 clusters for all of them, with average ARI and PC values between the LPCM’s posterior modal cluster labels and the true labels of $0.87 \,(\mbox{sd} =0.12)$ and $0.96 \,(\mbox{sd} = 0.04)$ , respectively. In terms of computational cost, fitting the LSPCM on a computer with an i9-13900H CPU and 32 GB RAM took on average 18.5 minutes. The LPCM with the correct $p$ and $G$ took 5.55 minutes on average to run for the same number of iterations. Across the various simulated networks, the total time taken to fit the 25 LPCMs with different combinations of $p$ and $G$ was comparable to or longer than the time taken to fit the LSPCM. However, notably, when fitting the LPCMs no quantification of the uncertainty in $p$ and $G$ is provided.

4.2 Scenario 2: moderately sized networks

Networks with $n=200$ were generated with $p^*=3$ latent dimensions and $\boldsymbol{\delta }=(1,1.1,1.05)$ . The true number of clusters $G^*=7$ with $\boldsymbol{\mu } =\{(\!-5,0,0), (\!-5,5,0),(0,-5,5),(0,0,-5), (2,0,2),$ $(\!-2,2,-2),(0,-2,0)\}$ and $\psi _g= 1$ for $g = 1, \ldots , 7$ . The 7 clusters have different degrees of separation across the dimensions. Among the 30 simulated networks, the smallest cluster had 4 nodes while the largest had 54. The parameter $\alpha =20$ , which resulted in networks with density varying between 23% and 31%.

Figure 3a shows that the posterior modal number of clusters is correctly inferred with $G_m = 7$ and Figure 3b suggests the modal effective number of dimensions is also correctly inferred with $\hat {p}_m=3$ .

The ARI values indicate very accurate and robust inference of cluster membership with mean value of 0.941 (sd = 0.024). The average PC value was 0.994 (sd = 0.002). The PC values are calculated using only dimensions up to and including the modal effective number of dimensions; in cases where $\hat {p}_m \lt p^*$ , the higher dimensions are not included in the comparison resulting in lower PC values. Additional visualization of the posterior distributions of the variances, shrinkage strengths, and the parameter $\alpha$ for this setting are available in Appendix D. In scenarios with larger $p^*$ , underestimation of the true number of dimensions often occurred as empirically with only $n=200$ and $G^*=7$ the relative variance in later dimensions was intuitively small.

Figure 3. (a) Posterior distribution of the number of active clusters $G_+$ and (b) the distribution of the effective latent space dimension $\hat {p}$ across 30 moderately sized simulated networks. The true number of clusters and latent dimensions are highlighted in red.

Upon fitting the LPCM with 3 dimensions and 7 clusters the average ARI and PC values between the LPCM’s posterior modal cluster labels and the true labels were $0.84 \,(0.09)$ and $0.98 \,(0.01)$ , respectively (standard deviation in brackets). The average time for the completion of one MCMC chain for the LSPCM was $149$ minutes. However, fitting the LPCM for multiple combinations of $p$ and $G$ would incur a considerably larger computational cost. In fact, solely for $p = 3$ and $G = 7$ it took $142$ minutes to run the MCMC via latentnet for the same number of iterations, and again no uncertainty quantification is provided.

4.3 Scenario 3: networks with clusters of different volumes

Networks are generated with the same settings as in Section 4.1 except with clusters of different volumes. Two settings are considered: clusters of slightly different volumes where $\psi _g$ values are set as $(\psi _1, \psi _2, \psi _3) = (\frac {4}{5}, 1, \frac {5}{4})$ and clusters of highly different volumes with $(\psi _1, \psi _2, \psi _3) = (\frac {1}{5}, 1, 5)$ . For comparison purposes, the networks simulated in Section 4.1 will be considered as networks with clusters of the same volume as $\psi _g = 1 \forall g$ . For the setting with slightly different volumes, network densities varied between 26% and 38% while in the highly different volumes setting, networks had densities between 22% and 38%.

Figure 4 shows that, for clusters of slightly different volumes, the number of clusters and effective dimensions were correctly inferred, but with lower certainty than shown in Figure 2 for networks with clusters of the same volume. For the highly different volumes setting, Figure 5a shows the posterior modal number of clusters $G_m=4$ which underestimates the true number of clusters, while 5b shows the effective number of dimensions is correctly inferred but again with less certainty than in the setting with clusters of the same volume. For networks with clusters of highly different volumes, intuitively the ARI values indicate poor clustering performance with mean value of 0.617 (sd = 0.208); for the slightly different volumes setting the average ARI was 0.882 (sd = 0.105). Similarly, the average PC values were 0.899 (sd = 0.091) and 0.973 (sd = 0.032) for the highly and slightly different volumes settings, respectively.

Figure 4. (a) Posterior distribution of the number of active clusters $G_+$ and (b) the distribution of the effective latent space dimension $\hat {p}$ across 30 small simulated networks with clusters of slightly different volumes. The true number of clusters and dimensions are highlighted in red.

Figure 5. (a) Posterior distribution of the number of active clusters $G_+$ and (b) the distribution of the effective latent space dimension $\hat {p}$ across 30 small simulated networks with clusters of highly different volumes. The true number of clusters and dimensions are highlighted in red.

Figure 6a highlights the intuitive difficulty in clustering nodes in networks with clusters of highly different volumes. The nodes from the cluster with the largest volume (indicated by black squares in Figure 6a) are far apart in the latent space, and many lie closer to nodes from another cluster e.g., to nodes from the medium volume cluster indicated by red circles. The result, as illustrated in Figure 6b, is that the LPSCM intuitively fits more clusters with smaller volumes than the truth.

Figure 6. One of the 30 simulated networks when clusters have highly different volumes with (a) the true positions and cluster labels and (b) the LSPCM posterior mean positions conditioned on the modal effective number of dimensions ( $\hat {p}_{m}=2$ ) and the estimated cluster labels.

5.1 Football players network

This binary network consists of directed edges indicating the presence of Twitter mentions from one English Premier League football player’s Twitter account to another. The data were adapted from those provided in Greene and Cunningham (Reference Greene and Cunningham2013) by considering only the top 3 clubs that have the most player Twitter accounts: 55 players playing for 3 different Premier League clubs are considered with 15 players from Stoke City football club (Stoke), 23 players from Tottenham Hotspur (Spurs), and 17 players from West Bromwich Albion (West Brom). In total, there were 497 mentions between players, giving a network density of 16.73%. To fit the LSPCM to these data, 10 MCMC chains are run, each for 1,000,000 iterations with a burn-in period of 100,000 and thinned every 1,000th. The average time for the completion of one MCMC chain was 30.6 minutes. With 10 MCMC chains, the Gelman–Rubin convergence criterion (Gelman et al., Reference Gelman, Carlin, Stern, Dunson, Vehtari and Rubin2014), computed conditional on the modal effective number of dimensions, was satisfied with $1.0 \leq \hat {R} \lt 1.1$ for the $\alpha$ , $\nu$ , and $\psi$ parameters, and with $\hat {R}=1.3$ and $\hat {R}=1.2$ for $\delta _1$ and $\delta _2$ respectively. Trace plots are provided in Appendix E.

Figure 7 illustrates that under the LSPCM the posterior modal number of clusters $G_m =$ 1 (1, 4) and the modal effective number of dimensions is $\hat {p}_m=2$ (1, 2), with 95% empirical intervals in brackets. There is uncertainty in $G_+$ as demonstrated by the wide posterior interval and notable support for $G_+ = 3$ , but there is strong certainty on the number of effective dimensions $\hat {p}$ .

Figure 7. For the football players network, (a) the posterior distribution of the number of non-empty components $G_+$ , and (b) the distribution of the number of effective latent space dimensions $\hat {p}$ .

Despite $G_m =1$ , maximizing PEAR across the 10 chains resulted in an estimate of 3 clusters, with an ARI of 0.94 between this final clustering and the players’ clubs. Figures 8a and 8b show the Fruchterman-Reingold layouts (Kolaczyk and Csárdi, Reference Kolaczyk and Csárdi2020) of the network with the players’ clubs and inferred clusters detailed respectively. Only one player was clustered differently to the other players in his club: the West Brom player Romelu Lukaku who was originally from Chelsea football club and was on a season-long loan deal with West Brom. Lukaku mentioned and was mentioned by only one player, the Spurs player Jan Vertonghen and intuitively the LSPCM has clustered them together. Figure 8c shows the posterior mean latent positions colored by the cluster labels that maximizes the PEAR across the 10 chains. Cluster 1 (which contains all of the Stoke players only) and cluster 2 (all players from Spurs, and Romelu Lukaku) are separate from each other on the first dimension, while in the second dimension cluster 3 (which captures West Brom players except Romelu Lukaku) is separated from clusters 1 and 2, indicating both effective latent dimensions are necessary for representation of the nodes in the three clusters.

Figure 8. Football players network, (a) the Fruchterman-Reingold layout with players colored by club, (b) the Fruchterman-Reingold layout with players colored by inferred cluster label and (c) posterior mean latent positions on the $\hat {p}_m = 2$ effective latent dimensions colored by inferred cluster label.

Through the posterior similarity matrix, Figure 9 illustrates the uncertainty in the football players’ cluster labels where the color intensity indicates the probability of a player being clustered together with another player. Cluster 2 (predominantly players from West Brom) has the strongest certainty in its cluster labels, while there is some uncertainty in the membership of players in clusters 1 and 3.

Figure 9. Heat map of the posterior similarity matrix of the cluster labels inferred from the football network, ordered by the cluster labels.

Upon fitting 16 LPCMs from all combinations of $p = \{1, \ldots , 4\}$ and $G = \{1, \ldots , 4\}$ , the BIC suggests 3 clusters and 2 dimensions as optimal. The ARI between the cluster labels of the optimal LPCM and LSPCM is 1, indicating that both approaches provide the same cluster allocations. The average Procrustes correlation between the posterior mean latent positions under the optimal LPCM and the LSPCM was 0.973 (standard deviation of 0.005) across the 10 LSPCM chains. Additional results regarding the posterior distributions of variance and shrinkage strength parameters can be found in the Appendix E.

5.2 Irish politicians network

The LSPCM is used to analyze a Twitter network between 348 Irish politicians from 2012. The network consists of binary directed edges indicating if one politician follows another (Greene and Cunningham, Reference Greene and Cunningham2013). Each of the politicians is affiliated with one of the seven Irish political parties: 49 are affiliated with Fianna Fáil (FF), 143 with Fine Gael (FG), 7 with the Green Party (Green), 79 with the Labour Party (Lab), 31 with Sinn Féin (SF), 8 with the United Left Alliance (ULA) and 31 are Independent (Ind). There are 16,856 directed relationships between the politicians, giving a network density of 13.96. For inference, 10 MCMC chains are run, each for 2,000,000 iterations with a burn-in period of 100,000 and thinned every 4,500th sample. Here, the setting of $G=20$ and $b_{\nu }=5$ resulted in very many sparsely populated clusters which provided little clustering insight. Motivated by the context, $G=10$ and $b_{\nu }=1000$ were instead used resulting in fewer, more populated clusters the results of which are presented here; results under $G=20$ and $b_{\nu }=5$ , and $G=10$ and $b_{\nu }=100$ are provided in Appendix E for the interested reader. The average time for the completion of one MCMC chain was 11.71 hours. With 10 MCMC chains, the Gelman–Rubin convergence criterion (Gelman et al., Reference Gelman, Carlin, Stern, Dunson, Vehtari and Rubin2014), computed conditional on the modal effective number of dimensions, was satisfied with $1.0 \leq \hat {R} \lt 1.1$ for the $\alpha$ , $\nu$ , and $\psi _g$ parameters, and $1.0 \leq \hat {R} \lt 1.6$ for $\boldsymbol{\delta }$ parameters. Trace plots are provided in Appendix E.

Figure 10 shows that, under the LSCPM, the posterior modal number of clusters is 6 (5, 7) and the modal number of effective dimensions is 4 (3, 4), with the 95% empirical intervals reported in brackets. Upon fitting multiple LPCMs across all combinations of $p = \{2, \ldots , 8\}$ and $G = \{2, \ldots , 8\}$ , the BIC suggests 6 clusters and 6 dimensions. The ARI between the cluster labels of the optimal LPCM and LSPCM is 0.55, indicating different clustering solutions under the 2 approaches. The mean Procrustes correlation between the posterior mean positions under the LPCM and LSPCM was 0.71 (standard deviation of 0.01) across the 10 LSPCM chains. Additional results regarding the posterior distributions of variance and shrinkage strength parameters can be found in Appendix E.

Figure 10. For the Irish politicians network: (a) the posterior distribution of the number of non-empty components $G_+$ , and (b) the distribution of the effective latent space dimension $\hat {p}$ .

Despite $G_m =6$ , maximizing the PEAR across the 10 chains resulted in 5 clusters and the ARI between the inferred cluster labels and the politicians’ political affiliations is 0.89. Table 1 presents the cross-tabulation of the political party membership and the LSPCM cluster labels. Additionally, Figure 11 shows the Fruchterman-Reingold layout of the network with nodes colored by political party affiliation and LSPCM inferred cluster label. The inferred clustering structure captures the composition and nature of the 2012 Irish political landscape: a coalition government of Fine Gael and the Labour Party were in power with Fianna Fáil the main opposition party. Intuitively, cluster 3 captures the large number of Fine Gael politicians with cluster 1 capturing the Labour politicians in power with them at the time. Cluster 5 captures all of the opposition’s Fianna Fáil politicians. Cluster 2 solely contains the Sinn Féin candidates, with cluster 4 capturing many independent candidates and those from the United Left Alliance. Through the posterior similarity matrix, Figure 12 shows that politicians in cluster 2 (Sinn Féin) have the strongest certainty in their cluster labels while other politicians have some uncertainty.

Table 1. Cross-tabulation of political party membership and the LSPCM representative cluster labels

Figure 11. Fruchterman-Reingold layout of the Irish politicians network with nodes colored by (a) political party affiliation and (b) LSPCM inferred cluster labels.

Figure 12. Heat map of the posterior similarity matrix of the Irish politicians ordered by the cluster labels.

Figure 13 shows the politicians’ posterior mean latent positions on the $\hat {p}_m = 4$ dimensions with nodes colored by the maximized PEAR estimated cluster allocation across the 10 chains. On each dimension, one cluster tends to be located separately to the others, indicating that all the dimensions provide relevant information to distinguish the clusters. For example, on dimension 1 the politicians in cluster 3 are located separately from the other politicians. Similar patterns are apparent for politicians in cluster 5 on dimension 2, politicians in cluster 2 on dimension 3 and politicians in cluster 4 on dimension 4. Visually, the combination of all four dimensions allows for separation of the five clusters.

Figure 13. LSPCM inferred posterior mean latent positions of the Irish politicians on the $\hat {p}_m = 4$ dimensions with nodes colored by cluster membership.