2.1. The OECD Public Acceptability Tool as a framework for analysis

In 2021, the OECD set up its interdisciplinary Expert Group on New Measures of the Public Acceptability of Reforms to take stock of recent advances in this emerging field and explore its potential for improving policy advice. The focus of the Expert Group was on developing policy tools that can help integrate considerations of public acceptability more systematically into the reform process and on identifying relevant sources of data. Its main output consisted of a proposed OECD PAT, designed to act as a “compass”Footnote ² that can help policymakers:

• • Map the growing body of evidence provided by perceptual and behavioural data to the main dimensions that matter for understanding public acceptability; and

• • Harness the insights from these data to improve the design and communication of needed reforms.

Based on the existing literature, the PAT identifies four key dimensions that are essential for understanding public acceptability across reform areas.

1. 1. Economic dimension, which covers the expected net economic impact of a reform on different groups and how it is perceived by the public.

2. 2. Fairness dimension, which covers the expected distributional impact of a proposed reform and citizens’ views on its overall fairness, including broader evaluations on deservingness and burden-sharing across groups.

3. 3. Behavioural dimension of social change, which considers the “demands” that reforms make on individuals, alongside their capacity and willingness to adapt. The PAT identifies behavioural barriers that may undermine the case for reform and limit public support. Some of these barriers vary across reform areas. For the purpose of the analysis conducted in this article, we focus on a subset of behavioural barriers, deemed most relevant for the case studied (i.e., pension reform), termed the “Risk and time” dimension.Footnote ³

4. 4. Process dimension, which aims to capture the sense of legitimacy that a given reform can draw, or not, from the institutional and political processes used to design, implement, and justify it.

The PAT aims to provide a general framework for assessing the public acceptability of reforms, in line with existing models of this type in the literature on political economy (Stokes, Reference Stokes1996).Footnote ⁴ Its design builds on numerous recent studies focusing on the public acceptability of reforms across policy areas, such as structural reforms (Duval et al., Reference Duval, Furceri and Miethe2018; Alesina et al., Reference Alesina, Furceri, Ostry, Papageorgiou and Quinn2024), climate policies (Dabla-Norris et al., Reference Dabla-Norris, Helbling, Khalid, Khan, Magistretti, Sollaci and Srinivasan2023; Hoy et al., Reference Hoy, Kim, Nguyen and SosaMand Tiwari2023), labour regulation (Duval et al., Reference Duval, Ji, Papageorgiou, Shibata and Spilimbergo2024), and fiscal reform (Bierbrauer et al., Reference Bierbrauer, Boyer and Peichl2021; Hoy, Reference Hoy2022). It emphasizes economic impacts and fairness as key dimensions through which citizens assess reform outcomes, supported by evidence on equity-efficiency trade-offs in structural reforms (Causa et al., Reference Causa, de Serres and Ruiz2015; Ostry et al., Reference Ostry, Berg and Kothari2021) and experimental findings on perceived efficiency and equity in climate (Maestre-Andrés et al., Reference Maestre-Andrés, Drews and Van den Bergh2019; Dechezleprêtre et al., Reference Dechezleprêtre, Fabre, Kruse, Planterose, Chico and Stantcheva2022) and tax policies (Stantcheva, Reference Stantcheva2021). The framework also considers behavioural barriers that can limit public support, a focus in the literature on health interventions where addressing such barriers is crucial (Diepeveen et al., Reference Diepeveen, Ling, Suhrcke, Roland and Marteau2013; Reynolds et al., Reference Reynolds, Archer, Pilling, Kenny, Hollands and Marteau2019). Finally, it includes a process dimension, highlighting how institutional and political factors influence legitimacy, public support, and reform outcomes, as established in political economy studies (Williamson, Reference Williamson1994; Persson et al., Reference Persson, Roland and Tabellini2000) and supported by empirical evidence (Tompson and Price, Reference Tompson and Price2009).

In this article, we build on PAT and operationalise it through an analytics pipeline that demonstrates its potential application to support real-world reform.

2.2. The case study: The 2023 French pension reform

We combined the framework provided by the PAT with the analysis of nontraditional data to extract information on the public acceptability of reform through a retrospective study of a recent and relevant case: the 2023 French pension reform. The reform was announced by President Macron in his 2022 end-of-year speech and introduced in Parliament in February 2023. It was finally adopted in March 2023 through the application of Article 49.3 of the French Constitution, which allows the government to commit its responsibility and pass an executive proposal ( projet de loi ) into law without parliamentary approval, but subject instead to a vote of no-confidence in the government ( motion de censure ). The key elements of the reform included (i) an increase in the minimum legal retirement age from 62 to 64 years, introduced incrementally over a period of 7 years, and (ii) an increase in the necessary contribution period for a full pension to at least 43 years of work.

This case is relevant for the analysis for various reasons. First, the French pension reform was in line with the type of reforms put in place in many comparable countries. In recent years, the legal retirement age has risen in more than half of the OECD member states (OECD, 2023). Second, public acceptability was a critical issue in this case, as the reform sparked widespread opposition and remained under threat of repeal even after it was passed into law (Kagni, Reference Kagni2024). Reflecting this, the implementation of the reform has been temporarily suspended by a subsequent law passed by the French National Assembly on 12 November.Footnote ⁵ Finally, it followed many national reform efforts in the same field, including two parametric reforms in 2010 (Loi Woerth) and 2014 (Loi Touraine), as well as a proposed systemic pension reform in 2019, withdrawn during the COVID-19 pandemic.

The four dimensions of the PAT featured prominently both in the government’s case for reform and in public debates surrounding it. As such, the PAT constitutes a useful lens for understanding this case.

• • Economic dimension: The reform aimed to ensure the financial sustainability of the French pension system and eliminate its projected deficit by 2030, addressing concerns over the high level of public expenditure on pensions (13.4% of GDP) compared to the OECD average (7.7%) (OECD, 2023).

• • Fairness dimension: The government also built its case for reform on fairness grounds, emphasising its role in maintaining the French “pay-as-you-go” system, preserving intergenerational solidarity, and protecting vulnerable populations (Ficek, Reference Ficek2023). Despite this, criticism of the reform in the public debate focused mainly on issues of fairness, notably in terms of its impact on gender equality and on different occupations (Vazquez, Reference Vazquez2023).

• • Behavioural dimensions (limited in our case to risk and time): The behavioural complexities typical of pension decisions were taken into account, to a degree, in the design of the reform, which allowed for a gradual implementation to allow people time to adjust to the new parameters (Government of France, 2023). However, the balance between the present and future benefits was often criticised (Poznanski, Reference Poznanski2024).

• • Process dimension: The case for reform sought to build legitimacy from the fact that President Macron had a clear electoral mandate for pension reform (Macron, Reference Macron2022). Opponents drew on the use of Article 49.3, an exceptional procedure that bypasses parliamentary approval, to contest the legitimacy of the reform in public debates (Mathieu, Reference Mathieu2023).

To operationalise public acceptability in line with the OECD’s PAT, we defined a list of keywords for each dimension (available in Supplementary Materials, Annex A). The keywords have been identified in a two-step process. First, a set of keywords was defined by OECD domain experts for each dimension, a priori with respect to reading the corpora used as an input to this research. These keywords were selected for the specific purpose of analysing textual data on the pension reform; hence, they aimed to provide a general description of the dimensions of the framework (i.e., productivité , discrimination, futur , démocratie ) and explicitly cover topic-specific aspects (i.e., emploi , régime spécial , ajustement automatique , motion de censure ). These sets of keywords were then used to filter media articles along each of the four dimensions and select only the documents that contained relevant keywords. The keyword selection was conducted by experts through a consensus-based process. This collaborative approach ensured alignment with both the conceptual underpinnings of the PAT framework and the specific discourse context of the pension reform. At this point, we performed topic modelling on these subsets of articles, which provided us with a list of topics and related topic descriptors for each of the four dimensions. The topic descriptors of the four topic models were systematically reviewed and binarily classified by domain experts into the four categories. Descriptors that did not align with any dimension were excluded, and while descriptors could theoretically be associated with multiple dimensions, no overlaps were observed during this process. This classification constituted the second step of our two-step keyword identification strategy. The whole two-step process allowed us to obtain final sets of keywords that combine theoretical relevance—as the first set of keywords used for filtering was informed only by the reflections of domain experts—and relevance to the corpus, thanks to the consideration of the topic descriptors.

2.3. The data

We used news media articles published online by a set of the most-read French media outlets as input to this analysis. We complemented these data with parliamentary speeches. In this way, we were able to assess the differences between the two corpora and hence to characterise differences in the way in which the pension reform was discussed in the media and in Parliament.

Our dataset of online media articles consisted of 75k publications about pensions published between November 2022 and June 2023 by French outlets. We retrieved them from the Global Database of Events, Language, and Tone (GDELTFootnote ⁶), querying for all records whose Themes or V2Themes contained references to pensions or retirement, and we retained only those published by French outlets, identified via GDELT’s FIPS country code for France (FR) or a “.fr” domain. We then filtered the articles to include only those from 10 selected influential sources,Footnote ⁷ identified from the French Alliance for Press and Media Statistics (ACPM) and Statista’s consumer surveys. This selection resulted in a final set of 10,141 articles published over the relevant period. To obtain the text of the articles, we analysed the link obtained on GDELT with the Newspaper3kFootnote ⁸ Python library.

We retrieved data on parliamentary speeches from the open data portal of the French National Assembly,Footnote ⁹ collecting those on the debates in public sessions in the relevant period. We then filtered them to retain only (i) speeches from sessions specifically dedicated to the 2023 pension reform, identified by analysing the session titles, and (ii) speeches that mentioned “ retraite ,” regardless of the session title. The final dataset contains 1392 speeches.

All textual data in our study were pre-processed for consistency and noise reduction. We used spaCy to perform lemmatisation, reducing inflectional and derivational forms to their base forms (Honnibal, Reference Honnibal2015). Stop words were removed to filter out commonly used words that carry little semantic value. Punctuation was also excluded to eliminate non-essential symbols. A detailed description of the preprocessing pipeline is available in Appendix A.

The distributions over time of the media articles and parliamentary speeches we retrieved do not present the same dynamic. Figure 1 shows the daily number of news media articles and parliamentary speeches considered in our analysis. The volume of articles increased from the moment the reform was introduced into the public sphere by the French President Emmanuel Macron in the annual presidential New Year’s Eve speech on 31 December 2022. After a small drop in volume around the beginning of March, attention then peaked in the weeks in which the reform was enacted into law, after Article 49.3 was invoked by the government of Prime Minister Élisabeth Borne. Overall, media articles show a clear dynamic over time. On the contrary, the parliamentary speeches we are working with are sparsely distributed. This characteristic derives from the fact that our data are from the debates in public sessions of the National Assembly. As shown in the plot, many of the parliamentary speeches we collected are condensed in the period between February and the early days of March 2023.

Figure 1.

Daily number of news media articles and parliamentary speeches about pensions.

Figure 1. Long description

The x-axis represents Date from November 2022 to June 2023. The y-axis represents Volume from 0 to 120. A legend in the top right identifies a red line as Media articles and a blue line as Parliamentary speeches.

* The red line for media articles begins with low-level fluctuations between 10 and 30 until January 2023. It then rises sharply, oscillating between 40 and 100 through February and March. It reaches its highest peak of approximately 120 in early April before gradually declining with continued volatility through June.

* The blue line for parliamentary speeches remains near zero for most of the period, with three distinct spikes. The first spike occurs in February reaching 60, the second and largest spike reaches over 90 shortly after, and a third spike reaches 70 in March. Smaller peaks around 30 appear in May and June.

* Four vertical gray lines mark key events:

1. The reform is announced in early January.

2. The Law is introduced in Parliament in early February.

3. Article 49.3 is invoked in mid March.

4. Constitutional Council ratifies the Law in mid April.

2.4. Document selection

After obtaining news media articles and parliamentary speeches about the reform, we aimed to identify which documents pertain to the four dimensions of the PAT. For this task, we drew inspiration from the field of document retrieval and developed a document ranking method leveraging word embeddings—whose details are presented below and in Appendix A.

First, we trained a Word2Vec model on the two corpora using the Continuous Bag-of-Words (CBOW) model (Mikolov et al., Reference Mikolov, Sutskever, Chen, Corrado and Dean2013). Word2vec is an NLP technique that generates vector representations of words, capturing semantic relationships based on their surrounding context. By maximising the log conditional probability of a word given its neighbouring words within a fixed-sized window, the CBOW model learns embeddings for each word in the corpora. This results in vector representations for every word in our datasets.

Given a document D comprising a set of words $ {d}_1,{d}_2,\dots, {d}_N $ , we obtained an embedding representation of the document by computing the centroid of the word vectors. This approach provided a concise representation of the document as the average of its constituent word vectors:

(2.1) $$ \overline{D}=\frac{1}{\mid D\mid}\sum \limits_{d_i\in D}{d}_i $$

To measure the relevance of each document to the four dimensions of the PAT, we calculated the similarity $ S\left(W,D\right) $ between each document vector and the keywords $ {w}_i $ associated with each dimension of public acceptability $ W $ . To account for the frequency and specificity of keywords, we applied term frequency-inverse document frequency (TF-IDF) weighting. TF-IDF is a statistical measure that evaluates the importance of a word in a document relative to a collection of documents, increasing the weight of terms that are more specific and less frequent. This dampens the impact of common words and emphasises the importance of more specific and discriminative terms (Sparck Jones, Reference Sparck Jones1972).

The formula for calculating the TF-IDF-weighted similarity score for a document embedding $ \overline{D} $ with respect to a dimension $ W $ is as follows:

(2.2) $$ S\left(W,D\right)=\frac{\sum_{w_i\in W}\left(\frac{w_i\cdot \overline{D}}{\parallel {w}_i\parallel \cdot \parallel \overline{D}\parallel}\right)\times idf\left({w}_i\right)}{\sum_{w_i\in W} idf\left({w}_i\right)} $$

where $ idf\left({w}_i\right) $ is the TF-IDF weight of keyword $ {w}_i $ and denotes the Frobenius norm, which is used to compute the cosine similarity between the keyword embedding $ {w}_i $ and the document embedding $ \overline{D} $ . By computing this similarity for all the documents in the two corpora of news media articles and parliamentary speeches for each of the four dimensions, we were able to estimate how closely each article corresponds to each dimension. Figure A1 illustrates the distribution of similarities for both news media articles and parliamentary speeches. To facilitate comparison across dimensions, we normalised the distributions between 0 and 1. For each dimension, we then kept the articles and parliamentary speeches with similarity values ≥0.9. To ensure that our results are not overly sensitive to the choice of threshold, we tested the document selection and topic modelling pipeline using alternative cut-offs at 0.95 and 0.85 levels. These thresholds correspond to the top 5%, 10%, and 15% of the similarity value distribution, meaning that each dimension retains a different number of documents depending on how many exceed the similarity value. The resulting topic models showed strong convergence in topic structure and descriptors across thresholds, with only minor variations, as shown in Appendix A. The 0.9 threshold is robust and strikes a balance by retaining a sufficiently large number of documents for meaningful topic modelling, while limiting the inclusion of potentially noisy or weakly relevant texts.

To evaluate the embedding-based document classification, we conducted a document intrusion task inspired by the word intrusion proposed by Chang et al. (Reference Chang, Gerrish, Wang, Boyd-Graber and Blei2009). For each PAT dimension, we sampled 10 sets of four documents: three randomly drawn from those with similarity $ \ge 0.90 $ for that dimension, and one intruder document drawn from documents with similarity $ \ge 0.90 $ to a different dimension. Annotators were asked to identify the document that did not belong. This task assesses whether the documents selected as most relevant to a dimension form a semantically coherent group. Across dimensions, annotators correctly identified the intruder in 74.06% of cases on average, compared to a 25% random baseline expected if selections were made by chance (i.e., one out of four documents). We observed some variation across dimensions: Process and Economic documents showed the highest performance (92.5% and 87.5%, respectively) while Risk and time exhibited lower results (51.25%), still meaningfully above the 25% random baseline but substantially below the other dimensions. This weaker validation may reflect the more overlapping semantic space this dimension occupies in public discourse, as many of its core terms (e.g., those relating to long-term costs and intergenerational trade-offs) also appear in discussions of economic impacts and fairness. Consequently, findings pertaining to the Risk and time dimension should be interpreted with greater caution than those for the other three dimensions (see Appendix A). In general, these results indicate that the document similarity method yields dimension-specific document sets that are meaningfully coherent to human readers.

2.5. Topic modelling with NMF

After identifying documents related to the four dimensions of the PAT, we used topic modelling to uncover latent themes within these sets of documents. Topic modelling is a statistical technique that identifies underlying topics or themes in a collection of documents by analysing the frequency and co-occurrence of words. In this case, we utilised NMF, a classical technique for topic modelling, to extract meaningful insights from our document sets (Paatero and Tapper, Reference Paatero and Tapper1994). The implementation was performed with the Python package Gensim (Rehurek and Sojka, Reference Rehurek and Sojka2011).

NMF is a dimensionality reduction technique that decomposes a non-negative matrix into two smaller non-negative matrices. Thanks to its non-negativity constraint, it lends itself well to interpreting data from physical applications (Lee and Seung, Reference Lee and Seung1999) and has been extensively used in the field of text mining for topic modelling and document clustering (Pauca et al., Reference Pauca, Shahnaz, Berry and Plemmons2004). We applied NMF to a term-document matrix where rows represent unique terms, columns represent documents, and the values indicate the TF-IDF frequency of each term in each document.

The NMF algorithm factorises the term-document matrix $ A $ into two matrices that capture the underlying structure of the data.

(2.3) $$ V\approx WH $$

The first matrix $ W $ , called the term-topic matrix, shows the importance of each term to each topic. The second matrix $ H $ , known as the document-topic matrix, represents the relevance of each topic to each document in the corpus.

NMF topic modelling requires selecting the number of topics. We chose to extract 10 topics for 2 reasons. First, our goal was to provide policy-relevant insights, and too many topics would reduce usability. Second, coherence analysis, which measures the semantic consistency of topics, showed that coherence decreased with a larger number of topics, making them harder to interpret. We ran this analysis both on the full corpus and on the filtered subsets used for dimension-specific topic modelling. As scores across different coherence measures declined monotonically with increasing number of topics $ k $ , we opted for a relatively low number of topics ( $ k=10 $ ). While even smaller values of $ k $ produced high coherence scores, they lacked sufficient granularity for policy insight. Conversely, higher values led to fragmented and less interpretable topics. Choosing $ k=10 $ thus provided a balance between semantic coherence and interpretive usefulness. Details of the coherence analysis and model stability checks are provided in Appendix A.

In line with recommendations from the literature on topic model evaluation, we complemented the coherence analysis with a word intrusion task to assess the semantic interpretability of the learned topics (Chang et al., Reference Chang, Gerrish, Wang, Boyd-Graber and Blei2009; Ying et al., Reference Ying, Montgomery and Stewart2022). For each topic, we presented eight human annotators with sets of six terms: five high-probability words drawn from the topic and one intruder word sampled from a different topic with low probability weight. Annotators were instructed to identify the intruder, and accuracy is taken as a measure of topic coherence. Across dimensions, annotators selected the correct intruder 71.88% of the time on average, substantially higher than the 16.7% random baseline expected under uniform guessing. While performance was consistently above baseline for all dimensions, we observed modest variation: topics associated with the Fairness and Process dimensions showed the highest accuracy, whereas topics in the Risk and time dimension scored slightly lower (see Appendix A). These results indicate that the topics identified by the NMF models are interpretable and meaningful to human readers, while also highlighting areas where themes may be more conceptually entangled.

It is worth noting that the word intrusion task, like the document intrusion one, seems to favour dimensions with more lexically distinctive vocabularies. For instance, a dimension like Process employs vocabularies that are relatively specialised and distinguishable, whereas Risk and time shares substantial lexical ground with Economy and Fairness, with terms relating to costs, benefits, and intergenerational equity. As a result, the validation scores may also reflect the degree of lexical separability between dimensions rather than solely the conceptual validity of the PAT framework as applied to this corpus. This distinction is important for interpreting the results: dimensions that validate strongly may do so in part because their vocabulary is more distinctive, not necessarily because the underlying concept is better represented in the data.

By training topic models for each dimension of public acceptability and for each corpus obtained by selecting the most relevant decile of documents, we uncover latent topics that characterise the discourse within each dimension. Moreover, by incorporating temporal information on the publication dates of articles and the dates of parliamentary speeches, we are able to analyse the evolution of these topics over time and track their evolution in relation to significant events.

2.6. Corpora comparison with CMF

Beyond analysing topics linked to each dimension of the PAT, we aim to detect analogies and differences in the way in which pension reform is discussed in different corpora. For this analysis, we implemented a comparative approach based on CMF to analyse media articles and parliamentary speeches (De Lathauwer and Kofidis, Reference De Lathauwer and Kofidis2017). This method could in principle be adopted to detect similarities and differences of other corpora, such as the coverage of the reform by left- and right-leaning media outlets or social media content produced by different segments of the population (Barberá et al., Reference Barberá, Casas, Nagler, Egan, Bonneau, Jost and Tucker2019).

CMF is a technique that extends traditional matrix factorisation methods to simultaneously decompose multiple matrices. Formally, given two document-term matrices $ {X}^{(1)} $ (parliament) and $ {X}^{(2)} $ (media), CMF approximates each as

(2.4) $$ {X}^{(i)}\approx {B}^{(i)}{D}^{(i)}{C}^{\top } $$

In this factorisation, $ {B}^{(i)} $ represents the document-topic matrix for the i-th dataset, where each row corresponds to a document (e.g., an article or speech) and each column corresponds to the weight of a specific topic. $ {C}^{\top } $ is the shared factor matrix across all matrices $ {X}^{(i)} $ , representing the topic-term matrix. Each column of $ {C}^{\top } $ contains the terms associated with each topic, making it a common set of topics across both datasets. The matrix $ {D}^{(i)} $ is a learned diagonal scaling matrix that weights the contributions of the shared topics $ {C}^{\top } $ in each dataset, allowing the topics to be weighted differently across the different sources (e.g., media articles and parliamentary speeches). We do not impose any structure on $ {D}^{(i)} $ ; rather, it is inferred from the data during optimisation to capture the differential salience of each topic across corpora. All the factor matrices are learned by minimising the reconstruction error across corpora, measured as the sum of squared differences between each original matrix and its approximation. This makes CMF closely related to NMF, but extended to handle multiple datasets with shared topics.

In our context, CMF has been adopted to simultaneously analyse media articles and parliamentary speeches by identifying a single set of topics. This joint factorisation allows us to capture shared thematic structures while still preserving the differences in how these topics are represented in each dataset.

We implemented it using the Python package MatCoupLy (Roald, Reference Roald2023). We imposed non-negativity constraints on the factors, leading to more interpretable results. Consequently, our implementation of CMF is effectively a simultaneous application of NMF on two corpora, with the added benefit of a shared topic structure.

In our implementation of CMF, we take the two TF-IDF term-document matrices of media articles and parliamentary speeches as input. The factorisation then produces two different document-topic matrices, two different diagonal matrices that define how much each corpus covers each topic, and one shared term-topic matrix. This shared term-topic matrix is used to obtain a list of topics, defined through their respective most relevant terms.

To measure the relative salience of each topic across corpora, we computed the weight of each topic $ {t}_i $ in each corpus as the L2 norm of the topic-specific reconstruction component (based on the factor matrices), normalised by the L2 norm of the full reconstructed corpus. Let $ {t}_i^{(m)} $ and $ {t}_i^{(p)} $ denote the normalised weight of the topic $ i $ in the media and parliament corpora, respectively. We then computed their relative divergence:

(2.5) $$ \Delta {t}_i=\frac{\left({t}_i^{(m)}-{t}_i^{(p)}\right)\times 100}{\max \left({t}^{(m)}\right)+\max \left({t}^{(p)}\right)} $$

where $ \max \left({t}^{(m)}\right) $ and $ \max \left({t}^{(p)}\right) $ are the maximum topic weights in the media and parliamentary corpora, respectively. This formula allows us to quantify the relative difference in topic emphasis between the two corpora, and the normalisation facilitates comparison across dimensions while preventing large weights from dominating the result.

3.1. Studying the evolution of the public acceptability dimensions over time

Figure A2a shows the evolution over time of the volume of media articles related to each of the four dimensions of the OECD PAT. The number of articles assigned to each dimension has been aggregated at a weekly level and normalised over the total of articles published each week for the period between 1 November 2022 and 15 June 2023, to avoid distorting the plotted volumes due to the quantity of articles published. Because documents are assigned to dimensions when their similarity value is $ \ge 0.90 $ , the number of articles associated with each dimension naturally varies, reflecting the relative salience of each dimension in the public debate. It is visually clear that elements relating to the Process dimension have dominated the news media coverage of the reform, and the focus on this dimension increased once the law was introduced in the National Assembly. Conversely, focus shifts away from elements relating to the Economic, Fairness, and Risk and time dimensions, which initially got greater coverage.

We do not observe the same pattern in parliamentary speeches. As shown in Figure A2b, the number of speeches covering the Process dimension of public acceptability grows over time, in particular starting from the beginning of March. Yet, this increase does not result in a clear predominance of this topic. Further, it does not co-occur with a decrease in the discussion of other dimensions.

The difference in focus between media articles and parliamentary speeches is not surprising and may simply reflect the fact that topics discussed in parliament are of a more technical nature, more comprehensive and must deal with the legal and procedural aspects of the reform. In turn, the technical nature of parliamentary speeches means that they do not always resonate directly with the broader issues and themes that drive public acceptability. These are more likely to be captured in the public’s discussion and in media coverage.

In addition, we should consider that the time distribution of parliamentary speeches is relatively sparse, as shown in Figure 1. The data reflect only speeches made during open sessions of the National Assembly, which limits the overall volume and frequency of speeches compared to the more continuous media coverage, in particular during the period between March and May, when the reform was passed through the use of Article 49.3, and protests were particularly intense. This might contribute to the observed differences in topic focus over time.

3.2. Analysing topics along the four dimensions of the PAT

By performing topic modelling with NMF, we have identified 10 latent topics within the discussion related to each of the four key dimensions of public acceptability identified by the PAT. By considering media articles’ publication date, we were then able to combine this information with the 10 topics and plot their evolution over time as time series.

Figure A4 provides information about the discussion on the Economic dimension of public acceptability, showing the temporal coverage of four selected topics. These topics were chosen based on policy considerations, with a focus on those that provide deeper insights and reveal meaningful patterns and information on the nuances of public debate relating to this dimension. The time series of the other topics for this and the other dimensions are provided in Supplementary Materials, section B. The topics are represented by the three most relevant words—the topic descriptors—which were identified considering each word’s contribution to the topic from the term-topic matrix. The results indicate the salience of the issue of public debt ( dette milliard public)—one of the main objectives of the reform and a key element in its rationale—in news media coverage. Other relevant topics include discussions about inflation ( prix production consommation ), housing ( logement immobilier taxe ), billionaires ( milliardaire riche fortune ), and private insurance ( assurance fonds assureur ). Furthermore, the evolution in the coverage of different topics can provide an indication of how the focus of public debate responded to external events and to key moments in the legislative process. We can appreciate, for instance, the gradual increase in the coverage of the topic on public debt or a clear focus on inflation—a central economic feature of 2023, often linked to the war in Ukraine and the energy crisis, which was, on average, over the whole 2023, up by 4.9% after +5.2% in 2022 (INSEE, 2024). This topic peaked right before the pension reform was adopted through the use of Article 49.3.

We can further explore the topics by analysing the most relevant documents related to each of them. With respect to public debt, the articles show that reducing public expenditure and ensuring the financial sustainability of the pension system were clearly seen as a goal of the reform (Echos, Reference Echos2023). Yet, others argued against the reform, claiming it was “emblematic of the government’s inability to control public spending, despite its touching desire to do so” (Guyot and Vranceanu, Reference Guyot and Vranceanu2023). Looking at insurance and pension savings, the debate highlighted the importance of life insurance as an investment and asset-management tool, as well as its increasing popularity among the French population. Here, the discussion put significant emphasis on the new plan épargne-retraite (PER), which came into force on 1 October 2019 and aims to unify existing retirement savings products, whether individual or collective, with a view to making them easier for savers to understand (Boccara, Reference Boccara2022; Figure A3).

Figure A4 shows the topics surfaced through the analysis of articles relating to the Fairness dimension, along with the temporal evolution in their coverage. We find some explicitly economic topics, which is because the reform acts on economic factors (see emploi an salaire , fonds assurance placement , capitalisation pension cotisation ). Furthermore, we observe significant attention given to topics related to gender dynamics ( femme homme couple) and how the reform would impact them, to wealth ( riche fortune fiscal ), to the effect of the reform on specific sections of the population, such as agricultural workers ( exploitation agricole agriculteur ), as well as the clergy ( culte maladie ministre ) (Senèze, Reference Senèze2023b). The salience of these topics clearly grew after the reform was announced, often peaking in the period between the introduction of the Law into Parliament and its final passage through Article 49.3.

Considering the most representative media articles for each topic can offer further insights about the public debate, and hence into the issues that drive the public acceptability of the reform. For instance, documents linking the reform to gender issues emphasise how it may discriminate against women, who (i) often tend to work in lower-paid occupations and have smaller pensions as a result; and (ii) find it more difficult to reach the necessary contribution period for a full pension than men, as they are more likely to experience career interruptions or reduce their working-hours due to the uneven and gendered division of caring duties and responsibilities within the household (Vazquez, Reference Vazquez2023). Furthermore, analysing the wealth-related aspects of the debate, we find significant discussion on the proportionality of taxation, sharing of the tax burden, and alternative sources of financing. Illustrating this last point, for example, a report by the NGO Oxfam was often mentioned. This report proposed a 2% tax on the wealth of France’s 42 billionaires, which, according to Oxfam’s projections,Footnote ¹⁰ would draw sufficient revenue to cover the anticipated deficit in the pension system by 2027 (Dupont, Reference Dupont2023; Oxfam, 2023).

The analysis of the discussion on the Risk and time dimension surfaces topics shared with other dimensions, as demonstrated by Figure A5. Clear examples of this can be found, for example, with topics related to private insurance ( assurance fonds assureur ) and women ( femme pension revenu ). The presence of many of the elements pertaining to this dimension in other dimensions suggests that, in the context of the 2023 French pension reform, the Risk and time dimension was semantically overlapping with others. As discussed in Section 2.4, the comparatively lower validation score for this dimension could indeed suggest that the topic structure identified here partly overlaps with adjacent dimensions; hence, these results should be considered accordingly.

Finally, we conducted the topic modelling analysis on the set of articles related to the Process dimension. Figure A6 shows the 10 topics identified along with their coverage over time. We find mention of the key political actors associated with the reform ( ministre borne macron),Footnote ¹¹ but also topics related to parliamentary dynamics, party positions, policy discussion, and voting (LIOT ^{Footnote 12} proposition groupe , vote texte gouvernement , amendement LFI ^{Footnote 13} insoumis ). Furthermore, we also find topics that refer to crucial and specific steps in the reform process. First, we detect a focus on the use of Article 49.3 of the Constitution, which allowed the government to adopt the reform without the approval of the National Assembly (motion censure 49). Second, the discussion also focused on the reform’s review by the French Constitutional Council, which issued a ruling on the legality of the draft law following a challenge filed by opposition parliamentarians. This review found the draft law to be constitutional and rejected an initial request for a referendum (conseil constitutionnel RIP ^{Footnote 14} ). Considering the temporal evolution of the topics, we find that, for most of them, the coverage started increasing steeply from March, when the most important steps of the reform process took place.

Analysing the most representative articles for each topic, we obtain further insights into their meaning. Considering discussions of the role played by Prime Minister Borne and President Macron, we find them depicted as striving to navigate the public and political tension. Articles describe violent protests and social upheaval, with strong criticism aimed towards the Prime Minister and the President by opposition parties and unions for their use of Article 49.3. Articles underline the high stakes for both of them and suggest that their political futures (at that date) may depend on how well they manage the crisis and navigate the ongoing opposition (Gatinois and Trippenbach, Reference Gatinois and Trippenbach2023; Quinault-Maupoil and Conruyt, Reference Quinault-Maupoil and Conruyt2023). The documents related to the broad topic politique social peuple highlight the fact that there was a sense that the government was becoming increasingly authoritarian and that people were losing faith in the political system. Segments of the population, including researchers, claimed that the reform was determining a violation of the French Constitution (Collectif, 2023). Considering the role of the Constitutional Council, the articles note that its validation reiterated the legal basis of the reform but did not quell public dissatisfaction entirely. Indeed, doubts about the choice of the Council were raised after its decision (Schneegans, Reference Schneegans2023).

While the prominence of the Process dimension in media coverage may initially appear predictable, the analysis unpacked the specific institutional and political mechanisms that received attention, such as motions of censure, party amendments, and constitutional review, and showed how media framing evolved over time, highlighting a marked escalation in focus during key legislative moments. Importantly, this detailed analysis of topics and articles verified that the media articles most strongly associated with the Process dimension were not merely reporting on parliamentary speeches. Rather, they conveyed independent narratives that critically engaged with institutional and political dynamics described above. Importantly, our qualitative reading of the most representative articles suggests that the media articles most strongly associated with the Process dimension appeared to convey independent narratives that critically engaged with the institutional and political dynamics described above. For instance, the most Process-associated articles focused on themes such as the constitutional implications of Article 49.3, the political strategies of opposition parties, the role of the Constitutional Council, and public protest dynamics. These topics, even if related to legislative events, were framed in the news through distinct journalistic angles rather than as summaries of parliamentary proceedings. To enable readers to evaluate this claim directly, we share data and code, allowing independent verification of the relationship between media and parliamentary framing of procedural themes (see Data availability statement.

3.3. Detecting analogies and differences between media and parliamentary debates

Through CMF, we were able to perform topic modelling on two corpora simultaneously and thereby identify a unique set of topics from both of them. Considering the outputs of the topic modelling, in particular the two document-topic matrices, we were able to assess to what extent a topic was relatively more discussed in one or the other corpus.

Figure A7 shows the result of this analysis, which should be interpreted recognising the structural differences of the corpora. While media coverage can follow the full reform cycle—from early leaks and public reactions to final decisions—parliamentary speeches are more temporally concentrated and shaped by institutional constraints. When considering the Economy dimension, we find that the media focused relatively more on private insurance ( PER plan rente , fonds assurance investissement ) and housing (SCPI ^{Footnote 15} immobilier rendement , logement taxe immobilier ). Looking at the most representative articles for these topics, we find that they often mention the challenges faced by retirees and future retirees regarding their financial stability, especially in the context of rising living costs and inflation. SCPIs are presented as a viable alternative for retirees to invest in real estate indirectly and receive monthly or quarterly rents, potentially offering higher returns than traditional savings accounts (Tribune, Reference Tribune2023). Considering the topics that received greater attention in parliamentary speeches, we find that the most diverging one is about research ( recherche france chercheur ). The most representative speeches for this topic put emphasis on a possible increase in the research budget to 3% of GDP by 2027, with a particular focus on biomedical research. We also find that the coverage of public debt and inflation, in particular in relation to the agricultural sector, was almost equally discussed in the two corpora.

Considering the discussion related to the Fairness dimension, we find that the media focused relatively more not only on private insurance plans and investments ( fonds rendement assurance , PER rente plan) but also on the role of parenting ( pension alimentaire parent ). Regarding this topic, the analysis of articles shows that media outlets frequently linked the pension reform to the newly established public service for child support payments. This service allows a public agency to automatically deduct and distribute child support payments, regardless of the type of separation, as long as the amount is officially determined. In parliamentary speeches, topics that received relatively more attention include special pension regimes ( régime spécial général ), hazardous or arduous jobs ( pénibilité travail métier ), and the effect of the reform on the agricultural sector ( agricole agriculteur exploitation ). Considering the first of the three topics, we find that parliamentary discussions emphasised how the reform aims to prepare for future management of pensions by integrating special regimes into the general system, ensuring stability and fairness for future generations. However, other voices defended the autonomous pension regime of clerks and employees of notaries, which is self-managed, balanced, and does not rely on public funds. They argued for maintaining this well-managed system. When it comes to hazardous or arduous work, the multifaceted nature of the notion was emphasised, describing at least four aspects, including objectively measurable factors, notoriously harmful working conditions, experienced hardships due to health fragility or organisational conditions, and the desire to leave one’s job due to these hardships. Further, parliamentary speeches highlight the fact that current regulations only account for objectively measurable forms of arduous work, which applies to a small, specific population exposed to extreme demands for long durations.

The comparison between articles and speeches related to the Risk and time dimension does not yield particularly new topics, with one exception. Among the topics discussed almost equally between the two fora, we find one related to special regimes for the clergy ( culte ministre professionel ). The articles discuss the integration of religious ministers into the French social security system and the specifics of the Caisse d’assurance vieillesse et maladie des cultes (Cavimac). The discussion on this topic is informed by two crucial dynamics. First, the number of pensioners covered by this fund has decreased significantly from 67,000 in 2003 to 38,158 in 2020, due to higher death rates compared to new retirees. Second, the number of contributors has been increasing, with a 13% rise from 2018 to 2022, driven by the inclusion of foreign religious personnel and improved affiliation practices (Senèze, Reference Senèze2023a).

Finally, considering the Process dimension, we find that only one topic was clearly more discussed in parliamentary speeches, which is related to the left-wing electoral alliance of political parties in France ( groupe banc NUPES ^{Footnote 16} ). The speeches emphasise that the government does not have a majority in the National Assembly or support from the public for the pension reform. Furthermore, they often contain criticism of the government for undermining democratic principles by bypassing parliamentary debate and disregarding public opinion. The speeches use emotional and forceful language to convey urgency and seriousness. Terms like “madness,” “folly,” and “shame” are used to describe the government’s actions. Many topics surfaced in this dimension have been relatively more discussed in the media. Among them are those about unions ( syndicat CGT ^{Footnote 17} CFDT ^{Footnote 18} ) and the Constitutional Council (conseil constitutionnel RIP). This divergence reflects the roles and logics of the two arenas: parliamentary debate formalises positions within the legislature and is limited to procedural stages, while media coverage tracks protest dynamics and institutional milestones in near real-time. Furthermore, this unequal coverage of topics between media and parliament might result from the fact that only a minority of the speeches analysed were delivered in the months after the first week of March (see Figure 1). This is the result of using only the speeches about pensions made during the open sessions of the National Assembly. Using different data directly produced by policymakers, such as press releases or interviews, would probably result in a different distribution of topics among the two corpora.