Introduction
In this article, text mining processes are applied as an initial step to map and analyse the patterns and changes in the National Disability Insurance Scheme (NDIS) in Australia as a basis for interpreting the underlying logics of policy design and reform of the scheme. By mapping and analysing the substantive content of NDIS policy, including legislation, the operational framework, and review and inquiry reports, the broad purpose of this research was to consider the principles, values and issues that are emerging and declining in policy documents and elucidate current tensions. Text mining approaches were undertaken using a sample of ten key NDIS documents from 2011 (preceding the NDIS) through to December 2019 (the most recent major review which started the modern rapid reform era). Drawing on policy studies literature and policy agenda research, our specific interest is in how policy agendas are articulated within policy design and change over time.
A landmark reform in Australia, the NDIS was a culmination of grassroots advocacy and community pressure, multiple state and federal inquiries and reports detailing the systemic failings of government policy (Disability Investment Group, 2009; National People with Disabilities and Carer Council, 2009; Productivity Commission, 2011), and ratification of the United Nations Convention on the Rights of Persons with Disability (CRPD) (Kayess and French, Reference Kayess and French2008). The NDIS was introduced into legislation in 2013 as an individualised funding scheme to provide ‘reasonable and necessary’ supports for eligible Australian citizens living with disability. Although bringing positive change to the lives of many participants, it has also been a turbulent time for the Scheme, which has been plagued by design tensions and implementation issues (Carey et al., Reference Carey, Malbon, Reeders, Kavanagh and Llewellyn2017; Malbon et al., Reference Malbon, Carey and Dickinson2018, Reference Malbon, Carey and Reeders2019; Nevile et al., Reference Nevile, Malbon, Kay and Carey2018; Olney and Dickinson, Reference Olney and Dickinson2019; Foster et al., Reference Foster, Hummell, Fisher, Borg, Needham and Venning2021; United Nations, 2019). Since its inception, there have been numerous inquiries into various aspects of the Scheme, resulting in recommendations for change, and proposed reforms (see Table 1).
NDIS reforms and inquiries overview (2013-October 2022)
Designing policy in an environment of competing priorities and finite resources is a persistent challenge for governments, and understanding how and why policy problems and their solutions are shaped during design and implementation stages is a critical area of inquiry for policy researchers (Howlett, Reference Howlett2019). In examining continuity and change in NDIS policy, our approach is inspired by policy agenda literature (Dowding et al., Reference Dowding, Hindmoor and Martin2013; Baumgartner et al., Reference Baumgartner, Breunig, Grossman, Baumgartner, Breunig and Grossman2019). In this article, we use an innovative technique to quantitatively map the attention given to matters and principles in the NDIS policy (legislation, rules) and in the public domain (reviews and inquiries into the NDIS), and to draw some conclusions about the consistency of espoused aspirations of choice and rights, vis a vis the emergence and prominence of other issues and principles such as cost or sustainability. This case additionally serves as an exemplar for the utility of tracing the evolution of policy reform, which has potential application for other national and international ‘wicked problems’, including climate change, aged care, and poverty.
The work presented in this article is part of a broader project (titled ‘Adjudicating Rights for a sustainable NDIS’) examining how rights to funded support in the NDIS are interpreted and administered. This text mining approach sits within the initial phase of mapping institutional logics and operating principles and provides context for subsequent phases including qualitative interviews with key NDIS stakeholders and content analysis of administrative decisions. The specific aims of the article are therefore to: (1) examine the substantive content of the NDIS legislative, operational and reform documentation, (2) determine the patterns and shifts in objects, principles and issues over time (and in what ways these change), and (3) discuss the current implications in terms of the logics and agendas of NDIS policy reform. By way of introduction to our approach and analysis, the next section situates this article within the policy design and policy agenda literatures, before describing the NDIS. This is followed by a description of the methodology prior to presentation and discussion of the findings.
Policy design and agenda-setting
Policy design is a complex enterprise and is part of the field of policy studies, which is devoted to the systematic examination of policy, from agenda-setting to formulation and implementation, including the types of tools governments have at their disposal to achieve a stated purpose (see for example Dryzek and Ripley, Reference Dryzek and Ripley1988; Howlett and Lejano, Reference Howlett and Lejano2013). Although policy design can include both extensive reform and incremental change, Head (Reference Head2022) asserts that designing policy is never more difficult than when addressing ‘wicked problems’ and creating innovative reforms that have no parallel in any existing jurisdiction. Head reminds us that policy debates “always include key moments when the nature and scope of problems are intensely disputed and redefined” (Head, Reference Head2022: 15). Such policy interventions are often underpinned by contrasting rationales and political agendas, which are subject to change over time, as well as changes in government, public service capacity and implementation process (see Chindarkar et al., Reference Chindarkar, Howlett and Ramesh2017).
The NDIS is arguably an example of a ‘wicked problem’ and complex policy design. It is a deliberate “shift from Australian governments block-funding disability services to a personalised service model, in which packages of funding from a single pool are allocated to eligible individuals, according to their level of need and self-defined goals, to purchase services and support from providers under a range of conditions” (Olney and Dickinson, Reference Olney and Dickinson2019: 275). The complexity of NDIS policy is evidenced by the number of inquiries and reviews, reports and recommendations, public campaigns, incremental reform proposals and ongoing debates on appropriate priorities (see Table 1). Consequently, mapping and understanding the agendas and underlying logics that shape policy reform and how these might change over time is of particular interest.
Policy-agenda researchers identify and analyse periods of punctuated change in the content and style of policy over time. A ‘policy agenda’ is defined as “issues that at any given moment in a society are discussed in public or mass media or promoted by pressure groups or government” (Dowding et al., Reference Dowding, Hindmoor and Martin2013). The study of policy agendas is broad and considers the political and contextual dynamics which shape and reshape agenda-setting and public policy, and the prominence and diffusion of agendas embedded within the policy design itself and how these might fluctuate over time (Baumgartner et al., Reference Baumgartner, Breunig, Grossman, Baumgartner, Breunig and Grossman2019). Understanding the relative attention given to different agendas at any given time, and how these coalesce or not with intended purposes, is core to policy agenda research (Dowding et al., Reference Dowding, Hindmoor and Martin2013; Baumgartner et al., Reference Baumgartner, Breunig, Grossman, Baumgartner, Breunig and Grossman2019). Baumgartner and colleagues (Reference Baumgartner, Breunig, Grossman, Baumgartner, Breunig and Grossman2019) advocate mapping of policy agendas through various data sources, tracking the attention allocated to different issues within government institutions and the policy action itself, and within the broader public domain including the media and understanding fluctuations and their effects. This research commonly has a broad focus (and more recently a comparative focus) on the long-term patterns of issues and the analysis of agenda change, specifically, “the dynamics of how new ideas, new policy proposals, and new understandings of problems may or may not be accepted in the political system” (Baumgartner et al., Reference Baumgartner, Green-Pedersen and Jones2006: 959).
In this article, we particularly draw on the ideas from policy agenda study in Australia by Dowding and colleagues (Reference Dowding, Hindmoor and Martin2013), who argue that the fluctuation of political and budgetary attention is an inherent feature of public policy, and as such agendas can be changeable and unpredictable. They distinguish three related components: policy attention, which is about the prominence of particular issues discussed and debated at any given time by government and others; policy content, which is the intended and actual purpose of policy, including ideological as well as practical purpose; and implementation style, which concerns means and methods of policy (e.g., specific policy instruments and operational frameworks) (Dowding et al., Reference Dowding, Hindmoor and Martin2013: 84). These three can be applied across two levels—the level of policies being discussed (policy agenda) and what the government implements (government activity) (Dowding et al., Reference Dowding, Hindmoor and Martin2013: 85).
Dowding and colleagues (Reference Dowding, Hindmoor and Martin2013) undertake extensive and comprehensive coding according to an established policy agenda programme (PAP) coding framework (developed by Baumgartner and colleagues in the USA). This statistical analysis is described as a starting point which “enables some structural inferences to be drawn about the nature of the policy agenda, interpreting the patterns and explaining why the policy agenda has the general form it does…and why it has the pattern it does in Australia also require detailed historical and qualitative analysis of the processes of agenda formation and change” (Dowding et al., Reference Dowding, Hindmoor and Martin2013: 83). Guided by policy agenda research, the purpose in this article is to undertake an initial mapping of fluctuations in attention given to NDIS principles and issues and what this might tell us about tensions, drivers and contexts that influence the direction of policy change, both incremental and substantial. In doing so, we are not implementing the PAP coding framework of Baumgartner et al. (Reference Baumgartner, Breunig, Grossman, Baumgartner, Breunig and Grossman2019) and Dowding et al. (Reference Dowding, Hindmoor and Martin2013). Rather we are taking this approach as inspiration to apply a text mining approach to gain insight into the Australian policy process pertaining to the shifting nature of the NDIS policy agenda, especially in a fast moving and fast changing context, that can also inform further qualitative research in the broader project.
Policy background of the NDIS
From the outset, the NDIS was a highly ambitious and aspirational social policy reform—the largest Federal social reform in several generations (Productivity Commission, 2011: 72). Introduced following a landmark report of the Australian Productivity Commission (2011), the NDIS was a response to shortcomings and failures of disability care and support in Australia. Specifically, the Productivity Commission report found that support for people with disability and their carers was “inequitable, underfunded, fragmented, insufficient, and gave people with a disability little choice and no certainty of access to appropriate supports” (Productivity Commission, 2011: 5). As part of a broader strategy, the National Disability Strategy 2010–2020, the Commission recommended the introduction of a national scheme for disability care and support that would be based on insurance principles and would provide cover for all Australians with disability (Productivity Commission, 2017: 67). The Commission found that such a scheme would: provide individual and social benefits for many Australians; provide benefits to the broader Australian community and government; and broaden economic participation by people with disability and their carers (Productivity Commission, 2017: 67–71).
The NDIS was designed as a new approach to disability care and support not based on the previous welfare model of capped support. Rather, it is market-based, non-means tested, individualised and person centred, with more choice and control in the hands of participants (Productivity Commission, 2017: 70). Critical to the policy context, the NDIS was also designed to bring the principles and obligations of the CRPD (2006) into Australia’s domestic legislation by providing access to nationally consistent funding and support to help people living with disabilities realise their aspirations, and to participate in the social and economic life of the community. The NDIS legislation directed a focus on the individual rights of people with disability to exercise choice and control over their own lives and receive individualised uncapped funding which was responsive to their own life goals and ambitions.
Individualised funding models, designed to enhance the self-determination of people with disability, be person-centred and facilitate choice and control (Earle and Boucher, Reference Earle and Boucher2020), have also been implemented in disability sectors within the United Kingdom (UK), Ireland, and some European countries (Gadsby et al., 2013; Fleming et al., 2016; Pearson et al., 2018). While the mechanisms, evolution and maturity of individualised funding policy vary across countries (such as structural integration of health and social care systems across the UK), so too do degrees of success and outcomes (Fleming et al., Reference Fleming, McGilloway and Barry2016; Reed et al., Reference Reed, Oung, Davies, Dayan and Scobie2021). The complexity of factors impacting policy drift away from political commitment and reform agendas, has been particularly explored by Needham and Hall (Reference Needham and Hall2022) in the context of social care reforms for older people and people with disability across the UK.
The commitments and reform priorities of the NDIS are articulated in the objects of the National Disability Insurance Scheme Act 2013 (Australian Government, 2016) (‘NDIS Act’) and include supporting independence and social and economic participation of people with disability (s. 3 (1)(c)); providing reasonable and necessary support (s. 3 (1)(d)); enabling people with disability to exercise choice and control in relation to goals, planning and delivery of supports (s. 3 (1)(e)); developing a nationally consistent approach to access, planning and funding of supports (s. 3 (1)(f)); promoting provision of high quality and innovative supports to maximise independent lifestyles and full inclusion in the community (s. 3 (1)(g)); and to give effect to Australian’s obligations under a number of international conventions including the CRPD (s. 3 (1) (a) (i)). In giving effect to these objects, regard is to be given to (amongst other things) the ‘need to ensure the financial sustainability’ of the NDIS (s. 3 (3)(b)). General principles which guide action under the NDIS Act include that people with disability have the same right as other Australians to ‘realise their potential for physical, social, economic and intellectual development’ (s. 4 (1)); they should have certainty of care and support over their lifetime (s. 4 (3)); and should be able to exercise choice including taking reasonable risks, in relation to goals, planning and delivery of support (s. 4 (4)). However, again the NDIS Act states that the exercise of those principles in relation to the NDIS must be made with regard to ‘the need to ensure the financial sustainability’ of the NDIS (s. 4 (17)). Thus, the legislative architecture of the NDIS reflects complex and competing values, priorities and policy agendas.
The objects and principles of the NDIS Act are operationalised through the delivery of both general and ‘reasonable and necessary supports’ to people with disability who become participants in the NDIS. Participants must be under sixty five at the time they request access to the NDIS, must live in Australia and be a citizen or permanent resident, and must have a significant and permanent impairment. The NDIS Act, administered by the National Disability Insurance Agency (NDIA), commenced trial phase in the first half of 2013 in different jurisdictions across Australia. The transition to the full rollout began in most states and territories in July 2016 (Western Australian transitioned from July 2017) and reached full roll-out in mid-2020. As of 31 March 2023, there were 592,059 participants in the NDIS (NDIA, 2023: 6).
The NDIS has seen rapid growth of the scheme since 2013 (both by scheme participants and cost) and simultaneously, a turbulent policy environment with numerous emerging pressure points. Since its inception and creation in 2013, the government of Australia has changed twice, and there have been several changes of government leadership and ministers (see Table 1), bringing different ideologies. Complex policy reform designed to bring large-scale change is often challenging to implement and consequently, requires incremental changes as it evolves. Consistent with this, multiple reviews and inquiries have ensued since the Scheme’s introduction, highlighting policy failures and policy implementation issues (Ernst and Young, 2015; Tune, Reference Tune2019), questions of cost and financial sustainability (Productivity Commission, 2017), and raising concerns about diminishing rights and rationing (Tune, Reference Tune2019). Critical scrutiny of these changing circumstances and agendas is arguably a priority because of the potential erosion of the policy’s original intentions and the impact of this for people with disability. Despite the recognition of people with disability as rights holders, the Australian Government faces an ongoing dilemma about how to ensure their rights and entitlements, while balancing issues of cost and sustainability of the Scheme. We seek to explore whether and how the shifts are reflected across NDIS legislative and early reform documents.
Approach and methodology
A novel analysis approach utilising text mining techniques was applied to selected NDIS publicly available legislative and reform documents to map and understand the underlying principles and agendas emergent at different phases of NDIS policy. Text mining generally refers to the process of extracting data from different written resources—which is useful for identifying patterns and analysing to obtain insights into large-scale textual data (Gupta and Lehal, Reference Gupta and Lehal2009). Such processes facilitate linking together information to form new hypotheses to be further explored by more traditional research approaches (Gupta and Lehal, Reference Gupta and Lehal2009). Automated text-mining techniques can provide an objective quantitative analysis of document elements (Prior, Hughes and Peckham, Reference Prior, Hughes and Peckham2012).
Text mining has seldom been used in policy literature—with some recent applications including understanding public priorities for children’s disability policy (Cogburn et al., Reference Cogburn, Shikako-Thomas and Lai2020), and changes in South Korean environmental policy (Cho and Lee, Reference Cho and Lee2020). However, text-mining has useful applications in its capacity to handle large volumes of text, does not require preliminary development of dictionaries, and has capacity for analysis of both word frequencies as well as cluster and correspondence analysis between concepts (Bicquelet and Weale, Reference Bicquelet and Weale2011: 5). In the current context, the application of text mining processes to NDIS documentation will provide insight into the continuity and change in NDIS policy and reform documents, and to establish a baseline for future analyses.
Document selection
Ten key publicly available NDIS policy and reform documents were included in the analysis (Table 2). These have been organised into five phases and reflect the initial era of NDIS policy development and implementation. The included documents commence from the 2011 Productivity Commission report which predates the NDIS (i.e., Phase 1). Phase 2 consists of NDIS legislation and rules, and Phases 3 to 5 are critical phases of policy adjustment. The progress reports from the Joint Standing Committee on the NDIS make up Phase 3 and represent the two initial reports on the trial period of the NDIS. Phase 4 includes three documents from the Productivity Commission’s review into Scheme costs (a scheduled review as per a 2012 government agreement) to inform the final design of the scheme before full scheme commencement. Phase 5 includes the Joint Standing Committee’s progress report on scheme implementation and administration during the early stages of the full rollout, and the Independent Review of the NDIS Act (commissioned by the government to provide recommendations on how the NDIS legislation could be amended to support the introduction of a Participant Service Guarantee), commonly referred to as the Tune Review. The explicit purpose or terms of reference for each document is shown in Table 3.
NDIS documents included in the analysis
NDIS policy and reform documents – Purpose and Terms of Reference
Although media, public speeches and budgetary information are part of the PAP method that inspired this approach, for pragmatic reasons, the selection of documents for this analysis focused on the formal aspects of policy development and reform, including legislation and government reports.
The text mining analyses applied to these documents sought to determine the prevalence of key content, namely NDIS objectives and principles, and broader policy dialogues. To achieve this, several steps were completed, including pre-processing and preparation of text from PDF documents, analysis, and data visualisation.
Pre-processing and cleaning techniques were applied to the documents to prepare them for the text mining processes. Following conversion of portable document format (PDF) documents into textual data, the following pre-processing techniques were applied: (1) transformation of text to lower case (due to analysis being case sensitive), (2) removal of punctuation, (3) removal of numbers, (4) stripping the document of any white space (i.e., non-text areas) and (5) removal of unessential terms (i.e., stopwords). Stopwords are commonly used English words that hold little analytical meaning (e.g., ‘a’, ‘the’, ‘and’) and are removed from the textual data to ensure inappropriate terms are excluded from clouding findings. A list of standard stopwords were identified and reviewed to ensure NDIS-relevant terms (e.g., ‘necessary’) would be captured in the analysis. Further stopwords specific to NDIS documents were added based on initial word frequencies from all documents—for example, use of ‘minister’ or ‘chapter’ in the NDIS Act. Finally, words were stemmed, returning them to their base form, capturing all word variants (e.g., transparent becomes ‘transparen’ capturing transparent and transparency).
We acknowledge that the relationships between words was not possible using this approach (that is, how words are connected to each other to give greater context of their use). For example, we could not ascertain if ‘carer’ was used in the context of, for example, family as carer, carer support needs, carer burden, or carer advocacy. However, we can interpret that the increased prevalence of terms demonstrates it is an issue receiving policy attention and as per the policy agenda staged approach, the interpretation of policy attention requires further exploration through historical and qualitative research (Dowding et al., Reference Dowding, Hindmoor and Martin2013).
Analysis utilised indicative approaches, in particular keyword frequency analysis. This is achieved when the text is broken into a matrix of single word units for each document (i.e., a term-document matrix is generated). Term-document matrices are then used to generate frequencies, proportions and subsequent data visualisation. In this article, the text mining will generate (1) high frequency key words identified for the combined documents and each phase of policy (Table 4), and (2) key words from a secondary targeted word list for all documents and each document phase (Table 5). This secondary word list was developed by the authors based on the high frequency key word list which was then adapted to specifically explore some of the implementation pressure points and tensions identified in the NDIS policy literature, including for example, sustainability and rights, as well as key NDIS objects and principles. This enabled authors to directly compare specific word use across the phases. Analyses were conducted in R version 4.1.1 (R Core Team, 2021).
Top 30 words and proportion for all documents and each phase of documents
Note. N is number of words under each document group—excludes stopwords.
Proportion of stemmed key terms within all and subgroups of documents
Note. Values are the proportion of terms from the total number of words (n) in each document level, which excludes stopwords.
Colour denotes the density: low (red), medium (yellow) and high (green).
Findings
A descriptive keyword frequency was conducted as a first step to understanding the selected NDIS documents. The top thirty terms identified by the term frequency analysis is presented in Table 4 (see Supplemental 1 for percentage of total words included), having been completed for all NDIS documents and subsequently each phase of documents. Unsurprisingly across all documents, the top six words clearly convey the purpose and design of the Scheme: ‘support’, ‘participant’, ‘plan’, ‘provide’, ‘people’ and ‘fund’, with these largely appearing in the top ten words across each individual phase. ‘Person’- and ‘participant’-centred terminology remained consistent across the document phases (Phases 2-5), with ‘participant’ in the top four words across the phases. There was also a high focus on cost-based terminology including ‘funds’ or ‘costs’ across the document phases, overall appearing as the sixth and eleventh most frequent terms across all documents, respectively.
Initially, in Phase 1—which precedes the introduction of the NDIS—we see terms relating to the previous context leading to the introduction of the NDIS, such as ‘risk’ (#28) and ‘catastrophic’ (#29). Also evident are dominance of ‘community’, ‘disability’, ‘injury’, ‘employ’, ‘individual’, ‘local’, suggesting the social model of disability. In the legislative Phase 2 documents we see higher proportions of process-based language, including ‘decision’, ‘purpose’, ‘requirement’, ‘function’, ‘request’, ‘prescribe’ and ‘manage’. In the review of the trial phase of the Scheme, Phase 3 documents show a shift towards ‘service’, ‘community’, ‘response’, ‘develop’, ‘process’, ‘access’ and ‘family’. Following early full Scheme implementation, cost-based terminology in review documents of Phase 4 was unsurprising, with high proportions of ‘market’, ‘price’, ‘sustainable’ and ‘workforce’, but also the introduction of ‘risk’ and ‘outcome’. Phase 5 demonstrated another shift in attention, ‘review’, ‘health’, ‘response’, ‘service’, ‘family’, ‘community’ ‘process’ and ‘capacity’ all in the top twenty one words.
Comparative variations of terms
Determining the comparative prevalence and fluctuation of specific terms and principles within the document phases was also an area of interest and a means of understanding which principles and issues were influencing various stages of policy and reform over time. Proportionate use of terms is detailed in Table 5, for all documents and for document phases.
Rights-based language
The prevalence of ‘rights’ terminology was extremely low. ‘Right’ appeared in 0.03 per cent across all documents and decreased following Phases 1 and 2 (both 0.06 per cent). Principles such as ‘equity’, ‘fair’ and ‘dignity’ were not strong in any phase of the NDIS documentation, and were absent from several phases, for example, ‘dignity’ did not appear at all in Phases 1, 3 and 5. The term ‘justice’ however was counter to this trend, and although still appearing in low proportions, experienced a slight increase from Phase 1 (0.02 per cent) to Phase 5 (0.07 per cent). None of these terms appeared in Table 4.
Interpersonal relationship language
The proportions of interpersonal relationship terms, including ‘family’, ‘carer’ and ‘community’, experienced similar vacillations over time. All had lowest proportions in Phase 2-legistative documents and second lowest in the Phase 4 costing reports. However, ‘family’ experienced a surge in Phase 5 (0.67 per cent) up from Phase 3 and Phase 1. This is consistent with the data in Table 4 of ‘family’ fifthteenth in Phase 5 and seventeenth in Phase 3. ‘Community’ was highest in Phase 1 (0.86 per cent) and largely steady in Phases 3 and 5 (0.68 per cent and 0.66 per cent respectively). In Table 3, we see that ‘community’ was highest in Phase 3 (#11), followed by Phase 1 (#13) and Phase 5 (#16), but also featured in the top thirty words in Phase 4 (#23). ‘Carer’ however showed the most stability across Phases 1, 3 and 5, consistent with the ranking in Table 4–twenty fourth in Phase 1, twenty second in Phase 3, and twenty seventh in Phase 5.
Decision-making language
The prevalence of decision-making language, including ‘decision’, ‘review’, ‘planner’, ‘flexibility’, ‘consistency’ and ‘transparency’, showed some similarities over the Phases. Overall, these words appeared less in Phase 1 than they did in Phase 5. ‘Review’ and ‘consistency’ all peaked in Phase 5, while the largest fluctuations over time were seen in ‘review’ and ‘decision’. Although highest in Phase 2 (1.31 per cent), ‘decision’ decreased sharply in the following two phases before a strong rise in Phase 5 (1.05 per cent), aligning with Table 4 (#6 in Phase 2 and #9 in Phase 5). Similarly, ‘review’ was second highest in Phase 1 (1.02 per cent; #11), dropped in Phases 2 and 3 and then reached highest proportion in Phase 5 (1.93 per cent; #4). Each of the terms’ prevalence in Phase 2 of government policy on processes to allocate resources is not surprising, however the significant jump in Phase 5 alerts us to concerns. The gradual increase in later Phases of ‘consistency’, the higher proportion of ‘transparency’ in Phases 4 and 5, and the appearance of ‘planner’ in Phases 3, 4 and 5, together indicate a stronger focus on decision-making processes in NDIS implementation.
Cost-based language
The terms ‘cost’ and ‘sustainable’ were highest in Phase 4 (2.67 per cent and 0.59 per cent respectively) which was consistent with the cost-centric reports of this phase (while ‘cost’ appeared as a driver in Phase 1 (1.01 per cent)). Both terms had their lowest proportions in Phase 5 documents (0.17 per cent and 0.04 per cent respectively). These results are also reflected in Table 4, in which ‘sustainable’ only appeared in Phase 4, and ‘cost’ was nineth in Phase 1, eighteenth in Phase 3 and third in Phase 4. Other cost-based language evident in Table 4, included ‘fund’ across all levels; ‘compensation’, ‘debt’ and ‘payment’ in Phase 2 legislative documents; and ‘market’, ‘price’ and ‘financial’ in Phase 4 costing reports.
Discussion
In this study, a novel text mining approach was used to identify the prominence and fluctuations in the NDIS policy agenda (i.e., key principles, issues given attention in policy) over eight years of the development, implementation and early reform of the Scheme. This approach—seldom used in policy studies literature—was inspired by the work of policy agenda researchers but utilised a rapid technique to overview key word prevalence and fluctuation. Findings from the analysis of ten key NDIS policy documents identified a low prevalence of some key NDIS principles, including ‘rights’-based terminology across this period, as well as evidence of fluctuations across the reform period, including attention towards the Scheme’s ‘cost’ and ‘sustainability’, and increasing prominence of interpersonal (e.g., ‘family’ and ‘community’) and decision-making terminology (e.g., ‘decision’, ‘review’ and ‘consistency’).
As per policy agenda theory, high policy attention does not necessarily equate to policy change (Dowding et al., Reference Dowding, Hindmoor and Martin2013), but it does point to issues of significance and give pause to evaluate what may have precipitated these changes as well as reflect on whether and what change did occur and suggest directions for more in-depth research. These patterns and fluctuations borne out of the analyses are no doubt indicative of the complexities of designing large-scale reform and operationalisation of numerous allocative principles (Foster et al., Reference Foster, Henman, Tilse, Fleming, Allen and Harrington2016; Olney and Dickinson, Reference Olney and Dickinson2019). Nonetheless, there are some important observations to consider from these findings, as well as the need to contextualise the findings. First, the low attention on rights in policy documents across the period only adds to the scepticism that rights-based discourse is more often part of the persuasive techniques of government that do not necessarily translate in practice without perseverance and an ongoing commitment (Barnes, Reference Barnes1995). In the case of the NDIS, this under-attention to rights in documentation is significant because unlike other areas of welfare such as social security, where rights are realised once eligibility and compliance conditions are met, or healthcare where there is a right to a basic standard of publicly funded services through Medicare (Foster et al., Reference Foster, Henman, Fleming, Tilse and Harrington2012), the right to disability supports is far more ambiguous, reliant on the interpretation of multiple ‘reasonable and necessary’ criteria which are often in conflict (Venning et al., Reference Venning, Hummell, Foster, Burns and Harris-Rimmer2021; Hummell et al., Reference Hummell, Borg, Foster, Fisher and Needham2023).
Although the NDIS has its anchor in ‘rights’, ‘choice’, ‘dignity’ and other main NDIS principles, there is always the challenge of implementing these in a context of finite resources. The sustained reference to cost and increasing attention to the interpersonal and decision-making content in the findings to some extent reinforce concerns about rights and rationing. This shift in attention to what are arguably more implementation and allocative challenges (that is ‘family’, ‘community’, ‘decision’, ‘review’ and ‘consistency’), is therefore a second important consideration from the findings. Rights and entitlements only become real through administrative decisions – decisions that apply and interpret policy, decisions that weigh up evidence, decisions that grant or deny access, decisions that can be more or less discretionary. From early in the NDIS implementation, there have been concerns expressed about poor decision-making skill in the NDIA and inconsistent application of reasonable and necessary criteria (Olney and Dickinson, Reference Olney and Dickinson2019), fairness and equity of funded supports (Mavromaras et al., Reference Mavromaras, Moskos, Mahuteau, Isherwood, Goode, Walton, Smith, Wei and Flavel2018), lack of transparency of decisions and drastically increasing internal NDIA reviews and external Administrative Appeals Tribunal (AAT) cases (Queensland Advocacy for Inclusion, 2022). Although decision-making is largely an internal process (often informed by algorithms), as is the handling of review of initial decisions (Carney et al., Reference Carney, Then, Bigby, Wiesel and Douglas2019), the AAT provides a particularly important view on areas of contestation in decision-making.
The first NDIS case to the AAT was in 2014. From 2014 to 019 there were thirty one AAT appeals against decisions on reasonable and necessary support, the majority of which were resolved through conciliation rather than hearing (with confidential outcomes) (Venning et al., Reference Venning, Hummell, Foster, Burns and Harris-Rimmer2021). There has been a steady growth in both internal NDIA reviews and external AAT appeals since this time (NDIA, 2022). What this indicates is that there are contestable aspects of the administration of the NDIS (Venning et al., Reference Venning, Hummell, Foster, Burns and Harris-Rimmer2021). Although what constitutes reasonable and necessary support in the NDIS is becoming clearer as the scheme matures, a vital recommendation within the Tune Review (2019) was for the NDIA to make more explicit how reasonable and necessary support decisions are made. There remain unresolved challenges in the interpretation and application of both access and support rules, and ongoing contestation within policy criteria (especially Section 34). This lack of transparency of decision-making within the NDIA and unknown outcomes of appeals that reach settlement prior to an AAT hearing impacts the ability for participants and their advocates to learn from past decisions and prevents the community from holding the NDIA accountable to modify decision-making practice (Carney et al., Reference Carney, Then, Bigby, Wiesel and Douglas2019; Fisher and Hummell, Reference Fisher, Hummell, Brown and Dovers2021).
Also critically impacting decision-making on the allocation of resources in the assessment of reasonable and necessary, is the external context of finite resources, evidenced in our findings thorough the sustained use of ‘cost’ and the high attention to ‘sustainability’ in Phase 4. Downward financial pressures on the administration of the NDIS, which increases future likelihood of tough rationing (Productivity Commission, 2017: 4211–42), is only likely to worsen as the Scheme grows (Ernst and Young, 2015), prompting the Productivity Commission in 2017 to recommend improved clarity, transparency and accountability of the review processes (Rec. 11.5). Increasing government rhetoric on the threat to the financial sustainability of the scheme (NDIA, 2021) raises the likelihood of rationing through the application of more legislative conditions and stricter administrative rules such as capping specific services, limiting options, or use of ‘standard’ packages, which potentially threaten rights and entitlements.
Within this implementation context of heightened policy attention to ‘cost’, ‘sustainability’, ‘decisions’, ‘review’ and ‘consistency’, and associated dilemmas for governance, we can view the government’s response. This is the third contribution from the findings, that is, how particular framings of policy problems shape (proposed) policy solutions. In 2020 the NDIA proposed controversial reforms to the NDIS which would have introduced ‘Independent Assessments’ (IAs – ‘independent’ standardised functional assessments) to be used for access decisions and for determining ‘personalised budgets’ based on automated personas of people with disability with similar functional levels (Joint Standing Committee, 2021). The NDIA rationale for the IA proposal was to redress inconsistencies in decision-making and avoid the unfairness of assessment costs for participants. With funding no longer guided primarily by the goals and aspirations of individual participants, it was argued by the NDIA that the proposed reforms would enable greater overall equity and consistency in the NDIS and respond to increasing cost pressure in the scheme (Department of Social Services, 2020). However, it was strongly rebuffed, consistently met with arguments about inadequate evidence that compulsory IAs for all participants was the most effective way to ensure equity and consistency in NDIS planning (Joint Standing Committee, 2021), and criticising the prioritisation of procedural efficiency and financial objectives ahead of choice, control and individualisation of funding which maximises the rights of people with a disability (Burns et al., Reference Burns, Foster, Harris-Rimmer, Hummell and Venning2021).
Clearly, social policy must manage the unavoidable issue of how to distribute limited resources in a fair and sustainable way, and high-cost areas such as health and disability inevitably involve intense value clashes. Based on the multiple reviews and reports into the NDIS, and issues gaining more attention since its introduction, it is reasonable to propose that a fiscal control and rationing mindset now surround the NDIS and will likely become more obvious with growing economic pressures. The critical issue for the NDIS is how the rights of people with disability are protected and promoted amidst concerns about rationing which may accompany the challenges of cost and sustainability? On a more positive note, following the election of the Labor Government in 2022, the Disability Minsters have brought forward the scheduled 2023 review of the NDIS, with one of the original architects of the Scheme now the NDIS Minister, Bill Shorten. Whether this change heralds NDIS policy renewal and return to original aspirations remains to be seen.
As a research tool to rapidly garner an overview of the policy issues, emphasis and agendas receiving attention in policy documents and debates, this text mining approach could be applied to a wide array of policy areas. For example, comparative analysis on the use of human rights-based language across policy domains of education, aged care, homelessness or mental health, both nationally and internationally, could assist in locating knowledge and implementation gaps, and highlight areas for policy and research priorities. Formal policy documents could additionally be complemented with traditional print media and social media coverage (as used in Alhuzali et al., Reference Alhuzali, Zhang and Ananiadou2022 and Park et al., Reference Park, Mistur, Kim, Mo and Hoefer2022). Especially in a fast moving and fast changing context, the text mining approach enables policy researchers to gain exploratory insights that can inform further qualitative inquiry.
The findings contribute to subsequent phases of the broader study, especially highlighting the focus on exploring the perspectives of NDIS stakeholders (including government officials, advocates, legal representatives, allied health professionals and NDIS Participants) on the role of rights, decisions, cost and sustainability in the development and implementation of the NDIS, including where and why shifts have occurred. Content analysis of AAT administrative decisions from 2014 until 2022, will also pay attention to the above findings regarding the emphasis of various NDIS objects and principles as well as the use of specific reasonable and necessary criteria.
Beyond this project, another avenue for qualitative research within the NDIS, as highlighted by the findings, is an exploration of the role of families and family support in the NDIS, including ways in which family support is represented in NDIS policy, considered in decision making of NDIS funded supports, and the experiences of family members in support decisions and the provision of both formal and informal supports.
While the text mining approach is limited in offering qualitative insights, it also presents further limitations. This approach lacks contextual nuance relating to the connections or relations between words which would elucidate further context—for example, the context of control could be ‘increase participant control’ or ‘remove government control’ or ‘provider control’. This thus limits the utility of the approach in enabling broader contextual considerations that may explain and enhance the findings. Additionally, in this study, the choice of preprocessing techniques, in particular, the development of the secondary list of words (Table 5), has the potential to introduce bias and influence the reliability of results. In acknowledging these limitations, we have interpreted the findings with caution and with a broader understanding of the purpose and context of the documents included in the analysis (see Table 3).
Conclusion
Radical and complex policy reform designed to bring large-scale change is often challenging to implement and, consequently, requires incremental changes as it evolves. Consistent with this, since the introduction of the NDIS there have been ongoing inquiries and reforms to identify and address new challenges. As we have demonstrated with the above findings, these are also accompanied by shifts in policy attention. The article provided insight into these shifting agendas within selected NDIS policy documents from 2011 until 2019, including low prevalence of ‘rights’-based terminology, sustained attention towards the Scheme’s ‘cost’ but fleeting latterly attention to ‘sustainability’, and increasing prominence of interpersonal (e.g., ‘family’, ‘community’) and decision-making terminology (e.g., ‘decision’, ‘review’, ‘consistency’). How these shifting agendas have played out in subsequent debates and reforms was explored and identified a subsequent surge in reforms seeking administrative efficiencies. These findings are a reminder that despite the recognition of people with disability as rights holders, the Australian Government faces an ongoing dilemma about how to design policy that ensures rights and entitlements, while balancing issues of cost, sustainability and consistency. The next phase of work in the broader study will focus on empirical findings that map the institutional logics and operating principles surrounding decisions on reasonable and necessary supports within the NDIS.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1474746424000101.
Financial support
This work was supported by the Australian Research Council (ARC) Discovery Project under Grant DP200100742.
Open access
