Critically ill patients present considerable impairment of nutritional status, due to the systemic repercussions inherent to the clinical condition(Reference Preiser, Ichai and Orban1). Nutritional care is recognised as an important component in the management of these patients, acting as a strategy with a potential impact on clinical outcomes(Reference Casaer and Van den Berghe2). Clinical practice guidelines (CPG) on nutritional care for critically ill patients bring evidence for guiding professionals in making safer and more assertive decisions in clinical practice(Reference Graham, Mancher, Miller Wolman, Greenfield and Steinberg3). Although CPG are considered the main tools for determining more accurate and ethical practices, their methodological quality(Reference Brouwers, Kho and Browman4) and reporting completeness(Reference Chen, Yang and Marušic5) are suboptimal in different fields of health science(Reference Hou, Ge and Wang6–Reference Wei, Shang and Nan9). This precludes the reliability of CPG and limits their applicability in clinical practice.
The methodological quality of CPG and their recommendations can be evaluated using the Appraisal of Guidelines for Research and Evaluation II (AGREE II)(Reference Brouwers, Kho and Browman4) and AGREE-Recommendation Excellence (AGREE-REX) tools, respectively(Reference Brouwers, Spithoff and Kerkvliet10). Across various health science fields, evidence indicates suboptimal methodological quality of CPG(Reference Molino, Leite-Santos and Gabriel11,Reference Gao, Wang and Luo12) , a trend also observed in the nutrition field(Reference Bakaloudi, Papaemmanouil and Vadarlis13–Reference Cattani, Teixeira and Eckert15). Studies evaluating CPG on the nutritional management of cancer, severely burned and critically ill patients classified most of them as having low methodological quality, according to AGREE II. Additionally, most were only recommended with modifications and presented global scores lower than those recommended(Reference Bakaloudi, Papaemmanouil and Vadarlis13–Reference Cattani, Teixeira and Eckert15). A systematic review of critically ill patients conducted by our research group demonstrated that none of the CPG recommendations had a score above 70 %, thus being classified as moderate quality according to AGREE-REX(Reference Cattani, Teixeira and Eckert15). These findings were also demonstrated by other similar publications(Reference Gillespie, Latimer and Walker16,Reference Steeb, Hayani and Förster17) .
Methodological quality is crucial for ensuring the internal validity of studies. Added to this, among the practices of transparency and reproducibility in research, the reporting completeness of publications stands out, regardless of their nature. Accurate, complete and transparent reporting of all health research publications is recommended to support the reproducibility and usefulness of research, in addition to helping minimise avoidable waste of financial and human investments(Reference Miguel, Camerer and Casey18,Reference Gorman and Ferdinand19) . The specific reporting guideline for CPG proposed by the EQUATOR Network is the Health Care Practice Guidelines Reporting Tool (RIGHT)(Reference Chen, Yang and Marušic5). The RIGHT statement was published in 2016 and since then studies that have assessed the reporting completeness of CPG in different fields of health science have shown that the guidelines require improvement(Reference Samaan, Mbuagbaw and Kosa20,Reference Jin, Sanger and Shams21) . In the nutrition field, there is also suboptimal reporting of CPG, although it has not been extensively explored until now. In an international review of nutritional recommendations for type 2 diabetes, adherence to RIGHT was less than 60 % across all CPG evaluated(Reference Hou, Ge and Wang6). Another study assessing CPG on nutrition in critically ill adults found that adherence ranged from 20 to 72 %(Reference Li, Chang and Yue22).
The methodological quality of studies is frequently associated with reporting completeness, irrespective of the publication type(Reference Samaan, Mbuagbaw and Kosa20,Reference Jin, Sanger and Shams21) . The correlation between AGREE II and RIGHT has been scarcely investigated. We identified three studies exploring this aspect and two of them showed a positive correlation, ranging from 0·371 to 0·652, while the other revealed a weak and negative correlation (r –0·250)(Reference Li, Chang and Yue22–Reference Wayant, Cooper and Turner24). Reporting completeness of CPG has been also significantly associated with the organisation that developed the guideline, the version and the year of publication(Reference Yun, Yaolong and Zhao23). Other features are also related to the reporting completeness of studies such as systematic reviews and randomised clinical trials, year of publication, number of authors, presence of a methodologist/statistician in the study’s team, country and funding(Reference Samaan, Mbuagbaw and Kosa20,Reference Jin, Sanger and Shams21) . However, we have not identified studies exploring these features as factors related to CPG reporting completeness.
Assessing the reporting completeness of CPG is crucial for identifying and disclosing major limitations, thereby facilitating their avoidance of updates and/or new publications of the guidelines. Therefore, our objective was to evaluate the reporting completeness of guidelines on nutritional care for critically ill patients, identify associated factors and investigate the correlation between the RIGHT items and the methodological quality of the guidelines evaluated by AGREE II.
Methods
Design and registration
We conducted a meta-research of CPG on the nutrition care of critically ill patients. The protocol for this study was registered on the Open Science Framework (Identifier: DOI 10·17 605/OSF.IO/YUZ9F). This study is a follow-up to a previous systematic review conducted by our research group that aimed to evaluate the methodological quality of these CPG(Reference Cattani, Teixeira and Eckert15).
The reviewers involved in this meta-research possess experience in clinical nutrition, conducting systematic reviews and performing methodological assessments of the literature. Reviewers underwent training, which included reading relevant publications, manuals and elaboration and explanation of each checklist. Additionally, a pilot study involving two CPG was conducted prior to the main review, during which key points were discussed to standardise the application of the assessment tools. Reviewers were instructed to share any relevant supplementary materials with the group to ensure consistent evaluation of information across all reviewers.
Eligibility criteria
We included CPG for nutritional care of critically ill adult patients that employed a classification system to assess the quality of evidence, were published in a peer-reviewed journal or guideline database, represented the last published version of the CPG and included recommendations about nutrition care process steps. CPG exclusively focused on paediatric patient populations were excluded, as were those containing recommendations only for specific types of patients, such as surgical or burn patients. Additionally, position or consensus statements, as well as commentaries summarising evidence from published CPG and recommendations adapted to specific countries, were not eligible for inclusion.
Search strategy
The literature search initially constructed for the previously carried out systematic review(Reference Cattani, Teixeira and Eckert15), aimed at evaluating the quality of CPG in the nutritional care of critically ill patients, was updated in December 2024 for the current meta-research. The search was conducted across databases PubMed (via the National Library of Medicine), Embase (via Elsevier), Cochrane, National Institute of Health and Clinical Excellence (NICE - until its closure in 2022), National Guidelines Clearinghouse - Healthcare Research and Quality Agency (until its closure in 2018), Intercollegiate Guidelines Network and data from the Guideline International Network-bases. Specific keywords were utilised for each database, limited to the last 15 years, with no restrictions on language. The complete electronic search strategy used in PubMed, Embase and Cochrane is presented in online Supplementary Box 1. We also utilised artificial intelligence by consulting ChatGPT to confirm the adequacy of our literature search in identifying all CPG relevant to our study. Our search strategy included guidelines and clinical societies related to intensive care, with a particular focus on Asian societies.
Data selection and extraction
The EndNote reference management software (version X7·17; Thomas Reuters) was utilised to coordinate the selection process. Two trained reviewers (AC and KPZ) independently screened the titles and abstracts and subsequently evaluated the full versions of all potentially relevant records, as well as supporting materials for each included CPG. Any disagreements were resolved by a third reviewer (FMS). In case of doubts regarding developing CPG, authors were contacted via email on three separate occasions. If no response was received, the information available in the publication was considered. General information from CPG was extracted using Google Forms®, exported to Microsoft Office Excel® and independently evaluated by two reviewers (AC and KPZ). In case of disagreements, a third reviewer (FMS) provided appraisal.
Assessment of methodological quality of clinical practice guidelines
Data from most of the included CPG were obtained from the database constructed for our previous publication. For the CPG that have been updated and new CPG, we followed the same methodology used in our first study(Reference Cattani, Teixeira and Eckert15). The AGREE II tool was applied to evaluate the methodological quality of CPG. It consists of twenty-three items distributed across six domains: scope and purpose, stakeholder involvement, rigour of development, clarity and presentation, applicability and editorial independence. Each of the twenty-three items was rated on a seven-point agreement scale, ranging from 1 (totally disagree) and 7 (totally agree). Data extraction was conducted using Google Forms® and exported to Microsoft Office Excel®. It was guided by an electronic form and carried out independently by four reviewers (AC, FMS, KPZ and PPT). Disagreements were discussed, analysed and resolved collaboratively. We did not reevaluate the methodological quality of CPG recommendations by AGREE-REX since it was not covered by the aims of this study.
Assessment of reporting completeness of clinical practice guidelines
We evaluated the reporting completeness using the RIGHT checklist, which comprises twenty-two sections and thirty-five items. These items cover basic information (1–4), background (5–9), evidence (10–12), recommendations (13–15), review and quality assurance (16 and 17), financing and declaration and management of interests (18 and 19) and other information (20–22)(Reference Chen, Yang and Marušic5). We created a more detailed version to precisely identify the points of the guidelines that demonstrated greater or lesser adherence to the evaluated criteria. The online Supplementary Table 1 presents the sixty broken-down questions. Following the evaluation, the items were reintegrated to present the RIGHT checklist in its original version. The application of the RIGHT checklist was conducted using REDCap® by two independent reviewers (AC and KPZ), with disagreements resolved by a third reviewer (FMS).
Data analysis
The data were analysed using the Statistical Package for the Social Sciences (SPSS) version 22·0 and RStudio version 2024·04·2, P values <0·05 were considered as statistically significant. The normality of the domains was assessed using the Shapiro–Wilk test. Means and standard deviations or medians and interquartile ranges (depending on distribution) were computed for each item, domain and overall scores.
Inter-rater agreement was determined using the intraclass correlation coefficient method with a two-way random effects model for each domain and total score. The level of agreement (intraclass correlation coefficient) was categorised according to the cut-off points as follows: minor (0·01–0·20), fair (0·21–0·40), moderate (0·41–0·60), substantial (0·61–0·80) and very good (0·81–1·00)(Reference Kramer and Feinstein25).
The AGREE II score for each domain was calculated by summing all the scores of each reviewer for all items in a domain and then standardised as using the formula: (obtained score – minimum possible score)/(maximum possible score – minimum possible score)(Reference Brouwers, Kho and Browman4). One reviewer aggregated all scores, and each domain score was calculated and presented as a percentage in a results table. Given that the AGREE II manual does not provide cut-off points for interpreting the scores, after assessing the twenty-three items and engaging in comprehensive judgement, the selected CPG were evaluated for quality, followed by a decision on their recommendation for use in clinical practice. For overall quality assessment, mean domain scores were categorised as good (≥80 %), acceptable (60 %–79 %), moderate (40 %–59 %) or low (<40 %), as done in previous publications(Reference Sekercioglu, Al-Khalifah and Ewusie26,Reference Chiappini, Bortone and Galli27) . The following criteria were considered for recommendations: if four of the six domains were categorised as ≥60 %, including domain 3 (rigour of development), the CPG was recommended; if at least two domain scores were categorised above 60 %, the CPG was recommended with modifications and if three of the six domain scores were categorised as less than 30 % or none of the domains were above 60 %, the CPG was not recommended(Reference Sekercioglu, Al-Khalifah and Ewusie26). We opted for these criteria because we believe that the rigour of development is a crucial determinant of CPG quality.
For each sub-item of the RIGHT checklist, we computed the frequency of CPG with each possible answer to estimate the RC. Additionally, we determined the frequency of adherence to the original items of the RIGHT. To assess RC, we assigned scores as follows: ‘yes’ responses were assigned 1 point, ‘no’ and ‘unclear’ responses were assigned 0 points. A weighted score was then calculated for each CPG by summing the squared points dividing by the number of valid questions and multiplied by 100 to express the score as a percentage. We also calculated the mean of RIGHT domains to describe the CPG adherence to this reporting guideline.
We compared the reporting completeness score among CPG grouped according to predefined publication-related factors: type of society (Medical or Nutrition), country of origin (American or European), language (English or other), year of publication (based on the average), number of authors and references (based on the average), presence of a methodologist and/or statistician on the team (based on the roles and responsibilities of each developer described in the section of paper related to the authors affiliations), funding (referred or non-referred) and use of Grading of Recommendations Assessment, Development and Evaluation (GRADE) for grading the level of evidence. We used Student’s t test or one-way ANOVA with Tukey post hoc test for these comparisons.
We also evaluate the correlation (Pearson coefficient was calculated) between RIGHT adherence and methodological quality assessed by AGREE II. The correlation coefficient was classified as very high (0·90 to 1·00 or –0·90 to –1·00), high (0·70 to 0·90 or –0·70 to –0·90), moderate (0·50 to 0·70 or –0·50 to –0·70), low (0·30 to 0·50 or –0·30 to –0·50) or negligible (0·00 to 0·30 or 0·00 to –0·30)(Reference Hinkle, Wiersma and Jurs28).
Results
General characteristics of the clinical practice guidelines
A total of 9441 records were initially identified through database searches, out of which 893 were duplicates. Additionally, eight records were identified through manual searches and a grey literature review. The full texts of thirty-two records were assessed for eligibility, and twenty publications were included, representing eleven CPG. Fig. 1 illustrates the flow chart of CPG selection.
Fig. 1. Flow chart of guidelines selection.
The Spanish Society of Intensive Medicine and Coronary Units and the Spanish Society of Parenteral and Enteral Nutrition (SEMICYUC-SENPE) published ten articles(Reference Mesejo, Vaquerizo Alonso and Acosta Escribano29–Reference Ortiz Leyba, Montejo González and Vaquerizo Alonso38) derived from the same CPG, using a consistent methodology for each article, with each one addressing different stages of the nutrition care process. In contrast, the American Society for Parenteral and Enteral Nutrition (ASPEN) published two articles(Reference McClave, Taylor and Martindale39,Reference Compher, Bingham and McCall40) . The most recent ASPEN publication(Reference Compher, Bingham and McCall40) employed a different methodology than the first publication(Reference McClave, Taylor and Martindale39), as it relied exclusively on RCT as the source of evidence. Therefore, we evaluated the SEMICYUC-SENPE publications as a single CPG and the ASPEN publications as two independent CPG.
The eleven CPG included were published between 2012 and 2024 in eight different countries. The majority are from the USA (n 4)(Reference McClave, Taylor and Martindale39–Reference Reintam Blaser, Starkopf and Alhazzani42), published in English (n 9)(Reference Mesejo, Vaquerizo Alonso and Acosta Escribano29–Reference Guan, Chen and Xu46) and by nutrition societies (n 8)(Reference Mesejo, Vaquerizo Alonso and Acosta Escribano29–41,43–Reference Singer, Blaser and Berger45,Reference Castro, Ribeiro and Matos47) . Most CPG reported funding (n 7)(Reference Mesejo, Vaquerizo Alonso and Acosta Escribano29–Reference Ortiz Leyba, Montejo González and Vaquerizo Alonso38,41,Reference Reintam Blaser, Starkopf and Alhazzani42,Reference Elke, Hartl and Kreymann44–Reference Castro, Ribeiro and Matos47) and used the GRADE system to rate the body of evidence (n 8)(Reference Mesejo, Vaquerizo Alonso and Acosta Escribano29–Reference Compher, Bingham and McCall40,Reference Reintam Blaser, Starkopf and Alhazzani42,Reference Singer, Blaser and Berger45–Reference Lefrant, Hurel and Cano48) . In ten CPG, the names of all authors were described, with the mean number of authors of 20 (ranging from 7(Reference Compher, Bingham and McCall40) to 58(Reference Guan, Chen and Xu46)). Only four CPG had at least one methodologist and/or statistician on the team(Reference McClave, Taylor and Martindale39,Reference Compher, Bingham and McCall40,Reference Reintam Blaser, Starkopf and Alhazzani42,Reference Guan, Chen and Xu46) . Nine CPG listed the references used(Reference Mesejo, Vaquerizo Alonso and Acosta Escribano29–Reference Compher, Bingham and McCall40,Reference Reintam Blaser, Starkopf and Alhazzani42,Reference Elke, Hartl and Kreymann44–Reference Lefrant, Hurel and Cano48) , with the mean number of references in CPG being 270 (ranging from 42(Reference Compher, Bingham and McCall40) to 502(Reference Elke, Hartl and Kreymann44)). The general characteristics of the eligible CPG are detailed in Table 1.
Table 1. General characteristics of selected guidelines
AND, Academy of Nutrition and Dietetics; ASPEN, American Society for Parenteral and Enteral Nutrition; AWMF, Association of the Scientific Medical Societies in Germany; BRASPEN, Sociedade Brasileira de Nutrição Parenteral e Enteral; CSCCM, Chinese Society of Critical Care Medicine; DGEM, German Society for Nutritional Medicine; ESICM, European Society of Intensive Care Medicine; ESPEN, European Society for Clinical Nutrition and Metabolism; GRADE, Grading of Recommendations Assessment, Development and Evaluation; IFA, International Fluid Academy; NR, not reported; SCCM, Society of Critical Care Medicine; SEMICYUC, Sociedad Espanola de Medicina Intensiva, Crítica y Unidades Coronarias; SENPE, Sociedad Española de Nutrición Parenteral y Enteral; SFAR, French Society of Anesthesia and Resuscitation (Société Française d’anesthésie et de réanimation.
Methodological quality of updated guidelines according to AGREE II
The Brazilian Society of Parenteral and Enteral Nutrition (BRASPEN)(Reference Castro, Ribeiro and Matos47) and the European Society for Clinical Nutrition and Metabolism (ESPEN)(Reference Singer, Blaser and Berger45) CPG underwent updates in 2023, prompting a reassessment of their methodological quality using AGREE II. BRASPEN presented an increase in its overall score from 20·8 % to 25·5 %. This improvement can be attributed to the higher scores obtained in Domains 2 (Stakeholder Involvement), 5 (Applicability) and 6 (Editorial Independence) in the updated version. Domains 1 (Scope and Purpose) and 3 (Rigour of Development) experienced a slight reduction, while Domain 4 (Clarity of Presentation) remained the same. The CPG from the Chinese Society of Critical Care Medicine presented a mean overall score for AGREE II equal to 66·9 % and the score for the different domains ranged from 33·3 % in the Domain 5 to 91·7 % in the Domain 7(Reference Guan, Chen and Xu46). Based on this evaluation, the methodological quality was classified as acceptable, and the CPG was considered recommended for clinical practice(Reference Guan, Chen and Xu46).
The overall score for ESPEN CPG remained consistent (54·1 % in the 2019 and 56·3 % in the 2023 version). Although Domain 3 (Rigour of Development) and 6 (Editorial Independence) showed lower scores in the updated version, the other domains demonstrated higher scores. The classification for both CPG remained unchanged. According to AGREE II, the BRASPEN guideline has low quality and is not recommended for clinical practice, while the ESPEN guideline is of moderate quality and recommended with modifications.
Table 2 presents the comprehensive methodological quality of all CPG. The overall AGREE II scores ranged from 23·1 to 74·3 %, with three CPG classified as acceptable quality and recommended for clinical practice. Conversely, three CPG were deemed of low quality, and four were not recommended for clinical practice.
Table 2. Methodological quality of clinical practice guidelines: mean of each domain of AGREE II and the overall assessment
AND, Academy of Nutrition and Dietetics; ASPEN, American Society for Parenteral and Enteral Nutrition; BRASPEN, Sociedade Brasileira de Nutrição Parenteral e Enteral; CSCCM, Chinese Society of Critical Care Medicine; DGEM, German Society for Nutritional Medicine; ESICM, European Society of Intensive Care Medicine; ESPEN, European Society for Clinical Nutrition and Metabolism; IQR, interquartile range; SCCM, Society of Critical Care Medicine; SEMICYUC, Sociedad Espanola de Medicina Intensiva, Crítica y Unidades Coronarias; SENPE, Sociedad Española de Nutrición Parenteral y Enteral; SFAR, French Society of Anesthesia and Resuscitation (Société Française d’anesthésie et de réanimation).
* Updated results in the new version of guidelines.
† New guideline included.
Reporting completeness of clinical practice guidelines
Table 3 presents the frequency of CPG adhering to each item of the RIGHT checklist. All identified CPG appropriately themselves as guidelines in the title (item 1a). However, none CPG adhered to the following six specific items of the RIGHT: reporting on the basic epidemiology of the problem (item 5), describing the use of systematic reviews, their identification and the assessment of risk of bias for CPG development (item 11b), considering factors when formulating the recommendations (14c), conducting an independent review of the draft CPG (item 16), implementing a quality assurance process (item 17) and providing information related to conflicts of interest (item 19b). These findings are illustrated in Fig. 2.
Table 3. Relative frequency of nutrition care guidelines of critically ill patients’ adherent to each item of RIGHT checklist
Fig. 2. Percentage of guidelines adherent to each item of the RIGHT domains.
Table 4 displays the scores of each domain of the RIGHT checklist and the overall reporting completeness score for the included CPG in this meta-research. The highest scores were observed in the ‘Basic information’ and ‘Other information’ domains, while the lowest scores were attributed to the ‘Review and quality assurance’ and ‘Background’ domains. Specifically, reporting completeness scores for the domain ‘Basic information’ ranged from 33·3 %(41,43,Reference Castro, Ribeiro and Matos47,Reference Lefrant, Hurel and Cano48) to 66·7 %(Reference Compher, Bingham and McCall40,Reference Reintam Blaser, Starkopf and Alhazzani42,Reference Elke, Hartl and Kreymann44–Reference Guan, Chen and Xu46) . For the ‘Background’ domain, scores varied between 0 %(Reference Reintam Blaser, Starkopf and Alhazzani42–Reference Singer, Blaser and Berger45,Reference Lefrant, Hurel and Cano48) and 50 %,(Reference McClave, Taylor and Martindale39) and for the ‘Evidence’ domain, completeness scores ranged from 20 %(Reference Mesejo, Vaquerizo Alonso and Acosta Escribano29–Reference Ortiz Leyba, Montejo González and Vaquerizo Alonso38,43–Reference Singer, Blaser and Berger45,Reference Castro, Ribeiro and Matos47,Reference Lefrant, Hurel and Cano48) to 80 %.(Reference Reintam Blaser, Starkopf and Alhazzani42) The ‘Recommendations’ domain had scores ranging from 0 %(Reference Mesejo, Vaquerizo Alonso and Acosta Escribano29–Reference Ortiz Leyba, Montejo González and Vaquerizo Alonso38) to 71·4 %(Reference Reintam Blaser, Starkopf and Alhazzani42). None of CPG achieved full adherence to the ‘Review and quality assurance’ domain, resulting in a score of zero. The ‘Funding, declaration and management of interests’ domain score ranged from 0(Reference Mesejo, Vaquerizo Alonso and Acosta Escribano29–Reference Ortiz Leyba, Montejo González and Vaquerizo Alonso38,43) to 75 %(Reference Guan, Chen and Xu46). Lastly, the ‘Other information’ domain had reporting completeness scores ranging from 0(43,Reference Castro, Ribeiro and Matos47) to 100 %(41). The detailed reporting completeness of nutrition CPG was presented in online Supplementary Box 2.
Table 4. Reporting completeness of clinical practice Guidelines: scores of each domain of RIGHT checklist
AND, Academy of Nutrition and Dietetics; ASPEN, American Society for Parenteral and Enteral Nutrition; BRASPEN, Sociedade Brasileira de Nutrição Parenteral e Enteral; CSCCM, Chinese Society of Critical Care Medicine; DGEM, German Society for Nutritional Medicine; ESICM, European Society of Intensive Care Medicine; ESPEN, European Society for Clinical Nutrition and Metabolism; IQR, interquartile range; SCCM, Society of Critical Care Medicine; SEMICYUC, Sociedad Espanola de Medicina Intensiva, Crítica y Unidades Coronarias; SENPE, Sociedad Española de Nutrición Parenteral y Enteral; SFAR, French Society of Anesthesia and Resuscitation (Société Française d’anesthésie et de réanimation).
The overall reporting completeness score, calculated based on the evaluation of the 60 broken-down sub-items of RIGHT, was 33·5 ± 15·5 %, ranging from 10·2(43) to 54·2 %(Reference Guan, Chen and Xu46). Online Supplementary Table 2 summarises the frequency of CPG that completely reported, reported unclearly or did not report the sixty questions related to the broken-down items of the RIGHT checklist.
Association of reporting completeness and the methodological quality
A significant moderate to strong correlation between mean of RIGHT domains and the reporting completeness score and the Right domains 2 (Stakeholder involvement), 3 (Rigour of development) and 6 (Editorial independence) was observed (online Supplementary Table 3). Also, we observed a strong positive linear correlation between the means of all RIGHT domains and the mean of AGREE II domains (r 0·908; P = 0·0001) (Fig. 3(a)). Additionally, reporting completeness score and the AGREE II domains were strongly and positively correlated (r 0·866; P = 0·0006) (Fig. 3(b)).
Fig. 3. Correlation between the AGREE II and RIGHT. (a) Mean of RIGHT domains and (b) mean of RIGHT pondered score.
The reporting completeness score significantly differed between CPG grouped according to methodological quality, being higher in these classified as acceptable or moderate quality in comparison to CPG with low quality. In accordance with this finding, it was also higher in in CPG recommended or recommended with modifications for clinical practice compared to those not recommended, as demonstrated in Table 5.
Table 5. Association between reporting completeness score and predefined characteristics of nutritional care guidelines for critically ill patients
Data are presented as mean ± sd.
* The weighted RIGHT score was calculated for each CPG by summing the squared points, dividing by the number of valid questions and multiplying by 100 to express the score as a percentage.
† One-way ANOVA with Tukey’s post hoc test. Equal subscript letters indicate non-significant differences and different subscript letters indicate P value <0.05.
‡ Student’s t test.
Other factors related to the reporting completeness of clinical practice guidelines
Table 5 presents the reporting completeness score based on the predefined characteristics of CPG. CPG that involved a methodologist and/or statistician on the development team exhibited higher reporting completeness scores compared to those developed without their involvement. It was also higher in CPG published in English in comparison to these published in other languages. The reporting completeness score was significantly higher in CPG that disclosed information about funding compared to those that did not (38·3 ± 12·5 v. 11·9 ± 2·5, P = 0·0003), regardless of the presence of funding.
Discussion
Key results
This meta-research evaluated eleven CPG on nutrition care of critically ill patients, yielding as key findings: (1) there is substantial room for improvement in reporting completeness in this field, as the reporting completeness score based on RIGHT checklist was below 55 % for all CPG; (2) a strong correlation between reporting completeness and methodological quality of CPG was observed, indicating that a reporting completeness is closely linked to high methodological quality; and (3) CPG involving a methodologist and/or statistician in the development team demonstrated higher RC as well as those published in English language and that disclosure funding.
Interpretation and comparison of results with other studies
The reporting completeness of CPG in health care has been scarcely explored, and the results are heterogeneous among the available publications assessing the CPG adherence to the RIGHT(Reference Hou, Ge and Wang6–Reference Zhang, Cao and Wang8,Reference Yun, Yaolong and Zhao23) . In a systematic search evaluating the reporting completeness of 539 CPG in traditional Chinese medicine, 92·9 % of them presented a reporting completeness score lower than 50 %(Reference Yun, Yaolong and Zhao23). Other systematic searches on CPG for cancer pain included 18 CPG and have demonstrated an average reporting rate of all CPG equal to 57·46 %(Reference Zhang, Cao and Wang8). A study of 26 CPG on insomnia showed a final average of the reporting rate of RIGHT items equal to 67·87 %, ranging from 33·33 to 97·14 %(Reference Zhu, Li and Pan7). In the nutrition field, we identified two studies assessing the reporting completeness of CPG: one study of 15 CPG on Nutritional Recommendations for Type 2 Diabetes with reporting completeness below 60 % in all of them(Reference Hou, Ge and Wang6). Another study of ten CPG for nutrition in critically ill adults with a mean score of RIGHT domains ranging from 20 to 70 %(Reference Li, Chang and Yue22). Not all studies present an overall mean reporting completeness score for the evaluated CPG, and among those that do, this score ranges widely. The calculation of the reporting completeness score can partially explain the differences observed since some studies graded the reporting as ‘reported’, ‘partially reported’ or ‘unreported’ and attributed punctuation for the three categories(Reference Hou, Ge and Wang6,Reference Yun, Yaolong and Zhao23) whereas we followed the definitions of RIGHT and graded the reporting as yes, no or unclear and have not given punctuation for unclear answers. The approach of scoring three reporting categories instead of two may overestimate the final completeness reporting score. Nonetheless, it is evident that the completeness of CPG reporting across different fields remains low and needs improvement.
The domains with the lowest rate of adherent CPG (ranging from 0 to 50 % in all CPG) in the current meta-research were ‘Background’ (domain 2) and ‘Review and Quality Assurance’ (domain 5). Other studies also demonstrated the lowest rate of RIGHT adherence in the same domains(Reference Hou, Ge and Wang6,Reference Yun, Yaolong and Zhao23) . The CPG document should include a summary of current key data related to the disease and describe the aims of the CPG and specific objectives and the primary population. This information helps the reader to understand the disease burden, priorities and rationale for developing the CPG and decide to what extent it is relevant(49). Describing who were the contributors to CPG development and their roles and responsibilities is also essential, since disciplines represented within a guideline development group have considerable influence on the recommendations(Reference Leape, Park and Kahan50). Indeed, we demonstrated that the presence of a methodologist/statistician in the development of a CPG team was related to higher reporting completeness score. Indeed, external review promotes scientific accuracy, clarity and the usefulness of the CPG. To ensure transparency, the detailed process of external review should be reported in the CPG(49). CPG developers should also report any potential influence of funders during the development process and describe how this may have affected the recommendations, since biased CPG can have serious implications for health care and patient outcomes(Reference Levinsky51).
The domain ‘Evidence’ (domain 3) presented an adherence rate between 20 and 80 % in our study and a large range of adherence to this domain was also observed in other studies including CPG on nutrition recommendations for diabetes (37·5–75 %), cancer pain (0–100 %) and insomnia (0–100 %). CPG should be based on systematic reviews with meta-analysis conducted to answer each question constructed according to the PICO acronyms. In addition, the certainty of the body of evidence needs to be graded using an established and standardised approach, such as GRADE. It allows the health professional and the other stakeholders to make decisions in the clinical practice(Reference Schünemann, Brożek and Guyatt52). A large range of adherence to the domain ‘Recommendations’ was also obtained for CPG in nutrition care of critically ill patients (0–71·4 %) and in other health fields (0–100 %)(Reference Hou, Ge and Wang6–Reference Zhang, Cao and Wang8). Recommendations should be explicit and unambiguous and presented with its strength to inform the users about the balance between the desirable and undesirable consequences of implementing the recommendation. Considering that many factors can influence the feasibility of implementing the recommendation (such as available resources, existing and necessary infrastructure, and health worker expertise) it is crucial that these points be discussed in the CPG. Aiming to ensure transparency in the CPG development, it is also essential that the approaches used by the group to formulate the recommendations (such as how consensus was defined and achieved and whether voting was used) be reported(Reference Samaan, Mbuagbaw and Kosa20,Reference Jin, Sanger and Shams21) .
The rate of adherence to the domain ‘Other information’ ranged from 0 to 100 % in our study, and a large range of adherence was also reported in other health fields(Reference Zhu, Li and Pan7,Reference Zhang, Cao and Wang8) . Material availability is a well-recognised practice of transparency(Reference Serghiou, Contopoulos-Ioannidis and Boyack53) and all documents related to the CPG should be included as appendices or be available in any repository since they help users to understand the development process and to implement the recommendations(49). Even the best-developed CPG can have limitations, and reporting those limitations increases its trustworthiness. It is particularly important to discuss whether limitations might have influenced the recommendations.
Factors related to the reporting completeness of guidelines
Our study found a strong positive correlation between adherence to the RIGHT and the methodological quality of the CPG evaluated, especially due to a significant correlation with the domains ‘Stakeholder involvement’, ‘Rigour of development’ and ‘Editorial Independence’. Similar results were found in other studies(Reference Li, Chang and Yue22,Reference Yun, Yaolong and Zhao23) . This can be explained by the overlapping of AGREE II and RIGHT. According to a study, eleven items from AGREE II completely matched with twelve items from RIGHT, and three items in AGREE II were partially covered by three items in RIGHT(Reference Yao, Ma and Wang54). A similar study on guidelines for nutrition in critically ill patients also demonstrated a positive correlation between the RIGHT report rate and the AGREE II score (r 0·652)(Reference Li, Chang and Yue22). Conversely, a study evaluating CPG in traditional Chinese medicine showed a weak correlation between RIGHT and AGREE II (r 0·371)(Reference Yun, Yaolong and Zhao23). Studies assessing CPG for nutritional care in various clinical conditions classified most as having low methodological quality according to AGREE II(Reference Bakaloudi, Papaemmanouil and Vadarlis13–Reference Cattani, Teixeira and Eckert15,Reference Wayant, Cooper and Turner24,Reference Noyahr, Tatucu-Babet and Chapple55) . Although reporting completeness in the field of nutrition has been scarcely explored, it is generally poor. The similarity between several RIGHT and AGREE items may explain this observed correlation, as mentioned earlier.
The presence of a methodologist and/or statistician on the team was also associated with high values of reporting completeness scores. It is well-known that these professionals can promote a more systematic, transparent and reproducible development of CPG, which can give the report fluidity and, consequently, greater agreement with the items proposed by RIGHT. The RIGHT checklist was published in 2016, and this may suggest greater attention to the topic after this year and justify the tendency towards greater adherence to reporting completeness from then on. Overall, enhanced reporting completeness of studies, regardless of their design, is demonstrated after the specific reporting CPG publication(Reference Page and Moher56,Reference Tan, Tan and Li57) . Disclosure of funding was also associated with a higher completeness reporting score, regardless of whether funding was received or not. This finding reinforces the alignment between various practices promoting transparency and reproducibility. We could not identify an explanation for the observed association between the language of publication and the completeness reporting score, as it did not reflect the country or society responsible for publishing the CPG.
Strengths, limitations and implications
We arbitrarily chose not to include guidelines published more than 15 years ago in our meta-research, as we believe they would not reflect recommendations applicable to current clinical practice. Probably, this criterion did not lead to the exclusion of a representative sample of guidelines in this field. On the other hand, a strength of our study is the methodological rigour. Although a previous study has already demonstrated low adherence to the RIGHT domains among CPG on critically ill patients, some methodological aspects differ from our study and are highlighted: (1) the publication was in 2022, prior to the ESPEN and Chinese Society of Critical Care Medicine update; (2) the authors restricted the language to Chinese and English, which excluded BRASPEN and French Society of Anesthesia and Resuscitation from the selection; (3) the authors included one CPG specific to surgical patients and one with regional recommendations for India, which were not eligible for us; (4) CPG with more than 15 years of publication were considered, which determined the inclusion of two CPG which were not considered by our group due to the outdated recommendations; and (5) the authors included only one publication of SEMICYUC-SENPE related to macronutrients and micronutrients requirements(Reference Bonet Saris, Márquez Vácaro and Serón Arbeloa32). These differences highlight that our meta-research is broader and, consequently, adds greater robustness while updating the evidence in this field.
Considering that healthcare professionals use the CPG to guide their decisions in clinical practice, and it can determine the nutrition care process in ICU. It is essential for CPG to exhibit high methodological quality and provide clear, concise and complete reporting. Our findings show substantial room for improvement in reporting completeness in this field. CPG developers should utilise both AGREE II and RIGHT, considering the overlapping of items, to elaborate news or update old versions of CPG. It is important to clarify that our aim was not to criticise the existing CPG but rather to critically appraise their development and reporting, emphasising that the AGREE II and RIGHT tools are not intended to assess the clinical applicability of guidelines. We hope that our findings will contribute to the future development of more rigorous, high-quality CPG, as health professionals need guidelines that are straightforward, up-to-date, reliable and accessible.
Conclusions
The reporting completeness in CPG for critically ill adult patients was low, with a strong correlation with the methodological quality assessed by AGREE II. High values of reporting completeness scores were observed between CPG recommended by AGREE II and in those including a methodologist/statistician in the development team highlighting the importance of the rigour of development in the CPG construction.
Supplementary material
For supplementary material/s referred to in this article, please visit https://doi.org/10.1017/S000711452500011X.
Acknowledgements
We are grateful to Paula Portal Teixeira for your involvement in the AGREE II application in the updated CPG included in this meta-research. K. P. Z. received a scholarship as a Doctorate Student from the Coordination for the Improvement of Higher Education Personnel. F. M. S. received a productivity scholarship from the Brazilian National Council for Scientific and Technological Development.
The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
F. M. S. and K. P. Z. contributed to the conception of the study. A. C., F. M. S. and K. P. Z., contributed to the acquisition, analysis and interpretation of data. F. M. S. and K. P. Z. drafted the manuscript. All authors critically revised the manuscript, provided their final approval and agreed to be accountable for all aspects of the work, ensuring its integrity and accuracy.
The authors declare that they have no conflict of interest.
