skip to content
Identifying signals for updating systematic reviews : a comparison of two methods Preview this item
ClosePreview this item
Checking...

Identifying signals for updating systematic reviews : a comparison of two methods

Author: Paul G Shekelle; United States. Agency for Healthcare Research and Quality.; Southern California Evidence-Based Practice Center/RAND.; Tufts Evidence-based Practice Center.; University of Ottawa Evidence-based Practice Center.
Publisher: Rockville, MD : Agency for Healthcare Research and Quality, [2011]
Series: Methods research report.; AHRQ publication, no. 11-EHC042-EF.
Edition/Format:   eBook : Document : National government publication : English
Database:WorldCat
Summary:
BACKGROUND: Methods of assessing the need for systematic reviews to be updated have been published, but agreement among them is unclear. OBJECTIVES: To compare two methods for assessing the need to update an evidence review, using three evidence reports on the effects of omega-3 fatty acids on cancer, cognition and aging, and cardiovascular diseases (with separate analyses for fish oil and alpha-linolenic acid). The  Read more...
Rating:

(not yet rated) 0 with reviews - Be the first.

Subjects
More like this

 

Find a copy online

Links to this item

Find a copy in the library

&AllPage.SpinnerRetrieving; Finding libraries that hold this item...

Details

Genre/Form: Comparative Study
Material Type: Document, Government publication, National government publication, Internet resource
Document Type: Internet Resource, Computer File
All Authors / Contributors: Paul G Shekelle; United States. Agency for Healthcare Research and Quality.; Southern California Evidence-Based Practice Center/RAND.; Tufts Evidence-based Practice Center.; University of Ottawa Evidence-based Practice Center.
OCLC Number: 774949082
Notes: "June 2011."
Description: 1 online resource (PDF file (various pagings)) : illustrations.
Series Title: Methods research report.; AHRQ publication, no. 11-EHC042-EF.
Responsibility: prepared for Agency for Healthcare Research and Quality, U.S. Department of health and Human Services ; prepared by Paul G. Shekelle [and others].

Abstract:

BACKGROUND: Methods of assessing the need for systematic reviews to be updated have been published, but agreement among them is unclear. OBJECTIVES: To compare two methods for assessing the need to update an evidence review, using three evidence reports on the effects of omega-3 fatty acids on cancer, cognition and aging, and cardiovascular diseases (with separate analyses for fish oil and alpha-linolenic acid). The RAND method combines a targeted literature search with the assessments of content experts. The Ottawa method relies on a quantitative and qualitative assessment of the study results from a similar targeted search. DATA SOURCES: A MEDLINE search was conducted on a limited set of journals, including five pivotal general medical journals and a small number of specialty journals, from 1 year prior to release of the original reports using their search strategies. METHODS: The search results were screened using the original eligibility criteria. Study-level data and findings of existing systematic reviews, randomized controlled trials, and large observational studies addressing the original key questions were abstracted. Using the RAND method, we contacted experts--including members of the original technical expert panels and the original peer reviewers--and sought their opinions regarding the status of the original reports and any new references. The results of the literature reviews and expert opinions were combined to determine the need for updating based on predetermined criteria. Using a modification of the Ottawa method, new trial data were meta-analyzed with the original meta-analysis results. A quantitative signal for the need to update was based on statistical differences with the original meta-analyses. Qualitative signals, such as differences in characterizations of effectiveness, new information about harm, and caveats about the previously reported findings, were sought for outcomes without existing meta-analyses. Agreement between the RAND and Ottawa methods was assessed for each report with the kappa statistic. RESULTS: Overall agreement between the two methods ranged from "nonexistent" (kappa = 0.19, for fish oil and cardiovascular disease) to "almost perfect" (kappa = 1.0 for cognitive function). Many of the disagreements between the methods were due to a situation where the original review had a Key Question with no evidence and some evidence was identified in the update. In these situations, the RAND method produced a positive signal for updating and Ottawa's method produced a negative signal. A sensitivity analysis that reclassified these situations as agreement between the two methods yielded much better estimates of agreement: for three of the four conditions, agreement was "substantial" to "almost perfect" and overall agreement was "substantial." CONCLUSIONS: The RAND method and the modified Ottawa method agree reasonably well in their assessment of the need to update reviews. Both methods alone or in combination may be considered as appropriate tools. Future research would confirm these conclusions for a larger cohort of reviews and assess the predictive validity of the methods with actual updates.

Reviews

User-contributed reviews
Retrieving GoodReads reviews...
Retrieving DOGObooks reviews...

Tags

Be the first.

Similar Items

Related Subjects:(2)

User lists with this item (1)

Confirm this request

You may have already requested this item. Please select Ok if you would like to proceed with this request anyway.

Linked Data


Primary Entity

<http://www.worldcat.org/oclc/774949082> # Identifying signals for updating systematic reviews : a comparison of two methods
    a schema:CreativeWork, schema:MediaObject, schema:Book ;
   library:oclcnum "774949082" ;
   library:placeOfPublication <http://id.loc.gov/vocabulary/countries/mdu> ;
   library:placeOfPublication <http://experiment.worldcat.org/entity/work/data/1077182144#Place/rockville_md> ; # Rockville, MD
   schema:about <http://experiment.worldcat.org/entity/work/data/1077182144#Topic/review_literature_as_topic> ; # Review Literature as Topic
   schema:about <http://experiment.worldcat.org/entity/work/data/1077182144#Topic/meta_analysis_as_topic> ; # Meta-Analysis as Topic
   schema:bookFormat schema:EBook ;
   schema:contributor <http://viaf.org/viaf/152834782> ; # Tufts Evidence-based Practice Center.
   schema:contributor <http://viaf.org/viaf/5649149919442706650003> ; # University of Ottawa Evidence-based Practice Center.
   schema:contributor <http://viaf.org/viaf/159631757> ; # Southern California Evidence-Based Practice Center/RAND.
   schema:contributor <http://viaf.org/viaf/129508251> ; # United States. Agency for Healthcare Research and Quality.
   schema:contributor <http://viaf.org/viaf/6961062> ; # Paul G. Shekelle
   schema:datePublished "2011" ;
   schema:description "BACKGROUND: Methods of assessing the need for systematic reviews to be updated have been published, but agreement among them is unclear. OBJECTIVES: To compare two methods for assessing the need to update an evidence review, using three evidence reports on the effects of omega-3 fatty acids on cancer, cognition and aging, and cardiovascular diseases (with separate analyses for fish oil and alpha-linolenic acid). The RAND method combines a targeted literature search with the assessments of content experts. The Ottawa method relies on a quantitative and qualitative assessment of the study results from a similar targeted search. DATA SOURCES: A MEDLINE search was conducted on a limited set of journals, including five pivotal general medical journals and a small number of specialty journals, from 1 year prior to release of the original reports using their search strategies. METHODS: The search results were screened using the original eligibility criteria. Study-level data and findings of existing systematic reviews, randomized controlled trials, and large observational studies addressing the original key questions were abstracted. Using the RAND method, we contacted experts--including members of the original technical expert panels and the original peer reviewers--and sought their opinions regarding the status of the original reports and any new references. The results of the literature reviews and expert opinions were combined to determine the need for updating based on predetermined criteria. Using a modification of the Ottawa method, new trial data were meta-analyzed with the original meta-analysis results. A quantitative signal for the need to update was based on statistical differences with the original meta-analyses. Qualitative signals, such as differences in characterizations of effectiveness, new information about harm, and caveats about the previously reported findings, were sought for outcomes without existing meta-analyses. Agreement between the RAND and Ottawa methods was assessed for each report with the kappa statistic. RESULTS: Overall agreement between the two methods ranged from "nonexistent" (kappa = 0.19, for fish oil and cardiovascular disease) to "almost perfect" (kappa = 1.0 for cognitive function). Many of the disagreements between the methods were due to a situation where the original review had a Key Question with no evidence and some evidence was identified in the update. In these situations, the RAND method produced a positive signal for updating and Ottawa's method produced a negative signal. A sensitivity analysis that reclassified these situations as agreement between the two methods yielded much better estimates of agreement: for three of the four conditions, agreement was "substantial" to "almost perfect" and overall agreement was "substantial." CONCLUSIONS: The RAND method and the modified Ottawa method agree reasonably well in their assessment of the need to update reviews. Both methods alone or in combination may be considered as appropriate tools. Future research would confirm these conclusions for a larger cohort of reviews and assess the predictive validity of the methods with actual updates."@en ;
   schema:exampleOfWork <http://worldcat.org/entity/work/id/1077182144> ;
   schema:genre "Government publication"@en ;
   schema:genre "Comparative Study"@en ;
   schema:genre "National government publication"@en ;
   schema:inLanguage "en" ;
   schema:isPartOf <http://experiment.worldcat.org/entity/work/data/1077182144#Series/ahrq_publication> ; # AHRQ publication ;
   schema:isPartOf <http://experiment.worldcat.org/entity/work/data/1077182144#Series/methods_research_report> ; # Methods research report.
   schema:name "Identifying signals for updating systematic reviews : a comparison of two methods"@en ;
   schema:productID "774949082" ;
   schema:publication <http://www.worldcat.org/title/-/oclc/774949082#PublicationEvent/rockville_md_agency_for_healthcare_research_and_quality_2011> ;
   schema:publisher <http://experiment.worldcat.org/entity/work/data/1077182144#Agent/agency_for_healthcare_research_and_quality> ; # Agency for Healthcare Research and Quality
   schema:url <https://www.ncbi.nlm.nih.gov/books/NBK56774> ;
   schema:url <http://www.ncbi.nlm.nih.gov/books/NBK56774> ;
   wdrs:describedby <http://www.worldcat.org/title/-/oclc/774949082> ;
    .


Related Entities

<http://experiment.worldcat.org/entity/work/data/1077182144#Agent/agency_for_healthcare_research_and_quality> # Agency for Healthcare Research and Quality
    a bgn:Agent ;
   schema:name "Agency for Healthcare Research and Quality" ;
    .

<http://experiment.worldcat.org/entity/work/data/1077182144#Series/ahrq_publication> # AHRQ publication ;
    a bgn:PublicationSeries ;
   schema:hasPart <http://www.worldcat.org/oclc/774949082> ; # Identifying signals for updating systematic reviews : a comparison of two methods
   schema:name "AHRQ publication ;" ;
    .

<http://experiment.worldcat.org/entity/work/data/1077182144#Series/methods_research_report> # Methods research report.
    a bgn:PublicationSeries ;
   schema:hasPart <http://www.worldcat.org/oclc/774949082> ; # Identifying signals for updating systematic reviews : a comparison of two methods
   schema:name "Methods research report." ;
   schema:name "Methods research report" ;
    .

<http://experiment.worldcat.org/entity/work/data/1077182144#Topic/meta_analysis_as_topic> # Meta-Analysis as Topic
    a schema:Intangible ;
   schema:name "Meta-Analysis as Topic"@en ;
    .

<http://experiment.worldcat.org/entity/work/data/1077182144#Topic/review_literature_as_topic> # Review Literature as Topic
    a schema:Intangible ;
   schema:name "Review Literature as Topic"@en ;
    .

<http://viaf.org/viaf/129508251> # United States. Agency for Healthcare Research and Quality.
    a schema:Organization ;
   schema:name "United States. Agency for Healthcare Research and Quality." ;
    .

<http://viaf.org/viaf/152834782> # Tufts Evidence-based Practice Center.
    a schema:Organization ;
   schema:name "Tufts Evidence-based Practice Center." ;
    .

<http://viaf.org/viaf/159631757> # Southern California Evidence-Based Practice Center/RAND.
    a schema:Organization ;
   schema:name "Southern California Evidence-Based Practice Center/RAND." ;
    .

<http://viaf.org/viaf/5649149919442706650003> # University of Ottawa Evidence-based Practice Center.
    a schema:Organization ;
   schema:name "University of Ottawa Evidence-based Practice Center." ;
    .

<http://viaf.org/viaf/6961062> # Paul G. Shekelle
    a schema:Person ;
   schema:familyName "Shekelle" ;
   schema:givenName "Paul G." ;
   schema:name "Paul G. Shekelle" ;
    .

<http://www.worldcat.org/title/-/oclc/774949082>
    a genont:InformationResource, genont:ContentTypeGenericResource ;
   schema:about <http://www.worldcat.org/oclc/774949082> ; # Identifying signals for updating systematic reviews : a comparison of two methods
   schema:dateModified "2017-09-03" ;
   void:inDataset <http://purl.oclc.org/dataset/WorldCat> ;
    .


Content-negotiable representations

Close Window

Please sign in to WorldCat 

Don't have an account? You can easily create a free account.