skip to content
Sniff tests in economics : aggregate distribution of their probability values and implications for publication bias Preview this item
ClosePreview this item
Checking...

Sniff tests in economics : aggregate distribution of their probability values and implications for publication bias

Author: Christopher Snyder; Ran Zhuo; National Bureau of Economic Research,
Publisher: Cambridge, Mass. : National Bureau of Economic Research, 2018.
Series: Working paper series (National Bureau of Economic Research), no. 25058.
Edition/Format:   eBook : Document : EnglishView all editions and formats
Summary:
The increasing demand for rigor in empirical economics has led to the growing use of auxiliary tests (balance, specification, over-identification, placebo, etc.) supporting the credibility of a paper's main results. We dub these "sniff tests" because standards for passing are subjective and rejection is bad news for the author. Sniff tests offer a new window into publication bias since authors prefer them to be  Read more...
Rating:

(not yet rated) 0 with reviews - Be the first.

Subjects
More like this

Find a copy online

Links to this item

Find a copy in the library

&AllPage.SpinnerRetrieving; Finding libraries that hold this item...

Details

Material Type: Document, Internet resource
Document Type: Internet Resource, Computer File
All Authors / Contributors: Christopher Snyder; Ran Zhuo; National Bureau of Economic Research,
OCLC Number: 1054234565
Notes: "September 2018"
Description: 1 online resource (51 pages) : illustrations.
Series Title: Working paper series (National Bureau of Economic Research), no. 25058.
Responsibility: Christopher Snyder, Ran Zhuo.

Abstract:

The increasing demand for rigor in empirical economics has led to the growing use of auxiliary tests (balance, specification, over-identification, placebo, etc.) supporting the credibility of a paper's main results. We dub these "sniff tests" because standards for passing are subjective and rejection is bad news for the author. Sniff tests offer a new window into publication bias since authors prefer them to be insignificant, the reverse of standard statistical tests. Collecting a sample of nearly 30,000 sniff tests across 60 economics journals, we provide the first estimate of their aggregate probability-value (p-value) distribution. For the subsample of balance tests in randomized controlled trials (for which the distribution of p-values is known to be uniform absent publication bias, allowing reduced-form methods to be employed) estimates suggest that 45% of failed tests remain in the "file drawer" rather than being published. For the remaining sample with an unknown distribution of p-values, structural estimates suggest an even larger file-drawer problem, as high as 91%. Fewer significant sniff tests show up in top-tier journals, smaller tables, and more recent articles. We find no evidence of author manipulation other than a tendency to overly attribute significant sniff tests to bad luck.

Reviews

User-contributed reviews
Retrieving GoodReads reviews...
Retrieving DOGObooks reviews...

Tags

Be the first.
Confirm this request

You may have already requested this item. Please select Ok if you would like to proceed with this request anyway.

Linked Data


Primary Entity

<http://www.worldcat.org/oclc/1054234565> # Sniff tests in economics : aggregate distribution of their probability values and implications for publication bias
    a schema:Book, schema:CreativeWork, schema:MediaObject ;
    library:oclcnum "1054234565" ;
    library:placeOfPublication <http://id.loc.gov/vocabulary/countries/mau> ;
    schema:about <http://experiment.worldcat.org/entity/work/data/5456313276#Topic/economics_research_methodology> ; # Economics--Research--Methodology
    schema:about <http://experiment.worldcat.org/entity/work/data/5456313276#Topic/economics_literature_evaluation> ; # Economics literature--Evaluation
    schema:about <http://experiment.worldcat.org/entity/work/data/5456313276#Topic/economics_research_evaluation> ; # Economics--Research--Evaluation
    schema:author <http://experiment.worldcat.org/entity/work/data/5456313276#Person/zhuo_ran> ; # Ran Zhuo
    schema:author <http://experiment.worldcat.org/entity/work/data/5456313276#Person/snyder_christopher_christopher_mark> ; # Christopher Mark Snyder
    schema:bookFormat schema:EBook ;
    schema:datePublished "2018" ;
    schema:description "The increasing demand for rigor in empirical economics has led to the growing use of auxiliary tests (balance, specification, over-identification, placebo, etc.) supporting the credibility of a paper's main results. We dub these "sniff tests" because standards for passing are subjective and rejection is bad news for the author. Sniff tests offer a new window into publication bias since authors prefer them to be insignificant, the reverse of standard statistical tests. Collecting a sample of nearly 30,000 sniff tests across 60 economics journals, we provide the first estimate of their aggregate probability-value (p-value) distribution. For the subsample of balance tests in randomized controlled trials (for which the distribution of p-values is known to be uniform absent publication bias, allowing reduced-form methods to be employed) estimates suggest that 45% of failed tests remain in the "file drawer" rather than being published. For the remaining sample with an unknown distribution of p-values, structural estimates suggest an even larger file-drawer problem, as high as 91%. Fewer significant sniff tests show up in top-tier journals, smaller tables, and more recent articles. We find no evidence of author manipulation other than a tendency to overly attribute significant sniff tests to bad luck."@en ;
    schema:exampleOfWork <http://worldcat.org/entity/work/id/5456313276> ;
    schema:inLanguage "en" ;
    schema:isPartOf <http://experiment.worldcat.org/entity/work/data/5456313276#Series/nber_working_paper_series> ; # NBER working paper series ;
    schema:isPartOf <http://experiment.worldcat.org/entity/work/data/5456313276#Series/working_paper_series_national_bureau_of_economic_research> ; # Working paper series (National Bureau of Economic Research) ;
    schema:name "Sniff tests in economics : aggregate distribution of their probability values and implications for publication bias"@en ;
    schema:productID "1054234565" ;
    schema:publisher <http://experiment.worldcat.org/entity/work/data/5456313276#Organization/national_bureau_of_economic_research> ; # National Bureau of Economic Research,
    schema:url <http://www.nber.org/papers/w25058> ;
    wdrs:describedby <http://www.worldcat.org/title/-/oclc/1054234565> ;
    .


Related Entities

<http://experiment.worldcat.org/entity/work/data/5456313276#Organization/national_bureau_of_economic_research> # National Bureau of Economic Research,
    a schema:Organization ;
    schema:name "National Bureau of Economic Research," ;
    .

<http://experiment.worldcat.org/entity/work/data/5456313276#Person/snyder_christopher_christopher_mark> # Christopher Mark Snyder
    a schema:Person ;
    schema:familyName "Snyder" ;
    schema:givenName "Christopher Mark" ;
    schema:givenName "Christopher" ;
    schema:name "Christopher Mark Snyder" ;
    .

<http://experiment.worldcat.org/entity/work/data/5456313276#Series/nber_working_paper_series> # NBER working paper series ;
    a bgn:PublicationSeries ;
    schema:hasPart <http://www.worldcat.org/oclc/1054234565> ; # Sniff tests in economics : aggregate distribution of their probability values and implications for publication bias
    schema:name "NBER working paper series ;" ;
    .

<http://experiment.worldcat.org/entity/work/data/5456313276#Series/working_paper_series_national_bureau_of_economic_research> # Working paper series (National Bureau of Economic Research) ;
    a bgn:PublicationSeries ;
    schema:hasPart <http://www.worldcat.org/oclc/1054234565> ; # Sniff tests in economics : aggregate distribution of their probability values and implications for publication bias
    schema:name "Working paper series (National Bureau of Economic Research) ;" ;
    .

<http://experiment.worldcat.org/entity/work/data/5456313276#Topic/economics_literature_evaluation> # Economics literature--Evaluation
    a schema:Intangible ;
    schema:name "Economics literature--Evaluation"@en ;
    .

<http://experiment.worldcat.org/entity/work/data/5456313276#Topic/economics_research_evaluation> # Economics--Research--Evaluation
    a schema:Intangible ;
    schema:name "Economics--Research--Evaluation"@en ;
    .

<http://experiment.worldcat.org/entity/work/data/5456313276#Topic/economics_research_methodology> # Economics--Research--Methodology
    a schema:Intangible ;
    schema:name "Economics--Research--Methodology"@en ;
    .

<http://www.worldcat.org/title/-/oclc/1054234565>
    a genont:InformationResource, genont:ContentTypeGenericResource ;
    schema:about <http://www.worldcat.org/oclc/1054234565> ; # Sniff tests in economics : aggregate distribution of their probability values and implications for publication bias
    schema:dateModified "2019-02-09" ;
    void:inDataset <http://purl.oclc.org/dataset/WorldCat> ;
    .


Content-negotiable representations

Close Window

Please sign in to WorldCat 

Don't have an account? You can easily create a free account.