Top 10 Bitss Papers Today in Social And Behavioral Sciences


0.0 Mikeys
#1. Likelihood Ratios: A Tutorial
Scott
Many in psychology view their choice of statistical approaches as being between frequentist and Bayesian. However, a third approach, the use of likelihood ratios, provides several distinct advantages over both the frequentist and Bayesian options. A quick explanation of the basic logic of likelihood ratios is provided, followed by a comparison of the likelihood-based approach to frequentist and Bayesian methods. The bulk of the paper provides examples with formulas for computing likelihood ratios based on t-scores, ANOVA outputs, chi-square statistics, and binomial data, as well as examples of using likelihood ratios to test for models that make a priori predictions of effect sizes. Finally, advice on interpretation is offered.
more | pdf
Figures
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 1
Total Words: 6155
Unqiue Words: 1768

0.0 Mikeys
#2. Declaring and Diagnosing Research Designs
Graeme Blair, Jasper Cooper, Alexander Coppock, Macartan Humphreys
Researchers need to select high-quality research designs and communicate those designs clearly to readers. Both tasks are difficult. We provide a framework for formally “declaring” the analytically relevant features of a research design in a demonstrably complete manner, with applications to qualitative, quantitative, and mixed methods research. The approach to design declaration we describe requires defining a model of the world (M), an inquiry (I), a data strategy (D), and an answer strategy (A). Declaration of these features in code provides sufficient information for researchers and readers to use Monte Carlo techniques to diagnose properties such as power, bias, correct identification of causal conditions, and other “diagnosands.” Ex ante declarations can be used to improve designs and facilitate preregistration, analysis, and reconciliation of intended and actual analyses. Ex post declarations are useful for describing, sharing, reanalyzing, and critiquing existing designs. We provide open-source software, DeclareDesign, to...
more | pdf
Figures
None.
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 4
Total Words: 32290
Unqiue Words: 6343

0.0 Mikeys
#3. Why We Need Open Policy Analysis
Fernando Hoces de la Guardia, Sean Grant, Edward Miguel
The evidence-based policy movement promotes the use of empirical evidence to inform policy decision-making. While this movement has gained traction over the last two decades, several concerns about the credibility of empirical research have been identified in scientific disciplines that use research methods and practices that are commonplace in policy analysis. As a solution, we argue that policy analysis should adopt the transparent, open, and reproducible research practices espoused in related disciplines. We first discuss the importance of evidence-based policy in an era of increasing disagreement about facts, analysis, and expertise. We then review recent credibility crises of empirical research (difficulties reproducing results), their causes (questionable research practices such as publication biases and p-hacking), and their relevance to the credibility of evidence-based policy (trust in policy analysis). The remainder of the paper makes the case for "open" policy analysis and how to achieve it. We include examples of...
more | pdf
Figures
Tweets
GrantSeanP: @fhoces from @UCBITSS presenting on Open Policy Analysis at #APPAM2018 Read our pre-print with @tedmiguel here! https://t.co/UpJ1mR2C9q https://t.co/5nv9Yv3FDk
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 3
Total Words: 7685
Unqiue Words: 2616

0.0 Mikeys
#4. Data availability, reusability, and analytic reproducibility: Evaluating the impact of a mandatory open data policy at the journal Cognition
Tom E Hardwicke, Maya Mathur, Kyle MacDonald, Gustav Nilsonne, George Christopher Banks, Mallory C. Kidwell, Alicia Hofelich Mohr, Elizabeth Clayton, Erica J. Yoon, Michael Henry Tessler
Access to data is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data (“analytic reproducibility”). To investigate, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition. Interrupted time-series analyses indicated a substantial post-policy increase in data available statements (104/417, 25% pre-policy to 136/174, 78% post-policy), although not all data appeared reusable (23/104, 22% pre-policy to 85/136, 62%, post-policy). For 35 of the articles determined to have reusable data, we attempted to reproduce 1324 target values. Ultimately, 64 values could not be reproduced within a 10% margin of error. For 22 articles all target values were reproduced, but 11 of these required author assistance. For 13 articles...
more | pdf
Figures
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 10
Total Words: 20760
Unqiue Words: 5776

0.0 Mikeys
#5. A Bayesian Model-Averaged Meta-Analysis of the Power Pose Effect with Informed and Default Priors: The Case of Felt Power
Quentin Frederik Gronau, Sara van Erp, Daniel W. Heck, Joseph Cesario, Kai Jonas, Eric-Jan Wagenmakers
Carney, Cuddy, and Yap (2010) found that --compared to participants who adopted constrictive body postures-- participants who adopted expansive body postures reported feeling more powerful, showed an increase in testosterone and a decrease in cortisol, and displayed an increased tolerance for risk. However, these power pose effects have recently come under considerable scrutiny. Here we present a Bayesian meta-analysis of six preregistered studies from this special issue, focusing on the effect of power posing on felt power. Our analysis improves on standard classical meta-analyses in several ways. First and foremost, we considered only preregistered studies, eliminating concerns about publication bias. Second, the Bayesian approach enables us to quantify evidence for both the alternative and the null hypothesis. Third, we use Bayesian model-averaging to account for the uncertainty with respect to the choice for a fixed-effect model or a random-effect model. Fourth, based on a literature review we obtained an empirically informed...
more | pdf
Figures
Tweets
Github

Bayesian Model Averaging for Random and Fixed Effects Meta-Analysis

Repository: metaBMA
User: danheck
Language: R
Stargazers: 2
Subscribers: 3
Forks: 2
Open Issues: 0
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 6
Total Words: 6572
Unqiue Words: 1582

0.0 Mikeys
#6. bridgesampling: An R Package for Estimating Normalizing Constants
Quentin Frederik Gronau, Henrik Singmann, Eric-Jan Wagenmakers
Statistical procedures such as Bayes factor model selection and Bayesian model averaging require the computation of normalizing constants (e.g., marginal likelihoods). These normalizing constants are notoriously difficult to obtain, as they usually involve high-dimensional integrals that cannot be solved analytically. Here we introduce an R package that uses bridge sampling (Meng and Wong 1996; Meng and Schilling 2002) to estimate normalizing constants in a generic and easy-to-use fashion. For models implemented in Stan, the estimation procedure is automatic. We illustrate the functionality of the package with three examples.
more | pdf
Figures
None.
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 3
Total Words: 11662
Unqiue Words: 2994

0.0 Mikeys
#7. A tutorial on bridge sampling
Quentin Frederik Gronau, Alexandra Sarafoglou, Dora Matzke, Alexander Ly, Udo Boehm, maarten marsman, David Leslie, Jonathan Forster, Eric-Jan Wagenmakers, Helen Steingroever
The marginal likelihood plays an important role in many areas of Bayesian statistics such as parameter estimation, model comparison, and model averaging. In most applications, however, the marginal likelihood is not analytically tractable and must be approximated using numerical methods. Here we provide a tutorial on bridge sampling (Bennett, 1976; Meng & Wong, 1996), a reliable and relatively straightforward sampling method that allows researchers to obtain the marginal likelihood for models of varying complexity. First, we introduce bridge sampling and three related sampling methods using the beta-binomial model as a running example. We then apply bridge sampling to estimate the marginal likelihood for the Expectancy Valence (EV) model---a popular model for reinforcement learning. Our results indicate that bridge sampling provides accurate estimates for both a single participant and a hierarchical version of the EV model. We conclude that bridge sampling is an attractive method for mathematical psychologists who typically...
more | pdf
Figures
None.
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 10
Total Words: 18868
Unqiue Words: 3474

0.0 Mikeys
#8. Fixed or Random?
Benjamin Scheibehenne, Quentin Frederik Gronau, Tahira Jamil, Eric-Jan Wagenmakers
Carlsson, Schimmack, Williams, and Bürkner criticize our analysis (Scheibehenne, Jamil, & Wagenmakers, 2016) because it collapses data across different studies. In addition, CSWB apply a random-effect model and argue that allegedly minor changes in the parameter priors render the evidence inconclusive. Here we address both issues.
more | pdf
Figures
None.
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 4
Total Words: 1827
Unqiue Words: 882

0.0 Mikeys
#9. Improving transparency in observational social science research: A pre-analysis plan approach
Fiona Burlig
Social science research has undergone a credibility revolution, but these gains are at risk due to problematic research practices. Existing research on transparency has centered around randomized controlled trials, which constitute only a small fraction of research in economics. In this paper, I discuss three scenarios in which study preregistration can be credibly applied in non-experimental settings: cases where researchers collect their own data; prospective studies; and research using restricted-access data. Finally, I outline suggested contents for observational pre-analysis plans, and highlight where these plans should deviate from those designed for experimental research.
more | pdf
Figures
None.
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 1
Total Words: 7282
Unqiue Words: 2374

0.0 Mikeys
#10. The evolving preprint landscape: Introductory report for the Knowledge Exchange working group on preprints.
Jonathan Tennant, Serge Bauin, Sarah James, Juliane Kant
Introductory report for the Knowledge Exchange working group on preprints.
more | pdf
Figures
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 4
Total Words: 4634
Unqiue Words: 1865

About

Assert is a website where the best academic papers on arXiv (computer science, math, physics), bioRxiv (biology), BITSS (reproducibility), EarthArXiv (earth science), engrXiv (engineering), LawArXiv (law), PsyArXiv (psychology), SocArXiv (social science), and SportRxiv (sport research) bubble to the top each day.

Papers are scored (in real-time) based on how verifiable they are (as determined by their Github repos) and how interesting they are (based on Twitter).

To see top papers, follow us on twitter @assertpub_ (arXiv), @assert_pub (bioRxiv), and @assertpub_dev (everything else).

To see beautiful figures extracted from papers, follow us on Instagram.

Tracking 56,474 papers.

Search
Sort results based on if they are interesting or reproducible.
Interesting
Reproducible
Feedback
Online
Stats
Tracking 56,474 papers.