Top 10 Psyarxiv Papers Today in Meta-Science


0.0 Mikeys
#1. False Positive Poisson
William Ryan, Ellen Evers, Don A. Moore
When analyzing count data (such as number of questions answered correctly), psychologists often use Poisson regressions. We show through simulations that violating the assumptions of a Poisson distribution even slightly can lead to false positive rates more than doubling, and illustrate this issue with a study that finds a clearly spurious but highly significant connection between seeing blue and eating fish candies. In additional simulations we test alternate methods for analyzing count-data and show that these generally do not suffer from the same inflated false positive rate, nor do they result in much higher false negatives in situations where Poisson would be appropriate.
more | pdf
Figures
Tweets
PsyArXivBot: False Positive Poisson https://t.co/rqVZ70oEOC
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 3
Total Words: 2521
Unqiue Words: 956

0.0 Mikeys
#2. Sex Differences in Trust and Trustworthiness - A Meta-Analysis of the Trust Game and the Gift-Exchange Game
Olmo van den Akker
Do men and women differ in trusting behavior? This question is directly relevant to social, economic, and political domains, yet the answer remains elusive. In this paper, we present a meta-analytic review of the literature on sex differences in the trust game and a variant, the gift-exchange game. Informed by both evolutionary and cultural perspectives, we predicted men to be more trusting and women to be more trustworthy in these games. The trust game meta-analyses encompass 77 papers yielding 174 effect sizes based on 17,082 participants from 23 countries, while the gift-exchange game meta-analyses covered 15 papers reporting 35 effect sizes based on 1,362 participants from 19 countries. In the trust game, we found men to be more trusting than women, g = 0.22, but we found no significant sex difference in trustworthiness, g = 0.09. In the gift-exchange game we found no significant sex difference in trust, g = 0.14, yet we did find that men are more trustworthy than women, g = 0.33. The results of the meta-analyses show that the...
more | pdf
Figures
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : [81, 32]
Authors: 1
Total Words: 22664
Unqiue Words: 4795

0.0 Mikeys
#3. Open Science challenges, benefits and tips in early career and beyond – manuscript
Christopher P G Allen, David MA Mehler
The movement towards open science is an unavoidable consequence of seemingly pervasive failures to replicate previous research. This transition comes with great benefits but also significant challenges that are likely to afflict those who carry out the research, usually Early Career Researchers (ECRs). Here, we describe key benefits including reputational gains, increased chances of publication and a broader increase in the reliability of research. These are balanced by challenges that we have encountered, and which involve increased costs in terms of flexibility, time and issues with the current incentive structure, all of which seem to affect ECRs acutely. Although there are major obstacles to the early adoption of open science, overall open science practices should benefit both the ECR and improve the quality and plausibility of research. We review three benefits, three challenges and provide suggestions from the perspective of ECRs for moving towards open science practices.
more | pdf
Figures
None.
Tweets
Protohedgehog: This is an important, balanced article on the potential challenges and opportunities of #openscience for early-career researchers https://t.co/qg7LCEzBCK #ecrchat #openscholarship @PsyArXiv
lisadebruine: Evidence that RRs do improve scientific practice https://t.co/2ZaZDBPnFo
OpenScienceMOOC: New article: Open Science challenges, benefits and tips in early career and beyond https://t.co/5uGHAYJnuZ
StuartBuck1: @ingorohlfing Footnote 2: https://t.co/xISbc14L0q
CUBRICcardiff: Read the @nature coverage of Chris Allen (@ChrisPGAllen) and David Mehler's (@neuroccino) recent preprint 'Open Science challenges, benefits and tips in early career and beyond'. Nature article: https://t.co/mCWlHiIvB7 Preprint available here: https://t.co/DjqzomNwQl #openscience
schopfel: Open Science challenges, benefits and tips in early career and beyond – manuscript #openscience https://t.co/iFJ9xMSu3X
ingorohlfing: On the study linking registered reports (RR) to more null findings https://t.co/4fp1IzgzI6 (link doesn't work right now). The finding could also be because 1/
ChrisBanks: "analysis now suggests that registering & peer-reviewing study protocols before research is conducted could improve this ‘file-drawer problem’, & help 2 correct the existing publication bias towards positive findings" https://t.co/4Mao8FupeZ #OAWeek18 #OA https://t.co/RnuUp7vAXI
RegReports: The current proportion of null findings in the lit is "drastically at odds with what we would expect without any publication bias” says @annemscheel. Allen & @neuroccino find that changes with #RegisteredReports on @PsyArXiv https://t.co/beHEdx00jG https://t.co/Fp40Q1eRHH
EvoMellor: @JohnHolbein1 And here's a preprint with some of the first evidence outside of clinical sciences https://t.co/04OKo4yEmG
neuroccino: @TranspariMED @cardiffuni @chrisdc77 That’s great, thanks for the shout out! The guarantee to publish really is one advantage of @RegReports - we recently estimated null findings at about 60% https://t.co/nA9vKTHjzJ
neuroccino: @joshrbaxter @KordingLab Thanks! @ChrisPGAllen and I discuss these and other challenges as well as some main benefits in our pre-print (that also contains the analysis Nature reported on): https://t.co/VZSQWUkzjL let us know what you think!
neuroccino: @avigotsky @joshrbaxter @KordingLab @AmSocBiomech Brilliant! Please feel free to refer them to our opinion piece Open Science challenges, benefits and tips in early career and beyond https://t.co/VZSQWUkzjL
roehamptonpsych: First analysis of pre-registered reports shows higher rate of null results compared to regular submissions. https://t.co/f6FybDia4t https://t.co/ruh3ELXozJ https://t.co/TtUqWKRxG6
OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
AmandaKMontoya: Spent some time day reading this excellent preprint about how early career researchers interface with the new #openscience movement (challenges and benefits) https://t.co/vPGeUf3PnO via @OSFramework #phdlife #replication #newprof
DMCpersonality: I just stumbled across this recently posted manuscript which addresses many of the issues I brought up in this thread. Nice work! https://t.co/I2SN7kvF6P
Gen_R_: #OpenScience challenges, benefits and tips in early career and beyond – manuscript by Christopher Allen and David MA Mehler. https://t.co/XcOBzKbR71
MeridianReports: Fascinating result of the impact of registered reports. https://t.co/TlEnmnyEBD. #reproducibleresearch #veterinaryresearch
ajshackman: https://t.co/lbAoTbrjPq
nafisajadavji: Open Science challenges, benefits and tips in early career and beyond – manuscript https://t.co/bTmFfEv4xi
OSFramework: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
JohnHolbein1: RT @EvoMellor: @JohnHolbein1 And here's a preprint with some of the first evidence outside of clinical sciences https://t.co/04OKo4yEmG
Christine_Brim: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
LivveyKirtley: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
ResearchersAtEd: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
scielomexico: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
sharoz: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
olivier_klein: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
rgaiacs: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
SimonCHunter: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
ametodico: RT @OpenScienceMOOC: New article: Open Science challenges, benefits and tips in early career and beyond https://t.co/5uGHAYJnuZ
Sam_D_Parsons: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
danieltomasz: RT @nafisajadavji: Open Science challenges, benefits and tips in early career and beyond – manuscript https://t.co/bTmFfEv4xi
karo_urb: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
YosoyelAfo: RT @OpenScienceMOOC: New article: Open Science challenges, benefits and tips in early career and beyond https://t.co/5uGHAYJnuZ
ianhussey: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
LZPhD: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
PaoloAPalma: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
katiedrax: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
DavidTroy79: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
BeckerLabUSC: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
lam_bis: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
LashuelLab: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
EvieVergauwe: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
nagyt: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
heyskeg: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
awakenting: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
AbambresM: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
data_cats: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
OpenScienceMov1: RT @OSFPrereg: Great preprint discussing the challenges and benefits of open science practices for ECRs. https://t.co/njE6vQnHSA
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 2
Total Words: 5041
Unqiue Words: 1950

0.0 Mikeys
#4. Protocol for a Systematic Review and Meta-Analysis of Racial Disparities in Executive Functions in the United States
Gianna Rea-Sandin, Kevin M. Korous, José M. Causadias
Executive functioning has important implications for multiple developmental processes and outcomes. However, the degree to which executive functioning varies between and within racial/ethnic groups in the United States is less understood. To address this issue, we will conduct a systematic review and meta-analysis of overall executive functioning and its three core components: inhibitory control, working memory, and cognitive flexibility. Our goals are to: 1) examine differences across these components between Whites and minorities, as well as differences between- (e.g., African-Americans, Latinos) and within- (e.g., Latinos: Mexican-Americans, Cuban-Americans) minority groups; and 2) test potential moderators of these differences (e.g., sex, age, measure). We will address the implications of these findings in terms of support for the cultural differences and similarities hypotheses.
more | pdf
Figures
None.
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 3
Total Words: 8716
Unqiue Words: 2361

0.0 Mikeys
#5. An Extended Commentary on Post-Publication Peer Review In Organizational Neuroscience
Guy A Prochilo, Winnifred R Louis, Stefan Bode, Hannes Zacher, Pascal Molenberghs
In this extended commentary, we critically evaluate a sample of empirical studies conducted in organizational neuroscience with respect to present-day best practices (i.e., using parameter estimation based on effect size and confidence intervals, and with consideration of the implications of inadequate research and reporting practices), with the primary goal of developing new recommendations to strengthen post-publication peer review. Our commentary yields several research themes we propose scholars must engage with in future critical evaluations of the literature, particularly with respect to seminal works in the field. This includes addressing the implications of a lack of transparency in reporting; interpretation of findings through extreme reverse inference; drawing conclusions by eyeballing effect size differences in lieu of statistical testing; and the use of analyses that preclude generalizability beyond the sample, among others. Because we endorse the view that organizational neuroscience provides a valuable research...
more | pdf
Figures
Tweets
GuyProchilo: "[it] has been cited in almost every review of the literature since its publication, yet it lacks necessary information to evaluate core aspects of the research for replication or unhindered critical appraisal.” –read more at: https://t.co/LPrfNjRDfW #openscience #phdchat #phd https://t.co/FYofAaZd3N
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 5
Total Words: 13629
Unqiue Words: 4260

0.0 Mikeys
#6. Pragmatism should not be a substitute for statistical literacy, a commentary on Albers, Kiers, and van Ravenzwaaij (2018)
Ladislas Nalborczyk, Paul-Christian Bürkner, Donald R. Williams
Based on the observation that frequentist confidence intervals and Bayesian credible intervals sometimes happen to have the same numerical boundaries (under very specific conditions), Albers et al. (2018) proposed to adopt the heuristic according to which they can usually be treated as equivalent. We argue that this heuristic can be misleading by showing that it does not generalise well to more complex (realistic) situations and models. Instead of pragmatism, we advocate for the use of parsimony in deciding which statistics to report. In a word, we recommend that a researcher interested in the Bayesian interpretation simply reports credible intervals.
more | pdf
Figures
Tweets
lnalborczyk: @sTeamTraen @hardsci @DrLTimmins @CochraneUK @lakens It also happens with not that hard stuffs (pretty common stuffs actually). We recently wrote a short commentary about that. https://t.co/5xd0DCDloD
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 3
Total Words: 3685
Unqiue Words: 1511

0.0 Mikeys
#7. The Unity and Diversity of Executive Functions: A Systematic Review and Re-Analysis of Latent Variable Studies
Justin E. Karr, Corson N. Areshenkoff, Philippe Rast, Scott M. Hofer, Grant L. Iverson, Mauricio Garcia-Barrera
Confirmatory factor analysis (CFA) has been frequently applied to executive function measurement since first used to identify a three-factor model of inhibition, updating, and shifting; however, subsequent CFAs have supported inconsistent models across the lifespan, ranging from unidimensional to nested-factor models (i.e., bifactor without inhibition). This systematic review summarized CFAs on performance-based tests of executive functions and reanalyzed summary data to identify best-fitting models. Eligible CFAs involved 46 samples (N=9,756). The most frequently accepted models varied by age (i.e., preschool=one/two-factor; school-age=three-factor; adolescent/adult=three/nested-factor; older adult=two/three-factor), and most often included updating/working memory, inhibition, and shifting factors. A bootstrap reanalysis simulated 5,000 samples from 21 correlation atrices (11 child/adolescent; 10 adult) from studies including the three most common factors, fitting seven competing models. Model results were summarized as the mean...
more | pdf
Figures
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : [9]
Authors: 6
Total Words: 47567
Unqiue Words: 7753

0.0 Mikeys
#8. A method to streamline p-hacking
Ian Hussey
The analytic strategy of p-hacking has rapidly accelerated our advancement of the goals of psychological science (i.e., publications, tenure and flair: Bakker, Dijk, & Wicherts, 2012), but has suffered a number of setbacks in recent years. In order to remediate this, this article presents a statistical approach that can greatly accelerate and streamline the p-hacking process: generating random numbers that are < .05, which we refer to as p_economical. Results of a simulation study are presented and an R script is provided. In the absence of systemic changes to modal p-hacking practices within psychological science (e.g., worrying trends such as preregistration and replication: Munafò et al., 2017), we argue that vast amounts of time and research funding could be saved through the widespread adoption of this innovative statistical approach.
more | pdf
Figures
None.
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 1
Total Words: 1263
Unqiue Words: 654

0.0 Mikeys
#9. An informal internet survey on the current state of Consciousness Science
Matthias Michel, Stephen M. Fleming, Hakwan Lau, Alan L. F. Lee, Susana Martinez-Conde, Richard E. Passingham, Megan A. K. Peters, Dobromir Rahnev, Claire Sergent, Ka-yuet Liu
The scientific study of consciousness became an organized field of research less than thirty years ago. Since then, a large number of empirical findings increased our understanding of consciousness. Scientific progress, however, does not only consist on the advancement of knowledge. Increased funding for consciousness research and the rising status of scientists engaging in the field are also fundamental indicators of scientific progress as well as ​sine qua non conditions for such progress, and should not be overlooked. Here, we present the results of a survey designed to investigate the current state of the neuroscience of consciousness in regards to these latter aspects.
more | pdf
Figures
Tweets
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 10
Total Words: 2377
Unqiue Words: 1004

0.0 Mikeys
#10. Publication bias and statistical power in gerontological psychology
Christopher Brydges
Objectives: Research has found evidence of publication bias, questionable research practices (QRPs), and low statistical power in published psychological journal articles. Isaacowitz’s (2018) editorial in the Journals of Gerontology Series B, Psychological Sciences called for investigation of these issues in gerontological research. The current study presents meta-research findings based on published research to explore if there is evidence of these practices in gerontological research. Method: 14,481 test statistics and p values were extracted from articles published in eight top gerontological psychology journals since 2000. Frequentist and Bayesian caliper tests were used to test for publication bias and QRPs (specifically, p-hacking and incorrect rounding of p values). A z-curve analysis was used to estimate average statistical power across studies. Results: Strong evidence of publication bias was observed, and average statistical power was approximately .70 – below the recommended .80 level. Evidence of p-hacking was...
more | pdf
Figures
None.
Tweets
daforerog: @CRChartier @hmoshontz Our initial PSA paper already has 5 citations https://t.co/rmPFIob0wA https://t.co/PVO2fcvRsn https://t.co/lNZydq4eej https://t.co/9iRWxG6fhK https://t.co/0Ndeoh4iAQ
Github
None.
Youtube
None.
Other stats
Sample Sizes : None.
Authors: 1
Total Words: 7193
Unqiue Words: 2188

About

Assert is a website where the best academic papers on arXiv (computer science, math, physics), bioRxiv (biology), BITSS (reproducibility), EarthArXiv (earth science), engrXiv (engineering), LawArXiv (law), PsyArXiv (psychology), SocArXiv (social science), and SportRxiv (sport research) bubble to the top each day.

Papers are scored (in real-time) based on how verifiable they are (as determined by their Github repos) and how interesting they are (based on Twitter).

To see top papers, follow us on twitter @assertpub_ (arXiv), @assert_pub (bioRxiv), and @assertpub_dev (everything else).

To see beautiful figures extracted from papers, follow us on Instagram.

Tracking 56,474 papers.

Search
Sort results based on if they are interesting or reproducible.
Interesting
Reproducible
Feedback
Online
Stats
Tracking 56,474 papers.