Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

Basit Öğe Kaydı
dc.authoridDurrheim, Kevin/0000-0003-2926-5953|Frankowska, Natalia/0000-0001-9487-0629|Lipsey, Nikolette/0000-0002-2367-0280|VAZQUEZ, ALEXANDRA/0000-0002-6040-9102|Lazarevic, Ljiljana B./0000-0003-1629-3699|Doğulu, Canay/0000-0002-5906-3706|Milfont, Taciano L/0000-0001-6838-6307|Vianello, Michelangelo/0000-0002-1530-1469|Alper, Sinan/0000-0002-9051-0690|Gandi, Joshua Chiroma/0000-0001-6724-1296|Gnambs, Timo/0000-0002-6984-1276|English, Alexander/0000-0002-0142-146X|Karabati, Serdar/0000-0002-4668-5274|Pilati, Ronaldo/0000-0003-2982-5033|Morales, Patricio Javier Saavedra/0000-0003-0921-6150|De Schutter, Leander/0000-0001-9826-4896|Gill, Tripat/0000-0001-9839-4113|Dalgar, İlker/0000-0003-3121-5728|Lakens, Daniel/0000-0002-0247-239X|Huynh, Ho Phi/0000-0002-9931-7467|Rosa, Anna Dalla/0000-0003-4862-4077|Theriault, Jordan/0000-0002-4680-0172|Klein, Richard/0000-0002-8713-1972|Cicero, David Colin/0000-0002-5666-9139|Giessner, Steffen/0000-0002-8035-5092|Grahek, Ivan/0000-0002-9247-5956|Knezevic, Goran D/0000-0001-8951-3774|Carmichael, Cheryl L./0000-0002-4474-3485|Cantarero, Katarzyna/0000-0003-2720-484X|Bocian, Konrad/0000-0002-8652-0167|Sobkow, Agata/0000-0002-5357-744X|Vega, Luis/0000-0003-3791-8440|Cabak Redei, Anna/0000-0003-4246-3153|Sirlopu, David/0000-0003-4683-1734|Bialobrzeska, Olga/0000-0002-0387-3187|Torres-Fernandez, David/0000-0003-3023-5129|Zelenski, John/0000-0002-3668-5764|Pollmann, Monique/0000-0003-3918-0646|Kamiloglu, Roza G./0000-0002-1018-2595|morris, wendy/0000-0002-5060-457X|Kervyn, Nicolas/0000-0001-9056-6283|Hasselman, Fred/0000-0003-1384-8361|Kappes, Heather/0000-0002-6335-3888|Smith, Michael/0000-0001-5107-6441|Zeng, Zhijia/0000-0001-8780-4922|Corker, Katherine S./0000-0002-7971-1678|Szeto, Stephanie/0000-0001-7742-765X|Van Lange, Paul/0000-0001-7774-6984|Hicks, Joshua/0000-0002-6237-4935|Babalola, Mayowa/0000-0002-9468-6183|Williams, Lisa A/0000-0002-0021-5613|JIMENEZ-LEAL, WILLIAM/0000-0002-8824-5269|van 't Veer, Anna/0000-0002-2733-1841|Neave, Nick/0000-0002-5380-4614|Haigh, Matthew/0000-0003-1549-697X|Vranka, Marek/0000-0003-3413-9062|Coleman, Jennifer/0000-0002-0158-2589|Adams, Byron/0000-0001-5968-1996|Perez Sanchez, Rolando/0000-0001-6321-2543|Chandler, Jesse/0000-0001-8151-0915|Schmidt, Kathleen/0000-0002-9946-5953|Graham, Jesse/0000-0001-8863-7978|Neto, Felix/0000-0003-0112-880X|Sekerdej, Maciej/0000-0001-7031-1523
dc.authorwosidDurrheim, Kevin/KLC-9041-2024
dc.authorwosidFrankowska, Natalia/AAH-5458-2021
dc.authorwosidTybur, Joshua M/P-5435-2014
dc.authorwosidLipsey, Nikolette/HDM-9426-2022
dc.authorwosidVAZQUEZ, ALEXANDRA/A-1640-2012
dc.authorwosidLazarevic, Ljiljana B./L-8779-2016
dc.authorwosidDoğulu, Canay/AAH-4518-2019
dc.contributor.authorKlein, Richard A.
dc.contributor.authorVianello, Michelangelo
dc.contributor.authorHasselman, Fred
dc.contributor.authorAdams, Byron G.
dc.contributor.authorAdams, Reginald B., Jr.
dc.contributor.authorAlper, Sinan
dc.contributor.authorAveyard, Mark
dc.date.accessioned2024-07-18T20:49:02Z
dc.date.available2024-07-18T20:49:02Z
dc.date.issued2018
dc.departmentİstanbul Bilgi Üniversitesien_US
dc.description.abstractWe conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance (p < .05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion (p < .0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely high-powered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohen's ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than .20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above .10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and low WEIRDness scores, respectively). Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied.en_US
dc.description.sponsorshipCenter for Open Science; Association for Psychological Science from the Laura and John Arnold Foundation; Fondap from the Center for Social Conflict and Cohesion Studies [15130009]en_US
dc.description.sponsorshipThis research was supported by the Center for Open Science and by a grant through the Association for Psychological Science from the Laura and John Arnold Foundation. The research in Chile was supported by Fondap Grant 15130009 from the Center for Social Conflict and Cohesion Studies.en_US
dc.identifier.doi10.1177/2515245918810225
dc.identifier.endpage490en_US
dc.identifier.issn2515-2459
dc.identifier.issn2515-2467
dc.identifier.issue4en_US
dc.identifier.scopus2-s2.0-85116789185en_US
dc.identifier.scopusqualityQ1en_US
dc.identifier.startpage443en_US
dc.identifier.urihttps://doi.org/10.1177/2515245918810225
dc.identifier.urihttps://hdl.handle.net/11411/8041
dc.identifier.volume1en_US
dc.identifier.wosWOS:000746381600001en_US
dc.identifier.wosqualityN/Aen_US
dc.indekslendigikaynakWeb of Scienceen_US
dc.indekslendigikaynakScopusen_US
dc.language.isoenen_US
dc.publisherSage Publications Incen_US
dc.relation.ispartofAdvances in Methods and Practices in Psychological Scienceen_US
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanıen_US
dc.rightsinfo:eu-repo/semantics/openAccessen_US
dc.subjectSocial Psychologyen_US
dc.subjectCognitive Psychologyen_US
dc.subjectReplicationen_US
dc.subjectCultureen_US
dc.subjectİndividual Differencesen_US
dc.subjectSampling Effectsen_US
dc.subjectSituational Effectsen_US
dc.subjectMeta-Analysisen_US
dc.subjectRegistered Reporten_US
dc.subjectOpen Dataen_US
dc.subjectOpen Materialsen_US
dc.subjectPreregistereden_US
dc.subjectSocioeconomic-Statusen_US
dc.subjectDirect Replicationen_US
dc.subjectIntentional Actionen_US
dc.subjectMoral Judgmentsen_US
dc.subjectData-Collectionen_US
dc.subjectSocial-Statusen_US
dc.subjectPsychologyen_US
dc.subjectSelfen_US
dc.subjectSensitivityen_US
dc.subjectDisgusten_US
dc.titleMany Labs 2: Investigating Variation in Replicability Across Samples and Settings
dc.typeArticle

Dosyalar

Koleksiyon

Web of Science Indexed Publications
Scopus Indexed Publications