henhaobio
Sign in

Behavior Research MethodsNeurosciencePeer-reviewed journal article

Gorilla in our Midst: An online behavioral experiment builder

Alexander Leslie Anwyl-IrvineiD, Jessica MassonniéiD, Adam Flitton, Natasha Z. KirkhamiD, Jo K. Evershed

PDFDOIPreprintCode
In-depth full-text digest available · Jump to digest ↓

Abstract

Behavioural researchers are increasingly conducting their studies online to gain access to large and diverse samples that would be difficult to get in a laboratory environment. However, there are technical access barriers to building experiments online, and web-browsers can present problems for consistent timing - an important issue with reaction time-sensitive measures. For example, to ensure accuracy and test-retest reliability in presentation and response recording, experimenters need a working knowledge of programming languages such as JavaScript. We review some of the previous and current tools for online behavioural research, and how well they address the issues of usability and timing. We then present The Gorilla Experiment Builder (gorilla.sc) a fully tooled experiment authoring and deployment platform, designed to resolve many timing issues, and make reliable online experimentation open and accessible to a wider range of technical abilities. In order to demonstrate the platforms aptitude for accessible, reliable and scalable research, we administered the task with a range of participant groups (primary school children and adults), settings (without supervision, at home, and under supervision, in schools and public engagement events), equipment (own computers, computer supplied by researcher), and connection types (personal internet connection, mobile phone 3G/4G). We used a simplified flanker task, taken from the Attentional Networks Task (Rueda, Posner, & Rothbart, 2004). We replicated the conflict network effect in all these populations, demonstrating the platforms capability to run reaction time-sensitive experiments. Unresolved limitations of running experiments online are then discussed, along with potential solutions, and some future features of the platform.

In-depth summary

Claude Opus 5.5 · xhigh effort · 5 Nov 2026 · from the full text of version 4 (25 Feb 2019) · CC BY-NC-ND 4.0

The Gorilla Experiment Builder (gorilla.sc) provides a browser-based graphical platform for authoring and hosting behavioral studies, replicating reaction-time flanker conflict effects across 268 children and 99 unsupervised adults.

Conducting behavioral experiments online requires programming JavaScript, managing servers, and mitigating browser timing variance that can exceed 100 ms. Gorilla integrates graphical task and experiment design with managed hosting, frame-counted stimulus presentation, and high-resolution timestamps so researchers can deploy reaction-time studies without coding.

Key results

  1. ShownThe Gorilla platform combines graphical questionnaire, task, and node-based experiment builders with browser timing controls and has scaled to over 5,000 users across more than 400 academic institutions.

    Launched in September 2016, Gorilla collected data from over 28,000 participants in the last three months of 2018 (~300 participants/day) and uses trial asset pre-caching, requestAnimationFrame duration counting to compensate for 16.667 ms (60 Hz) frame lag, and performance.now timestamps (1-5 ms browser resolution). · The Gorilla Experiment Builder; Timing Control

  2. ShownIn Experiment 1, Gorilla replicated the Attentional Networks Task flanker conflict effect across 268 primary school children tested in three different supervised school and university environments.

    Across 268 children aged 4.38-12.14 years (M = 9.21, SD = 1.58; 2 excluded from 270 for <60% accuracy) in Corsica schools, London schools (over 3G/4G mobile connections), and a London university event, incongruent trials were 62.33 ms slower than congruent trials (ANCOVA F(1, 264) = 18.92, p < .001; Friedman X^2(1) = 55.37, p < .001) with no group interaction (p = .553). · Participants; Procedure; Reaction Time; Discussion

  3. ShownIn Experiment 2, Gorilla replicated the flanker conflict effect in both reaction time and accuracy among 99 unsupervised adults tested remotely at home on their own computers.

    Among 99 UK adults recruited via Prolific.ac (aged 19-40, M = 30.32, SD = 6.64; 5 excluded from 104 for <60% accuracy; average completion 4 minutes 8.36 seconds), incongruent arrow trials were 29.1 ms slower (Wilcoxon W = 414, p < .001, rrb = .83) and 1.72% less accurate (W = 1242, p < .001, rrb = .49) than congruent trials. · Participants; Accuracy; Reaction Time; Discussion

Methods, data and reproducibility

Developed Gorilla (http://www.gorilla.sc), a web-based SaaS platform with graphical Questionnaire, Task, and Experiment Builders plus a JS Code Editor using trial caching, requestAnimationFrame frame-duration tracking, and performance.now timing. Validated timing accuracy using a flanker task (4 blocks of 24 trials) in supervised children (Safari/Mac OS X) and unsupervised adults on Prolific.ac.

  • Experiment 1: supervised child fish flanker task (12 practice plus 4 blocks of 24 trials) across Corsica schools (Group A), London schools (Group B), and a London university event (Group C) · human · n = 268 children analyzed (ages 4.38-12.14, M = 9.21; 270 recruited)
  • Experiment 2: unsupervised adult arrow flanker task (12 practice plus 4 blocks of 24 trials) completed at home via Prolific.ac · human · n = 99 adults analyzed (ages 19-40, M = 30.32; 104 recruited)
Methods & tools
Gorilla Experiment Builder, Prolific.ac, G*Power, Safari, WebGazer.js, jsPsych
Software & environment
Experiment 1 ran on Safari on Mac OS X over 3G/4G mobile or university internet; Experiment 2 ran on participant home computers across multiple browsers and operating systems
Reproducibility gaps
Participant-level datasets are not publicly shared because participants (mostly children) did not consent to data sharing; Gorilla is a managed SaaS platform rather than open-source software.
Limitations (3)
  • Browser-based response recording cannot access or control input device USB polling rates (typically 125 Hz, introducing an 8 ms sampling interval), and major browsers round performance.now timestamps to 1-5 ms for security reasons. · stated by the authors
  • Stimulus durations are constrained by participant monitor refresh rates (typically 60 Hz, or multiples of 16.667 ms) and can be affected by unmeasured CPU processing load on remote participant machines. · stated by the authors
  • Individual-level behavioral data from the two validation experiments are not publicly available due to lack of participant consent for data sharing, and three authors are employed by or founded Cauldron Science, the company behind Gorilla. · stated by the authors

Read this if you design online behavioral or cognitive psychology experiments and want a graphical platform that handles hosting, complex study flows, and millisecond-scale reaction time recording without programming.

Citations
1,874
FWCI
147.99
Field percentile
top 1%

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISWilliams '23Colcombe '26Tsay '21Kim '19Holden '19Kurzawski '23
58% semantic matchNeuroscience
Nikolas Williams, William L. King, Geoffrey Mackellar+3 more·Heliyon·Jul 24, 2023· 19 citations