Abstract:

In one form or another, most MIR research depends on the judgment of humans. Humans provide our ground-truth data, whether through explicit annotation or through observable behavior (e.g., listening histories); Humans also evaluate our results, whether in academic research reports or in the commercial marketplace. Will users like it? Will customers buy it? Does it sound good? These are all critical questions for MIR researchers which can only be answered by asking people. Unfortunately, measuring and interpreting the judgments and experiences of humans in a rigorous manner is difficult. Human responses can be fickle, changeable, and inconsistent—they are, by definition, subjective. There are many factors that influence human responses, some of which can be controlled or accounted for in experimental design, and others which must be tolerated but ameliorated through statistical analysis. Fortunately, researchers in the field of behavioral psychology have amassed extensive expertise and institutional knowledge related to the practice and pedagogy of human-subject research, but MIR researchers receive little exposure to research methods involving human subjects. This tutorial, led by MIR researchers with training (and publications) in psychological research, aims to share these insights with the ISMIR community. The tutorial will introduce key concepts, terminology, and concerns in carrying out human-subject research, all in the context of MIR. Through the discussion of real and hypothetical human research, we will explore the nuances of experiment and survey design, stimuli creation, sampling, psychometric modeling, and statistical analysis. We will review common pitfalls and confounds in human research, and present guidelines for best practices in the field. We will also cover fundamental ethical and legal requirements of human research. Any and all ISMIR members are welcome and encouraged to attend: it is never too early, or too late, in one’s research career to learn (or practice) these essential skills.

Bios:

Claire Arthur is an assistant professor in the School of Music and co-director of the Computational and Cognitive Musicology Lab at the Georgia Institute of Technology, and adjunct faculty in the School of Psychology. She received her PhD in music theory and cognition from Ohio State University under David Huron. Her research largely focuses on modeling musical structure from a statistical perspective, as well as examining the cognitive and behavioral correlates of those structures, especially as it relates to musical expectations and emotional responses. Her MIR-related research interests lie in the intersection of music perception, computational musicology, and emotion prediction, with an emphasis on melody, voice-leading, and harmony.

Nat Condit-Schultz is a Lecturer and the Director of the Graduate Program for the Georgia Tech School of Music. Nat is a musician, composer, and scientist, specializing in the statistical modeling of musical structure. Nat directs the Georgia Tech rock and pop bands, and teaches courses in research methodology, music psychology, and music production. Nat’s research interests include rhythm and tonality in popular music, the perceptual and structural roles of language and lyrics in music, and the music theory of hip-hop. Nat is a performer and composer, specializing in electric and classical guitar: as a composer, he specializes in imitative counterpoint and complex rhythmic/metric ideas like polyrhythm, “tempo spirals,” and irama, realized through classical guitar, rock instrumentation, and Indonesian Gamelan.

David Sears is Associate Professor of Interdisciplinary Arts and Co-Director of the Performing Arts Research Lab at Texas Tech University, where he teaches courses in arts psychology, arts informatics, and music theory. His current research examines the structural parallels between music and language using both behavioral and computational methods, with a particular emphasis on the many topics associated with pitch structure, including scale theory, tonality, harmony, cadence, and musical form. He also has ancillary interests in music on the global radio, music and emotion, and cross-cultural research. Recent publications appear in his Google Scholar profile.

John Ashley Burgoyne is Assistant Professor in Computational Musicology at the University of Amsterdam, teaching in the Musicology and Artificial Intelligence and conducting research in the Language and Music Cognition unit at the Institute for Logic, Language, and Computation. His current research focuses on using psychometric approaches in combination with representations and embeddings from deep learning models to improve the interpretability of AI models and flexibility in the design of musical stimuli and experiments. As director of the Amsterdam Music Lab, he is also interested in citizen science and online experimentation, and leads a team developing the MUSCLE infrastructure for facilitating online experiments requiring fine control of audio and music.

Joshua Albrecht is an Assistant Professor of Music Theory at the University of Iowa, and directs the Iowa Cognitive and Empirical Musicology lab. His current research blends statistical and computational musical analysis with behavioral studies to model listeners’ perception of musical affect, melodic and harmonic complexity, and intonation. Working in a traditional School of Music, his research also focused on applying computational methods to traditional historical and analytical problems, using compositional output as proxies for investigating the cognition of historical compositional practices.