Running perception and artificial grammar learning experiments
The Phonology Lab is equipped to run experiments that involve listening and responding to stimuli. Three sound-attenuating booths are set up with three PCs. A participant can sit inside one of the booths, look through the window at a computer screen, listen to stimuli on headphones, and respond using a mouse, a computer keyboard, or a Cedrus 7-button response pad (Figure 1).
Two kinds of experiments that may be run with this equipment are perception experiments (eliciting responses to naturally produced or synthesized audio stimuli), and artificial grammar learning experiments, which involve an exposure phase followed by a test phase that is similar to a conventional perception experiment. Perception and artificial grammar learning experiments both typically involve presenting stimuli to experiment subjects and eliciting a response, often in the form of a choice between a small set of alternatives (such as “same”/”different”, or “grammatical”/”ungrammatical”, or a rating on a 7-point scale).
You have a variety of options when choosing how to design your perception experiment; we suggest using PsychoPy or Praat. The Phonology Laboratory has the freely available PsychoPy on the three pcs located outside the sound booths. A good reason to use PsychoPy instead of Praat is if you want to measure reaction times. It is capable of recording more accurate reaction times than Praat. This level of accuracy is particularly important in experiments designed to measure some cognitive function or sub-conscious reaction.
Other professional experiment presentation programs include Presentation, E-prime, PsyScope, and DMDX. Whichever program you use to run your experiment, you need a script that tells the program how to present your stimuli and what kind of information to record about the responses. The easiest way to do this is to modify an existing experiment.
Running the Button Box for the First Time
If you’re running the Button Box for the first time on your user or using it with another computer, you will likely have to update and install the cedrus extension. Follow the instructions on the Cedrus website for step 1 only (if you are using PsychoPy) or steps 1-2 (if you are using Presentation.
Praat Multiple Forced-Choice (MFC) experiments
It is helpful to think about the similarity between a researcher auditorily coding data and a subject participating in a perception experiment. Both are situations where a person subjectively evaluates speech, and there is a lot of overlap in the methods. Praat’s Multiple Forced Choice (MFC) experiment feature is very handy for quickly eliciting responses to stimuli (whether you think of it as labeling or an experiment).
Creating a MFC script with one_script
one_script’s mfc() procedure will extract clips from corpora for you and make a Praat MFC script you can use to listen to them. Here is a sample command to pull out the seven tokens of “like” in the Buckeye corpus sample recordings and present them for you to categorize as monophthongized or not monophthongized:
praat /phon/scripts/one_script.praat /phon/Buckeye/buckeye_test.csv 'l/ay/k' 'mfc(categories="monophthongized/not_monophthongized")'
Most of this command works exactly the way you’re used to one_script commands working. The differences are in how the mfc() procedure works. Make sure you don’t include any spaces in the categories. Use underscores instead. They will get replaced with spaces in the end. If you want to include the words before and after your target word in the clips, add the argument trigrams=True:
praat /phon/scripts/one_script.praat /phon/Buckeye/buckeye_test.csv 'l/ay/k' 'mfc(trigrams=True,categories="monophthongized/not_monophthongized")'
Here is a sample set of output files:
- one_script_out_2017Apr03_11h37.csv (lists all the tokens)
- one_script_log_2017Apr03_11h37.csv (the log file)
- one_script_out_2017Apr03_11h37_MFC.praat (the MFC script itself)
- clips_2017Apr03_11h37/ (a directory containing all the sound clips reference in the MFC script)
If you aren’t going to run the MFC experiment directly on the computer where you ran one_script, you will want to zip the files download them. To zip all these files (recursively) to a file called mymfc.zip, do this (adjusting the date/time stamp to your situation):
zip -r mymfc *2017Apr03_11h37*
Download and unzip the files as usual, and then open the …MFC.praat file in Praat and follow these steps:
- select the ExperimentMFC and click Run (and listen to and code all the clips)
- select the ExperimentMFC again and click Extract results
- select the ResultsMFC and click Collect to Table
- select the Table and click Save…Save as comma-separated file…
If you gave the table the default name (allRestults.Table), this is how you can work with the data in R:
outfile <- read.csv('one_script_out_2017Apr03_11h37.csv', h=T) results <- read.csv('allResults.Table', h=T) names(results)[1:2] <- c('scriptname', 'clip_filename') data <- merge(outfile, results)
Now your data frame called “data” contains the token information from one_script_out and the judgments from the MFC experiment.
If you have responses from more than one listener, you will have more than one results table file (but still only one outfile). Open two (or more) results tables like this:
outfile <- read.csv('one_script_out_2017Apr03_11h37.csv', h=T) results_s01 <- read.csv('allResults_s01.Table', h=T) results_s02 <- read.csv('allResults_s02.Table', h=T)
To put them in a long data frame with a column for subject, do this:
names(results_s01)[1:2] <- c('scriptname', 'clip_filename') names(results_s02)[1:2] <- c('scriptname', 'clip_filename') data_s01 <- data.frame(subject='s01', merge(outfile, results_s01)) data_s02 <- data.frame(subject='s02', merge(outfile, results_s02)) data <- rbind(data_s01, data_s02) data$subject <- factor(data$subject)
To put them in a wide data frame with separate columns for each subject’s responses, do this:
names(results_s01) <- c('scriptname', 'clip_filename', 'response_s01', 'reactionTime_s01') names(results_s02) <- c('scriptname', 'clip_filename', 'response_s02', 'reactionTime_s02') data <- merge(outfile, results_s01) data <- merge(data, results_s02)
Doing these with more than two subjects works exactly how you would probably guess it works.
Other things involving MFC scripts
Another useful script for our purposes extract_and_label.praat (which is now superseded by one_script). This script will find all the matching sequences in a TextGrid so you can work with the corresponding audio clips. One thing it can do is generate an MFC script to play all your extracted sound back to you, and saving your responses in a spreadsheet. Two related files you will probably want are the instructions and the config file. This script can also facilitate coding speech while looking at the spectrogram in the editor, and other things like extracting frames from videos based on a TextGrid of the audio channel. Importantly, once you have used the script to make an MFC script, you can make modifications to the MFC script directly.
Since Praat gives fewer options than Presentation, you will probably find that using Praat to make experiments is more intuitive, at least at first. See this page for instructions on using Praat to run perception experiments. You can modify someone else’s script (e.g., frr_MFC_experiment.praat), or you can modify one of the sample scripts on the Praat experiment page. Topics covered on this page which may be relevant to designing your own Praat MFC experiment include: Goodness judgments, Randomization strategies, and The carrier phrase.
Modifying an MFC script
To modify frr_MFC_experiment.praat for use with your own recordings, the first thing you need to do is open frr_MFC_experiment.praat in a text editor so that you can make several changes. Praat is very picky about the format of this file, and the error messages for MFC scripts are not as helpful as the error messages for shell scripts, so be careful to follow the instructions closely the first time you try modifying an MFC script. You will probably find it useful to use your text editor’s search function to find the key phrases referred to here. Here are some things you will need to change before the script will work:
- On the line starting with stimulusFileNameHead, change the path from “~/Desktop/output/sounds/” to the path where the wav files are. If you use Windows, you will probably need to use backslashes instead of frontslashes.
- On the line starting with numberOfDifferentStimuli, change the value from 430 to the actual number of wav files you want to use.
- Use the next lines to list every wav file on a separate line. Each line should start with four spaces and have the wav file name in double quotes, followed by a space and two double quotes, and nothing else. Because stimulusFileNameTail is set to “.wav”, you should exclude the “.wav” part of each filename, because it will be added automatically. If you have more than a few sounds to list, don’t try to format these lines manually. Do something clever with a text editor or a spreadsheet program instead (and ask someone for help if you don’t know a way to do it). For example, if you are a Windows user, highlight all of the wav files you wish to include and select “copy path”. If you then paste this list of paths into a text editor, it will be much easier to format the files names such that they are compatible with a Praat MFC script.
After you make these changes and save your script, try running it before making any other changes. In Praat, choose “Read… Read from file…”. Then, with the ExperimentMFC object highlighted, click “Run”. After the subject finishes, highlight the ExperimentMFC object again and click “Extract results”. Then highlight the ResultsMFC object and save it using “Write… Write to text file…”. Then you can open this file in a text editor or spreadsheet to see the results. If it doesn’t work, you will need to figure out what went wrong and possibly start over. If it works, you will probably want to customize the script further. Here are some parameters you are likely to want to change:
- numberOfReplicationsPerStimulus does exactly what it sounds like it does.
- breakAfterEvery determines the frequency of breaks. 0 means no breaks, and 100 means break after every 100th trial.
- randomize specifies the randomization scheme. <PermuteBalancedNoDoublets> is a good choice, and you can read about others in the on-line documentation.
- Change startText and runText to instructions that are appropriate for your experiment.
- If you don’t want there to be three options, change numberOfDifferentResponses, and modify the next three lines so that the number of lines matches the number you entered. Each line generates a button. The first four numbers control a button’s size and position (you can experiment by making small changes to them), the first string is the text that will appear on the button, and the last string is the text that will appear in the output file.
- There are some other parameters at the bottom and top of the file that you may want to change (e.g., the amount of silence to include before playing a sound, and what text appears to announce a pause). The parameter names are mostly self-explanatory.
PsychoPy is a robust, open-source alternative to Presentation built on Python and developed for Windows, Mac, and Linux environments. The suite has a user-friendly Builder interface that allows you to visualize the order of stimuli and modules as well as an Coder interface for those interesting in tinkering with the code for advanced functions. PsychoPy supports many of the same features as Presentation, including audio, graphics, and text stimuli, randomization of stimuli, inclusion of loops and trials, and input from microphones, mouse/keyboard, and button box.
For an in-depth tutorial on PsychoPy with examples and suggestions, follow this link.
The Phonology Lab no longer maintains a license for Presentation, but the following information may still be useful. If you are running a perception experiment (rather than just coding some sound files), you will probably want to use Presentation instead of Praat, unless you are collaborating with someone who uses E-prime. The easiest way to make an experiment in Presentation is to modify an existing one that is already similar to what you want. That way you don’t have to learn about everything all at once. Just change one small thing at a time and then see whether or not that broke the experiment. The following instructions are meant to be a very superficial introduction, to get you started with tweaking existing experiments, with the idea that if you need to do more than that, you will need to learn more (by reading the documentation, asking someone, or hiring a consultant).
The documentation available under the Help menu can be helpful, but it’s easy to get lost because Presentation can do a lot more than what we use it for. NeuroBehavioral Systems also has consultants that will make an experiment for you, for a reasonable price (about $500 for a moderately complicated perception experiment).
Parts of a Presentation experiment
THE EXPERIMENT FILE
The experiment (.exp) file is the one you will typically open. When you double click on it, Presenation will open, and you will see all the files associated with your experiment. Typically you will have at least one Scenario file, at least one PCL file, a set of stimuli, a text file specifying how the stimuli are to be presented, and a few miscellaneous files that you may not ever see.
Scenario (.sce) files are the modules of your experiment. Your experiment may include one or more of these. If you click on the “Main” tab, you will see the active scenarios. If you will have similar scenarios (e.g. two blocks that are basically the same with minor differences). You will probably want to have a template scenario file that is referred to in each scenario file, like this:
This is very similar to the source command in R and the include command in the Praat scripting language.
Scenario files are analogous to the Praat MFC experiment files, in that they contain a lot of parameters, but the actual instructions for what to do are located somewhere else. With Praat MFC experiments, we don’t see these instructions. With Presenation experiments, these instructions are in PCL files.
The PCL (.pcl) files include the sequence of events that will take place in your experiment. Unfortunately, .pcl files and .sce files use different scripting languages, and refer to some of the same things in different ways. The structure of a .pcl file is more like a regular Praat script than it is like a Praat MFC experiment script.
Some other files you are likely to encounter are your audio stimuli, which you list in a scenario file, and a text file that says how to construct prompts from them. There are also Set Definition (.sdf) files that are related to how the output files are formatted.
The “Response” button, accessible from the “Settings” tab, deals with the response devices used in your experiment. Typically, you will use either a Cedrus response pad, a mouse, or a regular computer keyboard. Some experiments (those that prompt the participant to read words aloud) don’t have a response device associated with them. You define the response device and active buttons for each scenario in your experiment, and you can make use of the “copy to/from default” options you get by right-clicking on a scenario.
Presentation typically produces two types of output files, containing a lot of the same information. You can open them in a spreadsheet program or make a Python script to parse them.
Collaborating with people who use E-Prime
E-prime is similar to Presentation. We have a run-time license that allows us to run E-prime experiments that other people have created, and we can make minor changes to experiments using a text editor (but not E-prime’s graphical interface). The situation where you are most likely to use E-prime in our lab is when running subjects for an experiment involving collaborators in another lab where E-prime is used instead of Presentation.
Processing and analyzing perception data