FTAP

Capabilities
Distribution
Using FTAP
Performance
Hardware and Software Configuration
Schultz and van Vugt (2015)
Credit and Blame
Benchmarking Details
References
News (updated July 17, 2016)

FTAP is a Linux-based program (written by Steve Finney ) for tapping and music performance experiments. It connects to a standard MIDI input device (e.g., a keyboard or drum pad) for collecting MIDI keystroke data, and sends MIDI signals to a MIDI tone generator or sampler for audio output; MIDI messages are processed with millisecond accuracy and precision. FTAP is described in an article in Behavior Research Methods, Instruments, and Computers (Finney, S.A. (2001a), " FTAP: A Linux-based program for tapping and music experiments", v 33, pp 65-72). Please cite this article when writing up any research which makes use of FTAP.

FTAP is open source software and is available at no cost (but without warranty) in source code form under the GNU PUBLIC LICENSE. If you find it useful, let me know. If you have suggestions for improving it, let me know. If you find or fix bugs, let me know. I will try to respond to questions, and am prepared to fix bugs, clarify the documentation, etc. If FTAP looks interesting to you but you're having trouble getting it to work, please contact me.

Capabilities

The types of experiments FTAP can run includes, but is not limited to:

• Tapping experiments involving synchronization to a pacing signal, including synchronization/continuation paradigms. It is possible to specify a fairly complex pattern for the pacing signal (including simple polyrhythms), and the tempo of the pacing signal can be easily changed, either across trials or within a single trial. It is also possible to apply controlled perturbations to the pacing signal. (Example: Wing and Kristofferson, 1973)
• Auditory feedback manipulation experiments, including delayed feedback of any amount, combinations of synchronous and delayed feedback, and randomly delayed feedback. Delay can be changed in mid-trial, including isolated perturbations. (Examples: Aschersleben and Prinz, 1997; Repp, 2000; Wing, 1977)
• Pitch manipulation experiments: it is possible to control (within limits) the pitch which is produced by a particular keystroke. This is interesting for music performance; it also has potential application to sequence learning tasks. In addition, it's fun for musicians to play with. For instance, one built-in manipulation "reverses" the musical keyboard, so that low notes are on the right and high notes are on the left; Bach Two Part Inventions sound interesting that way. (Example: Finney, 1997)

In addition, two different labs have modified FTAP to sent signals to the parallel port and have used this to synchronize with EEG data collection; see, e.g., Maidhof, Kastner, and Makkonen (2014). This capability may appear in a future official FTAP release.

Back to top

Distribution (2.1.07)

The 2.1.07b distribution is basically a cleanup release: it compiles on 2016 vintage Linux systems, some resource limits have been increased, the documentation has been updated, and there have been other minor fixes and enhancements. See the Release Notes for more detail.

I also created a cleaned-up and more rigorous version of the loop test (again, see the Release Notes ). It runs for a longer period (60 seconds rather than 4 seconds), and can show up problems with MIDI interfaces that do not occur in the shorter test (see MIDI Interfaces). It continues to allow the user to verify on their own system that FTAP processes MIDI input and output messages with millisecond precision and accuracy, as it did in 2001 (Finney, 2001b). The 2.1.07 looptest parameter files may not work well with earlier FTAP releases.

The FTAP distribution can be downloaded as a gzipped tar file (about a megabyte). Extract it by first using "gunzip" and then the "tar" program; you will be left with a directory hierarchy named "ftap.2.1.07" (or something close). The README files and the included documentation should then get you started. You can also directly access the current User's Guide and Reference Manual in .pdf format.

See the News section below for bug reports/notes.

Download the FTAP 2.1.07b distribution

The MD5 checksum for the ftap2.1.07b distribution is 3038716858e0856bcb5bbdac6864a395. Run "md5sum ftap2.1.07b.tar.gz" on your downloaded file. The result should match.

NOTE: Although I have done a fair amount of testing, the 2.1.07b release should be considered preliminary until a few people besides me have compiled and run it on their systems (that's why I gave it an "b" suffix). I don't expect to have to make any significant changes, but you never know :-). If you have problems, please contact me. If you successfully run the looptest (see the Release Notes), or if it fails, please let me know what Linux release you're running and what MIDI hardware interface you used so I can add it to this web page. Thanks.

Older FTAP releases:

• FTAP 2.1.05 (2001)
• FTAP 2.1.06 (2004)
• FTAP 2.1.07a (2016)

Back to top

Using FTAP

Getting Started

See the User's Guide for an introduction to FTAP. You'll need a Linux PC with a hardware MIDI interface, a MIDI input device (keyboard or percussion pad), and a MIDI output device (a tone generator); see Configuration for some considerations. A trial is specified by a simple format ASCII text file, usually about 20 lines long. The easiest way to get some sense of how FTAP works (and its flexibility) is to look at and modify the samples file provided in the "params" directory in the distribution. Although actual data collection needs to be done with Linux "root" privileges to get millisecond accuracy and precision, I recommend doing initial evaluation as a normal user to avoid potential security issues. If you decide to collect data, you should run FTAP on a dedicated experimental machine. See the documentation for further details.

Suggested Practices

• When you create a parameter files for an experiment, run it and look at the output file in detail to make sure that what is happening is exactly what you intend.
• Don't use input devices that are obviously faulty (e.g., that miss or duplicate key presses).
• If you write code that parses the output (".abs" files), make sure any parsing of the file header uses keywords (e.g., "SUB" or "FACTOR1") rather than expected line number (which may change).

Back to top

Performance

The FTAP software runs on a Linux PC, running some version of the Linux OS (including drivers for USB, and the ALSA MIDI implementation), and uses a hardware MIDI interface (e.g., a USB-MIDI interface); I will refer to this collection of components as an "FTAP-MIDI" system. A properly configured FTAP-MIDI system performs with reliable millisecond accuracy and precision, and the user can (and is required to) confirm this with their own configuration by running the provided "looptest" parameter file (see the documentation and Release Notes). It is not necessary to install any custom Linux real-time patches (note that a millisecond is glacially slow on a 2 GHz processor).

To run an experiment, the FTAP-MIDI system must be attached to a MIDI input device and a MIDI output device; these devices will add some, hopefully small, latency. Some devices are known to be flawed (see the Devices section); it is the user's responsibility to choose appropriate devices for their requirements.

See Benchmarking for the gory details on validating the performance of an FTAP-MIDI system.

Back to top

Hardware and Software Configuration

The FTAP-MIDI system

As described above, by "FTAP-MIDI" system I mean the system comprised of FTAP software, a PC, the Linux OS and drivers, and a MIDI hardware interface. Such a system demonstrated millisecond accuracy and precision in 2001, and (not surprisingly) still does so on faster systems in 2016. My guess is that any recent PC and contemporary Linux distribution (using the default ALSA sound system) will be fine, but since you will be testing your own configuration with the loop test anyway, you'll be able to determine this for yourself. The loop test will also test your MIDI interface; as described below, some work better than others.

MIDI Interfaces

In 2001, driver choice (OSS vs 4Front) was an issue, and the FTAP looptest showed that some soundcard/driver combinations gave inadequate (10 ms resolution) MIDI performance. In 2016, the ALSA system used in current Linux system handles MIDI better, but individual MIDI interfaces still need to be verified with the loop test. I have tested 4 different USB-MIDI interfaces with FTAP 2.1.07a; three are acceptable, while one gave occasional errors and would best be avoided. These tests were run in a configuration in which my system (a quad-core Athlon running Ubuntu 14.04) was running in normal multi-user mode with networking and all services enabled; the worst-case performance would have been a little better if the system had been quiescent (as I recommend for data collection).

The 4 USB-MIDI interfaces I tested were:

• Roland UM-ONE mk2: Purchased in 2016 ($40).
• VicTsing: Purchased in 2016 ($8). (Do a web search for "VicTsing MIDI").
• Edirol UM-1EX: This was purchased 5 or 10 years ago and is no longer available; the Roland mk2 appears to be the replacement.
• M-Audio MIDISport UNO. This was purchased 5 or 10 years ago.

The results from 10 minutes of error-free, full-throttle MIDI throughput (using the "run_looptest.sh" and "run_loopcheck.sh" scripts included in the 2.1.07b release) are as follows; these statistics are based on the times between received input MIDI events (NOTEON and NOTEOFF).

                     Mean (ms)     Standard Deviation
Roland:                1.00               .22
VicTsing:               .96               .26
Edirol:                1.00               .22

The particular M-Audio interface I tested would often pass the short 4 second looptest, but would give errors on the more rigorous test (MIDI errors reported on the screen, or non-termination due to a dropped input event). Newer M-Audio devices may perform better; let me know. I will gladly add data on additional MIDI interfaces when provided by users.

Input and Output Devices

Schultz and van Vugt (2015) point out that the MIDI input and output devices will add some latency to data collection in FTAP; in particular, they provide data that a Roland HPD15 Handsonic percussion pad adds between 9 and 15 milliseconds of latency. They do not report data on any other input devices. On the other hand, Mills, van der Steen, Schultz, and Keller(2015) (apparently the same Schultz) report that the mean latency of the Roland Handsonic 10 percussion pad is less than 5 ms. Currently, there is a dearth of concrete data on the latency between tap and generation of the MIDI signal on both MIDI percussion pads and MIDI keyboards, as well as the latency between receipt of a MIDI signal and sound production on a specific tone generator (given that tone generators are purely electronic, my intuition is that the latency will be very low for a sharp-onset voice, but I have no data). Like USB-MIDI interfaces, there is likely to be a range of behavior. (See Finney (2016a) for additional discussion).

For now, I suggest not using the Roland Handsonic HPD-15 pad (or any other device that obviously misses and duplicates taps, as Schultz and van Vugt reported of the HPD-15). I will gladly report any data on the measured latency of specific MIDI input or output devices on this web page. Unfortunately, making such measurements is non-trivial, and I am not equipped to do it.

Back to top

Schultz and van Vugt (2015)

Benjamin Schultz and Floris van Vugt published a misleading article in Behavior Research Methods that incorrectly claimed that FTAP had poor latency (see the PubMed entry for Schultz and van Vugt, 2015) . I made a reply in "Pubmed Commons", but that facility has (unfortunately) been discontinued; however, my response is archived here (click on either of the annotation links). I made sure that both Benjamin Schultz and Floris van Vugt were aware of this comment, so their lack of response in PubMed Commons (during the few years it was available) suggests that they have no arguments against the statements there.

Schultz and van Vugt arrived at their conclusion by using a single input device that they knew to be flawed (a MIDI percussion pad with high latency that both missed and duplicated user input) and then attributing the percussion pad latency to the FTAP software package. Such confounding of conditions (and the use of a known flawed device in a paper purporting to do an objective technical comparison of experimental systems) would typically be grounds for rejection of a research paper.

However, the two manuscripts I submitted to correct their errors (e.g., their demonstrably false claim that USB-MIDI processing on a PC adds significant latency) were both rejected by the editor of Behavior Research Methods. You can read them and make your own evaluation.

My first submission provided a fair amount of background information about USB and MIDI devices, described FTAP's loop test, and tried to gently explain why Schultz and van Vugt's claims were incorrect. It also discussed why Linux continues to be a good operating system for millisecond-resolution data collection (addressing some of the concerns repeatedly raised in BRM by Richard Plant), and why MIDI is a good interface for sensorimotor experiments:
      Finney(2016a) , "In defense of Linux, USB, and MIDI systems for sensorimotor experiments: A response to Schultz and van Vugt (2015)".

In response to the concerns raised by the editor (see below), I submitted a second, shorter manuscript that more directly addressed the factual errors, poor experimental design, and misleading claims in Schultz and van Vugt (2015). It was also rejected by BRM. Anyone who reads the Schultz and van Vugt paper should also read this manuscript:
      Finney (2016b) , "Latency in sensorimotor synchronization experiments: Comments on Schultz and van Vugt (2015)".

Manuscript History

My initial submission ( Finney, 2016a) attempted to follow the editor's suggestion to be constructive, and had 3 reviewers. One reviewer was strongly positive ("Its points regarding the precision and accuracy of the Linux-FTAP-USB/MIDI connection are all valid and worth publishing"), while a second reviewer (anonymous, but clearly Schultz and/or van Vugt) was strongly negative (reviewer 3 was somewhat neutral). The manuscript was rejected as "correcting only a potential minor misconception". The editor also raised concern about "the broader reach of the paper".

The shorter manuscript ( Finney, 2016b ) that limited itself to correcting the factual errors and misleading claims in a published (and peer-reviewed) paper was rejected without any indication of textual review at all.

"Tap Arduino" limitations

Schultz and van Vugt (2015) suggest that their "Tap Arduino" is an inexpensive replacement for FTAP, which it is not. Tap Arduino is simply a button box that needs to be attached to a PC system to run any experiments at all, while an FTAP-MIDI system handles everything except the physical input and output devices. SVV provide no data that a complete experimental system using "Tap Arduino" has millisecond timing accuracy, and explicitly note (p 11) that this is an unresolved problem that may require "expensive options...for synchronizing responses and stimuli".

A validated Arduino device with a MIDI interface that could be attached to FTAP or MAX would be a useful contribution; unfortunately, Tap Arduino is not that device.

Back to top

Benchmarking Details

The claim that an FTAP-MIDI system (see definition above, and Figure 2 below) has millisecond accuracy and precision may surprise those who are aware of the potential issues with a multi-tasking operating system such as Linux, or who have read Schultz and van Vugt (2015). However, this was established in 2001 on a 200 MHz Pentium running Redhat 6.1 (see Finney, 2001b), and it is no less true on a 2016 vintage PC with a 2 GHz multi-core processor running a contemporary version of Linux (I'm using Ubuntu 14.04). For the skeptic, I will detail the argument here; see also Finney (2016a).



First, it is necessary to establish that the clock that FTAP uses to timestamp input and schedule output (the Linux "gettimeofday" system call) is accurate and precise at well below the millisecond level. This was established in Finney (2001b) using a set of simple C programs; these are included with the FTAP release in the "utils/gettod_test" directory. As shown in Finney (2001b), and as can be easily re-verified in 2016, this clock has resolution and accuracy at a little over 1 micro-second (since this clock is usually based on the same hardware clock that drives the multi-GHz processor, the accuracy is not surprising). FTAP uses these clock values as offsets from a fixed starting time, so the relation of the system clock to external time (e.g., UTC or whatever) is not important. Therefore, the millisecond times that FTAP uses (and reports in the output file) can be accepted as accurate and precise.



Second, it is necessary to run (and understand) what the supplied "looptest" parameter file is doing (this would be clearer with a diagram, but I don't have the resources for that right now; see Finney (2001b) and Finney (2016a) for further discussion). Because the connectors used for both MIDI input and output are identical, and because the MIDI message format for both input and output is identical, it is possible to connect the MIDI output back to the MIDI input using a cable or connector; by doing this, FTAP's output "tone" messages immediately reappear as "keystroke" messages. Since one of FTAP's most basic capabilities is providing synchronous feedback to keystrokes, it is possible to immediately send the received "keystroke" event back out as a "feedback" event. As a result, a single initial "output" event will be cycled through the system indefinitely. There are any number of possible sources for delay in this system: e.g., scheduling preemption in a multi-tasking operating system, hardware issues with the MIDI interface, or driver issues. Any of these issues, if important for millisecond resolution, will show up as delays in the output file of the "looptest".

However, if events (both input and output) are processed consistently once a millisecond (1000 events per second simultaneously on input and output), then FTAP can be claimed to be both precise and accurate at the millisecond level (note that the maximum hardware speed of MIDI transmission is 1,040 events per second, so it is not possible to go any faster than that). Finney (2001b) demonstrated that FTAP achieved this goal 15 years ago, and Schultz and van Vugt(2015) and Finney (2016a) both report that this was achieved in a more contemporary configuration. Further confirmation is provided in the USB-MIDI test results reported in the Configuration section above. Any FTAP user is required to perform this test on their configuration, as some MIDI interfaces may not work properly. Those who are concerned about validating with an external time reference can simply use a stopwatch to time the loop test.

I know of no other system for sensorimotor experiments that has been shown to pass such a rigorous test.

Back to top

Credit and Blame

FTAP has been a one-person project (me!). It is a fairly complex program, but one that I thought was of enough potential general use to be worth making public. I think it does a lot of stuff that no existing programs do (and some manuscript reviewers have expressed the same opinion), but I have discovered there is a huge gap between a useful program that the programmer can use and a program for public distribution. Two particular concerns:

• There's not a friendly installation script. On the other hand, FTAP itself is just a single executable file, so just place it in your home "bin" directory, or in "/usr/local/bin". Actual data collection should be be done with root privileges. More information is in the Release Notes, User's Guide and Reference Manual.
• If you're familiar with Linux or UNIX, then working with it might or might not be straightforward. If you can't get MIDI to work and you don't know what "setuid root" means, you'll probably need to get some outside assistance.

I would like to give a big "Thank you!" to Caroline Palmer and members of her lab, and to Pete Pfordesher, for both moral and logistical support (particularly in 2016). (An earlier postdoc with Caroline also enabled the initial FTAP port from SGI to Linux). I would also like to thank Clemens Maidhof for his published and GPL'd modifications to FTAP (Maidhof et al, 2014). Finally, I'm pleased to thank Saul Sternberg (who describes himself as an "enthusiastic FTAP user") for his ongoing assistance and encouragement, particularly with respect to trying to get a BRM correction published in 2016.

Back to top

References

1. Aschersleben, G., and Prinz, W. (1997), "Delayed auditory feedback in synchronization", Journal of Motor Behavior, v 29, pp 35-46.
2. Finney, S.A. (2001a), "FTAP: A Linux-based program for tapping and music experiments", Behavior Research Methods, Instruments, and Computers v 33, pp 65-72. Finney (2001a) in .pdf format.
3. Finney, S.A. (2001b), "Real-time data collection in Linux: A case study", Behavior Research Methods, Instruments and Computers, v 33, pp 167-173. Finney (2001b) in .pdf format.
4. Finney, S.A. (2016a), "In defense of Linux, USB, and MIDI systems for sensorimotor experiments: A response to Schultz and van Vugt". Finney (2016a) in .pdf format.
5. Finney, S.A. (2016b), "Latency in sensorimotor synchronization experiments: Comments on Schultz and van Vugt (2015)." Finney (2016b) in .pdf format.
6. Maidhof, C., Kastner, T., and Makkonen, T. (2014), "Combining EEG, MIDI, and motion capture techniques for investigating musical performance", Behavior Research Methods, v 46, pp 185-195.
7. Mills, P., van der Steen, M., Schultz, B., and Keller, P.(2015), "Individual differences in temporal anticipation and adaptation during sensorimotor synchronization", Timing and Time Perception, v 3, pp 13-31.
8. Repp, B.H.(2000), "Compensation for subliminal timing perturbations in perceptual-motor synchronization". Psychological Research, v 63, pp 106-128.
9. Schultz, B.G, and van Vugt, F.T. (2015). "Tap Arduino: An Arduino microcontroller for low-latency auditory feedback in sensorimotor synchronization experiments", Behavior Research Methods. Schultz and van Vugt(2015) abstract. (Please see Finney (2016b) and Finney (2016a) if you read or have read this article).
10. Wing, A.M. (1977), "Perturbations of auditory feedback delay and the timing of movement", Journal of Experimental Psychology: Human Perception and Performance, v 3, pp 175-186.
11. Wing, A.M., and Kristofferson, A.B. (1977), "The timing of inter-response intervals", Perception and Psychophysics, v 13, pp 455-460.

Back to top

News

Back to top