About ImpulsiveChoice

 Top Next

Purpose

Choice with delayed reinforcement (discrete-trial task).

Software requirements

Requires Whisker v2.0 or greater.

Data storage

· Text-based output to disk.
· ODBC data storage to a database (supplied).

Author

Rudolf Cardinal (rudolf@pobox.com).

Copyright

Copyright © Cambridge University Technical Services Ltd

Sample publications using this form of task (PMID refers to PubMed ID at http://www.pubmed.com)

· Cardinal RN, Robbins TW, Everitt BJ (2000). The effects of d-amphetamine, chlordiazepoxide, alpha-flupenthixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats. Psychopharmacology 152: 362–375. PMID 11140328.
· Cardinal RN, Pennicott DR, Sugathapala CL, Robbins TW, Everitt BJ (2001). Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292: 2499–2501. PMID 11375482.
· Cardinal RN, Cheung THC (2005). Nucleus accumbens core lesions retard instrumental learning and performance with delayed reinforcement in the rat. BMC Neuroscience 6: 9. PMID 15691387.
· Cheung THC, Cardinal RN (2005). Hippocampal lesions facilitate instrumental learning with delayed reinforcement but induce impulsive choice in rats. BMC Neuroscience 6: 36. PMID 15892889.
· Cardinal RN, Howes NJ (2005). Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neuroscience 6: 37. PMID 15921529.

The following articles illustrate a quantitative methodology (to be commended) to establish aspects of reinforcer delay/magnitude/probability sensitivity; this paradigm can also be accomplished with the present task.

· Kheramin S, Body S, Mobini S, Ho MY, Velázquez-Martinez DN, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2004). Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: a quantitative analysis. Psychopharmacology 165: 9-17. PMID 12474113.
· Kheramin S, Body S, Ho M, Velazquez-Martinez DN, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2003). Role of the orbital prefrontal cortex in choice between delayed and uncertain reinforcers: a quantitative analysis. Behavioural Processes 64: 239-250. PMID 14580695.
· Kheramin S, Body S, Ho MY, Velazquez-Martinez DN, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2004). Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology 175: 206-14. PMID 14991223.
· Bezzina G, Cheung TH, Asgari K, Hampson CL, Body S, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2007). Effects of quinolinic acid-induced lesions of the nucleus accumbens core on inter-temporal choice: a quantitative analysis. Psychopharmacology 195: 71-84. PMID 17659381.

Revision history

· First version written in 2002.
· v2.3 (22 Nov 2003). Switch to XML configuration file; minor bugfix; probabilistic choice added; quick-config buttons.
· ...
· Version 2.6 (22 Oct 2004): bugfix: TrialOverall ticked up one inappropriately between blocks.
· Version 2.7 (8 March 2007): easier compilation by users.
· Version 2.8 (29 Oct 2007): textual correction in delay/probability entry dialogue.
· Version 2.9 (13 June 2008): ImpulsiveChoice_AntiqueLevers version.
· Version 3.0 (12 Jan 2009): Server default changed from "loopback" to "localhost" (Windows Vista compatibility and more general standardization).


Required devices

 Top Previous Next

The program requires to claim devices in groups named box0, box1, box2… with device names as listed below in bold:

# Box 0 definition (sample)

line 0 box0 NOSEPOKE

line 3 box0 LEFTLEVER

line 6 box0 RIGHTLEVER

line 24 box0 HOUSELIGHT

line 27 box0 TRAYLIGHT

line 33 box0 PELLET

line 36 box0 LEFTLEVERCONTROL

line 39 box0 RIGHTLEVERCONTROL

line 42 box0 LEFTLIGHT

line 45 box0 RIGHTLIGHT

line 48 box0 PUMP

line 51 box0 DIPPER

Please ensure that these devices are available and listed in the device definition file in use by the server. (The snippet above shows an extract from a typical definition file.)

Note that if you are using the ImpulsiveChoice_AntiqueLevers version of the task, you do not neet LEFTLEVERCONTROL and RIGHTLEVERCONTROL, but you do need LEFTLEVERMOTOR (output), RIGHTLEVERMOTOR (output), LEFTLEVERPOSITION (input), and RIGHTLEVERPOSITION (input).


Using the task

When you run the task, the main screen looks as follows:

You must connect to a Whisker server, claim an operant chamber (box), and set up the parameters for your task. Once that's done, the traffic lights will turn amber. When you are ready, press Start to begin the task. When the task is running, it looks like this:

When the task finishes, it saves data to disk and pops up a new dialogue box for you to select a database to store the data to. (The data sources are configured under Control Panel → ODBC.) If you previously specified an ODBC data source in the parameters, that data source is used automatically and you will only see a dialogue box if something goes wrong and the program needs your input.


Parameters

The parameters dialogue box looks like this:

To pick an ODBC database in advance of finishing, click Pick and you will be offered the ODBC Data Source picker (below). Your choice will be recorded and will apply to this subject from now on (or until you specify a different source).

If you don't specify an ODBC data source now, or you delete the value in the "ODBC data source name" box, you'll be asked to choose when the task ends (and that choice will only apply to the session in progress).

To specify a list of delay values for reinforcer B, click the "Set delays" button:

Every time you click "Enter these values", your previous values are added to the list (shown near the top) and you can enter another pair of values.


Experimental methods (sample)

Excerpt from

Cardinal, R.N. (2001). Neuropsychology of reinforcement processes in the rat. Unpublished PhD thesis, University of Cambridge.

Copyright © Rudolf Cardinal 2001. All rights reserved.

Systematic technique for assessment of preference for delayed reinforcement

Eight identical operant conditioning chambers were used (30 × 24 × 30 cm; Med Instruments Inc, Georgia, Vermont, USA; Modular Test Cage model ENV-007CT). Each chamber was fitted with a 2.8-W overhead house light and two retractable levers with a 2.8-W stimulus light above each lever. Between the two levers was an alcove fitted with a traylight (60 mcd diffused green LED; RS Components Ltd, UK), an infrared photodiode to detect head entry (nosepokes), and a tray into which could be delivered 45-mg food pellets (Rodent Diet Formula P, Noyes, Lancaster, NH). The chambers were enclosed within sound-attenuating boxes fitted with fans to provide air circulation. The apparatus was controlled by software written by R.N. Cardinal in C++ using the Whisker control system (Cardinal & Aitken, 2002) [altered from original thesis, for which I used an antiquated control system].

Training. Subjects were first trained under an FR1 schedule to a criterion of 50 presses in 30 min, first for the left lever and then for the right. They were then trained on a simplified version of the full task. The session began with the levers retracted and the operant chamber in darkness. Every 40 s, a trial began with illumination of the houselight and the traylight. The subject was required to make a nosepoke response within 10 s, or the current trial was aborted and the chamber returned to darkness. If the subject nosepoked within this time limit, the traylight was extinguished and a single lever presented. If the rat failed to respond on the lever within 10 s, the lever was retracted and the chamber darkened, but if it responded, a single pellet was delivered immediately and the traylight was illuminated until the rat collected the pellet (or a 10-s collection time limit elapsed, whereupon the chamber was darkened). In the Houselight condition, the houselight was left on until 6 s after the food had been collected; in the Cue and No Cue conditions it was switched off at the moment the lever was pressed.

Cues during the delay

Choice of signalled and unsignalled delayed reinforcement. Subjects may choose between a small, immediate reward and a large, delayed reward. In the 'Cue' condition, a stimulus light is illuminated during the delay to reinforcement; this stimulus is therefore paired with the large reinforcer and may become a conditioned reinforcer.

In every pair of trials, the left lever was presented once and the right lever once, though the order within the pair of trials was random. Rats were trained to a criterion of 60 successful trials in one hour (the maximum possible with a 40-s period being 90).

Behavioural procedure. The task was based on Evenden and Ryan's (1996) procedure and is illustrated in figure…. Aside from the use of an extra signal during the delay, the present task differs from that of Evenden and Ryan in a number of ways; in particular, the subjects were required to initiate the trials and choose a lever within a limited time, and a forced-choice trial on each lever was given at the start of each block of choice trials at a given delay. Additionally, in their procedure the houselight was always on, whereas in the present studies the houselight was extinguished during the intertrial interval (ITI), making it an informative stimulus (in that food was delivered when the houselight was on, but never when it was off). Finally, subjects were not given exposure to the large reinforcer before delays were introduced into the task.

The session began in darkness with the levers retracted; this was designated the intertrial state. Trials began at 100-s intervals. Each trial began with the illumination of the houselight and the traylight. The rat was required to make a nosepoke response, ensuring that it was centrally located at the start of the trial (latency to poke was designated the initiation latency). If the rat did not respond within 10 s of the start of the trial, the operant chamber was reset to the intertrial state until the next trial began and the trial was scored as an omission. If the rat was already nosepoking when the trial began, the next stage followed immediately.

Evenden task schematic (with all three conditions)

Schematic of the task. On the right-hand side, the format of a single trial is shown. This diagram shows in detail the Houselight condition, in which the houselight remains on from the start of the trial until 6 s after the subject has collected the reward. On the left-hand side, the differences between the three lighting conditions are illustrated. In the No Cue condition, the houselight is switched off at the moment of choice. In the Cue condition, the houselight is similarly switched off when the subject responds on a lever, but a stimulus light is illuminated during the delay that precedes delivery of the large reinforcer.

Upon a successful nosepoke, the traylight was extinguished and one or both levers were extended. One lever was designated the Delayed lever, the other the Immediate lever (counterbalanced left/right). The latency to choose a lever was recorded. (If the rat did not respond within 10 s of lever presentation, the chamber was reset to the intertrial state until the next trial and the trial was scored as an omission.) When a lever was chosen, both levers were retracted. Choice of the Immediate lever caused the immediate delivery of one pellet; choice of the Delayed lever caused the delivery of 4 pellets following a delay. In the Cue condition, the houselight was switched off at the moment of choice and a stimulus light above the chosen lever switched on for the duration of the delay. In the No Cue condition, the stimulus light was not switched on. In the Houselight condition, the houselight remained on instead. These three conditions are illustrated in figure….

Following any delay, the stimulus light was switched off, the traylight was switched on and the reinforcer for that lever was delivered. Multiple pellets were delivered 0.5 s apart. If the rat collected the pellets before the next trial began, then the time from delivery of the first pellet until a nosepoke occurred was recorded as the collection latency. The traylight was switched off, and in the Houselight condition the houselight remained on for another 6 s (eating time). In other conditions there was no houselight illumination during this time. If the rat did not collect the food within 10 s of its delivery, the operant chamber entered the intertrial state, though collection latencies were still recorded up to the start of the next trial. The chamber was then in the intertrial state and remained so until the next trial. There was no mechanism to remove uneaten pellets, but failure to collect the reward was an extremely rare event (see Results).

The delay was varied systematically across the session. A session consisted of 5 blocks, each comprising two trials on which only one lever was presented (one trial for each lever, in randomized order) followed by ten free-choice trials. Delays for each block were 0, 10, 20, 40 and 60 s respectively. As trials began every 100 s, the total session length was 100 minutes; subjects received one session per day.


Version for antique levers

 Top Previous

Nearly all retractable/extendable operant chamber levers on the market are controlled by a single (output, from the computer's point of view) line. When the line is on (1), the lever extends and stays extended for as long as this control line is on. When the lever is off (0), it retracts. In addition, there is a response (input) line: 1 = lever depressed, 0 = lever not depressed.

However, some old (1980s?) levers from Campden Instruments, which are easily recognized because they require mains voltage (in the UK, 240 V AC) - and therefore require considerable respect when installing and handling them! - operate differently. They have the following control system (Julie Gill and David Maul, Campden Instruments, personal communication, June 2008):

· each lever has a response line (input): 1 = lever depressed, 0 = lever not depressed
· there is also a lever position line (input): 1 = lever retracted, 0 = lever extended
· and there is a lever motor line (output): this is normally held at 0, but a 40-100ms pulse to 1 (and then back to 0) latches the lever motor on. If the lever was extended, this pulse causes it to retract; if it was retracted, the pulse causes it to extend.

The ImpulsiveChoice_AntiqueLevers.exe program is a separate executable from the usual ImpulsiveChoice.exe program, and it supports these old levers. In all other respects it is identical to the main task. The required devices (q.v.) are slightly different; this is deliberate, so you can't accidentally run the wrong version of the task and not notice. There is also a message on the main window to announce the fact that you are using the "antique levers" version.

The program does not support the levers in a very sophisticated way, but as follows:

· Whenever the program wishes the levers to change state, it checks the current state by asking the lever. If the current state is what the computer thought it was, then it pulses the motor to change the state. If it wasn't, then the lever isn't working properly; it flags this on screen and in the text log file (and doesn't pulse the motor, since the lever is already in the target state).