Does mathematical modeling results in formulas that give exact values of real

While there is no consensus yet as to a precise definition of this term, mathematical modeling is generally understood as the process of applying mathematics to a real world problem with a view of understanding the latter. One can argue that mathematical modeling is the same as applying mathematics where we also  start with a real world problem, we apply the necessary mathematics, but after having found the solution we no longer think about the initial problem except perhaps to check if our answer makes sense. This is not the case with mathematical modeling where the use of mathematics is more for understanding the real world problem. The modeling process may or may not result to solving the problem entirely but  it will shed light to the situation under investigation. The figure below shows key steps in modeling process.

Mathematical modeling approaches can be categorized into four broad approaches: Empirical models,  simulation models, deterministic models, and stochastic models. The first three models can very much be integrated in teaching high school mathematics. The last will need a little stretching.

Empirical modeling involves examining data related to the problem with a view of formulating or constructing a mathematical relationship between the variables in the problem using the available data.

Simulation modeling involve the use of a computer program or some technological tool to generate a scenario based on a set of rules. These rules arise from an interpretation of how a certain process is supposed to evolve or progress.

Deterministic modeling in general involve the use of equation or set of equations to model or predict the outcome of an event or the value of a quantity.

Stochastic modeling takes deterministic modeling one further step. In stochastic models, randomness and probabilities of events happening are taken into account when the equations are formulated. The reason behind this is the fact that events take place with some probability rather than with certainty. This kind of modeling is very popular in business and marketing.
Examples of mathematical modeling can be found in almost every episode of the TV hit drama series The Numbers Behind NUMB3RS: Solving Crime with Mathematics

The series depicts how the confluence of police work and mathematics provides unexpected revelations and answers to perplexing criminal questions. The mathematical models used may be way beyond K-12 syllabus but not the mathematical reasoning and thinking involve. As the introduction in each episode of Numbers says:

We all use math every day;
to predict weather, to tell time, to handle money.
Math is more than formulas or equations;
it’s logic, it’s rationality,
it’s using your mind to solve the biggest mysteries we know.

The Mathematics Department of Cornell University developed materials on the mathematics behind each of the episodes of the series. You can find the math activities in each episode here.

The challenge in mathematical modelling is “. . . not to produce the most comprehensive descriptive model but to produce the simplest possible model that incorporates the major features of the phenomenon of interest.” -Howard Emmons

J Exp Anal Behav. 2006 Mar; 85(2): 275–291.

Abstract

The use of mathematical models in the experimental analysis of behavior has increased over the years, and they offer several advantages. Mathematical models require theorists to be precise and unambiguous, often allowing comparisons of competing theories that sound similar when stated in words. Sometimes different mathematical models may make equally accurate predictions for a large body of data. In such cases, it is important to find and investigate situations for which the competing models make different predictions because, unless two models are actually mathematically equivalent, they are based on different assumptions about the psychological processes that underlie an observed behavior. Mathematical models developed in basic behavioral research have been used to predict and control behavior in applied settings, and they have guided research in other areas of psychology. A good mathematical model can provide a common framework for understanding what might otherwise appear to be diverse and unrelated behavioral phenomena. Because psychologists vary in their quantitative skills and in their tolerance for mathematical equations, it is important for those who develop mathematical models of behavior to find ways (such as verbal analogies, pictorial representations, or concrete examples) to communicate the key premises of their models to nonspecialists.

Keywords: mathematical models, equations, behavior, reinforcement

Over the years, researchers who study basic behavioral processes have increasingly relied on mathematical models in their work. The establishment of the Society for the Quantitative Analyses of Behavior (SQAB) in 1978 and its subsequent growth is just one sign of the rise of interest in the mathematical modeling of behavior. Another measure is the number of articles in the Journal of the Experimental Analysis of Behavior (JEAB) that use mathematical models. For individual years separated by 10-year intervals, Figure 1 shows the percentage of JEAB articles that presented at least one equation to describe the relation between an independent variable and a dependent variable (not including articles that tested the implications of mathematical models but did not explicitly present the equations). There is some arbitrariness in this criterion, but the increasing trend toward mathematical analysis is obvious.

For individual years at 10-year intervals, the percentage of articles in JEAB that included at least one equation to describe the relation between an independent variable and a dependent variable.

For some behavior analysts who began their careers when mathematical modeling was not so commonplace in this field, or for those who do not use mathematical models in their own work, this trend may be disconcerting. For some, the sight of an equation or two in a JEAB article may be reason enough for them to skip over the article and move on to the next. They may feel that an article with mathematical equations is beyond their comprehension, or worse, irrelevant to their interests. After all, isn't the experimental analysis of behavior supposed to be about behavior, not about mathematical equations and symbols?

Failures of communication between those who do and those who do not use mathematical models in their work are not unique to psychology. As the science of physics became increasingly quantitative in the mid-nineteenth century, some very competent scientists felt as if they were being left behind. During the course of a very productive career, the British physicist Michael Faraday conducted some fifteen thousand experiments on electromagnetic fields, and yet he could not comprehend the beautiful equations of field theory that James Clerk Maxwell derived from Faraday's own research. Faraday asked Maxwell if he could use verbal descriptions or other means to make his equations comprehensible to a nonmathematician, and Maxwell made a sincere effort to do so. According to Timothy Ferris (1988):

Maxwell obligingly rendered some of his explanations of field theory into the mechanical cogwheels and sprocket formulations that Faraday could understand, but it was when stripped to bare equations that his theory flew. With fuguelike balance and power, Maxwell's equations demonstrated that electricity and magnetism are aspects of a single force, electromagnetism, and that light itself is a variety of this force. Thus were united what had been the separate studies of electricity, magnetism, and optics. (p. 187)

It is not clear whether mathematical models in psychology can ever achieve the breadth and power of Maxwell's equations; the behaviors of living organisms are very complex, and they are influenced by many different factors. Nevertheless, mathematical models can serve several useful functions in the experimental analysis of behavior. In psychology, mathematical models are designed to describe basic behavioral processes in a more precise way than can be done with simple verbal descriptions. It sometimes is difficult to derive unambiguous predictions from theories that are expressed in words, but the implications of competing theories (and differences between them) often become clear when the theories are presented in mathematical form. Critical tests that compare the quantitative predictions of two or more different models can indicate which hypotheses about a behavioral process are viable and which are not. Studies that test the quantitative predictions of mathematical models can identify insufficiencies in current theories and draw our attention to factors affecting behavior that might otherwise have been overlooked. In addition, just as Maxwell's equations unified different areas of physics, a good mathematical model in psychology can use a core set of principles to account for diverse behaviors. Finally, mathematical models can give guidance to workers in related fields, such as a behavior therapist attempting to predict and control behavior in an applied setting, or a neuroscientist seeking to understand the biological bases of a behavioral phenomenon.

One purpose of this article is to demonstrate the value of mathematical models in the experimental analysis of behavior, using several specific examples. A second purpose is to encourage better communication between those psychologists who use mathematical models in their research and those who do not, as exemplified by the communication between the experimenter Faraday and the mathematician Maxwell.

Better than Mere Words

In some cases, two theories that appear to make similar predictions when stated in words can be more readily compared and evaluated when they are put into mathematical form. For example, two different theories about punishment are the negative law of effect (e.g., Rachlin & Herrnstein, 1969) and the avoidance theory of punishment (e.g., Dinsmoor, 1954, 1977). The negative law of effect is simply the view that reinforcement and punishment have opposite effects on behavior: reinforcement strengthens behavior and punishment weakens behavior. The avoidance theory of punishment takes a different approach. According to this theory, when a response is followed by a punisher, the response becomes associated with fear, which is an aversive stimulus. Once this conditioning has occurred, fear will increase whenever the animal starts to make the response, and the animal can escape from this fear by performing some other response instead. Alternative responses are therefore reinforced by fear reduction, and they increase in frequency. Therefore, according to the avoidance theory, the effects of punishment are indirect: the reason a punished behavior decreases is that the level of reinforcement for alternative behaviors has increased.

Described in this way, the predictions of these two theories are difficult to distinguish. Although they make their predictions for different reasons, both theories seem to predict the same general result: punishment will cause a decrease in the punished behavior. Once they are translated into mathematical form, however, the different predictions of the two theories can be seen more easily. Deluty (1976) and de Villiers (1977, 1980) developed two different quantitative models of punishment, which can be viewed as mathematical versions of the avoidance theory of punishment and the negative law of effect, respectively. Both models begin with Herrnstein's (1961) matching law, but then proceed in different directions.

In its simplest form, the matching law can be written as follows:

1

where B1 and B2 are the rates of response on reinforcement schedules 1 and 2, and R1 and R2 are the rates of reinforcement on these two schedules. This equation has often been applied to choice situations in which the two alternatives are variable-interval (VI) schedules of food reinforcement. Imagine that a pigeon responds on two keys, with Key 1 delivering 75 reinforcers per hour and Key 2 delivering 25 reinforcers per hour, so Equation 1 predicts that the pigeon will make 75% of its responses on Key 1. Now suppose that in addition to producing food, responses on both keys begin to deliver punishers (electric shocks) at a rate of 20 shocks per hour for each key. How can Equation 1 be expanded to deal with this situation?

According to de Villiers (1977), if punishment is the opposite of reinforcement, as the negative law of effect states, then the punishers delivered by each alternative should be subtracted from the reinforcers delivered by that alternative:

2

where P1 and P2 are the rates of punishment on the two keys.

In contrast, Deluty (1976) took the view that punishing one response increases the reinforcement for other responses, as proposed by the avoidance theory of punishment. Therefore, in his equation, the punishers for one alternative are added to the reinforcers for the other alternative:

3

To keep this example simple, one shock is given the same weight as one food delivery, but both models could easily give food and shock different weights by multiplying P1 and P2 by some constant other than 1. Using such a constant would not change the general conclusions presented here. In this example, with R1 = 75, R2 = 25, and P1 = P2 = 20, Equation 2 predicts that the percentage of responses on Key 1 should increase from 75% to 92% when the shocks are added to both keys. Conversely, Equation 3 predicts that the percentage of responses on Key 1 should decrease to 68% when the shocks are added. In an experiment with pigeons, de Villiers (1980) found that preference for the key that delivered more reinforcers increased when shocks were added to both keys with equal frequency. This result therefore favors the predictions of Equation 2 over those of Equation 3.

It should be clear that the issue here is more fundamental than simply whether a plus sign or a minus sign should be used in an equation. These two models are based on two very different conceptions of how punishment exerts its effects on behavior. The experimental evidence suggests that punishment exerts its effect by weakening the target behavior, as the negative law of effect stipulates, not by strengthening alternative behaviors, as the avoidance theory proposes. This example illustrates how two psychological theories that seem to make similar predictions when stated verbally actually may make very different predictions when they are presented in mathematical form.

More than Curve Fitting

Consider the question of what equation best describes how the strength of a reinforcer decreases with increasing delay. One common suggestion (e.g., Hull, 1943; Killeen, 1994) is that the delay-of-reinforcement gradient can be described by an exponential equation:

4

where V is the value or reinforcing strength of a reinforcer delivered after a delay of D seconds, A represents the value of the reinforcer if it were delivered immediately, e is the base of the natural logarithm, and K is a parameter that determines how rapidly V declines with increasing delay. Another proposal is that the delay-of-reinforcement gradient is best described by a hyperbolic function (e.g., Mazur, 1987):

5

These two different equations describe decay curves that have fairly similar shapes. Figure 2 shows the data from 1 pigeon in an experiment in which the animals chose between 45 s of exposure to variable-time (VT) schedules and single presentations of a delayed reinforcer (Mazur, 2000a). The delay was adjusted over trials to obtain the indifference points shown in the graph, which depicts the decreasing value of the single reinforcer as its delay increased. The curves show the best-fitting predictions of Equations 4 and 5 with K treated as a free parameter, and both equations account for 99% of the variability in the data.

An exponential function (Equation 4) and a hyperbolic function (Equation 5) are fitted to the data from 1 pigeon from Mazur (2000a).

For both equations, K was treated as a free parameter.

One could argue that both equations describe the data very well, and that deciding which one to use is simply a matter of preference. Although that may be true for this single set of data, it would be wrong to conclude that the difference between Equations 4 and 5 is inconsequential. These two equations make profoundly different predictions about how individuals will choose between two reinforcers that are delivered at different times (as in the so-called self-control choice situation, in which an individual must choose between a small, more immediate reinforcer and a larger, but more delayed reinforcer). Economists generally have favored the exponential equation as a temporal discounting function because it seems more “rational”: all reinforcers are discounted by the same percentage as time passes, regardless of their sizes or when they are delivered. However, as discussed by Ainslie (1975), if the discounting parameter, K, is the same for both reinforcers, the exponential equation does not allow for preference reversals in a self-control choice situation: a person who prefers a larger more delayed reinforcer now should maintain this preference as time passes. In contrast, the hyperbolic equation does predict preference reversals (as when a person on a diet vows that he will not have a second helping at the evening meal, but then changes his mind during the meal and eats more than he should).

The situation is actually more complex than this because, as Green and Myerson (1993) pointed out, the exponential equation can predict preference reversals if the decay parameter, K, is greater for the smaller reinforcer than for the larger reinforcer. When human participants are asked to make choices between hypothetical monetary reinforcers of different sizes and delays, estimates of K do indeed decrease with increasing reinforcer amounts (e.g., Green, Fristoe, & Myerson, 1994; Green, Myerson, & McFadden, 1997), so preference reversals in these situations are not necessarily inconsistent with the exponential equation. To provide more convincing evidence for the hyperbolic equation, one needs to show that preference reversals occur even when estimates of K do not. Research with nonhuman subjects has provided some evidence of this type. Whereas preference reversals are reliably found with animals (e.g., Green, Fisher, Perlow, & Sherman, 1981), studies with rats and pigeons have found no evidence that the values of K decrease with larger reinforcer amounts (Grace, 1999; Green, Myerson, Holt, Slevin, & Estle, 2004; Ong & White, 2004; Richards, Mitchell, de Wit, & Seiden, 1997). There is also other evidence that favors the hyperbolic equation, such as the shapes of the indifference functions that are obtained when animals choose between different delay-amount combinations (Mazur, 1987). The main point is that although two equations may make similar predictions for some situations (e.g., the theoretical curves in Figure 2), they may make very different predictions for other situations, and these differences can have important theoretical and practical implications.

Not just Splitting Hairs

Competing mathematical models do not always make predictions that are so obviously different as those of the hyperbolic and exponential decay equations. Sometimes the predictions of two or more different mathematical models are fairly similar for a wide range of experiments, and the models appear to be roughly equal in their predictive accuracy. In these cases, it seems fair to ask, if we are interested in predicting behavior and if both models predict behavior equally well, what difference does it make which one we use?

My comparison of three models of concurrent-chains performance (Mazur, 2001) illustrates such a situation. In this analysis, I compared the predictive accuracy of Fantino's delay-reduction theory (DRT; Squires & Fantino, 1971), Grace's contextual-choice model (CCM; Grace, 1994), and my hyperbolic value-added model (HVA). To make comparisons among models fair, I provided each model with the same number of free parameters (between two and four free parameters, depending on the data set) to account for such factors as response bias and individual differences in sensitivity to the different reinforcement schedules. I used standard curve-fitting techniques to derive quantitative predictions from these three models for the results from 19 published experiments on concurrent-chains schedules, which included a total of 92 different data sets, most of them from individual subjects. For these data sets, CCM accounted for an average of 90.8% of the variance, HVA for 89.6%, and DRT for 83.0%.

Therefore, when supplied with the same number of free parameters, two models accounted for similar percentages of the variance, and the third was not that far behind. The small differences in the accuracy of the models could easily be due to random variations in the data or to arbitrary decisions about exactly how the free parameters were added to the different models. It seems fair to ask: Why is it important to know which of these models is best? If one is interested in predicting performance on concurrent-chains schedules, it seems that all three models are reasonably successful. (Richard Herrnstein once said to me, “If the matching law accounts for 90% of the variance, that's good enough for me. I'm not greedy.”)

The answer to this question is that although these three models frequently do make very similar predictions, they are based on different assumptions about the psychological processes that produce this behavior. To show this, I will first give an example of a typical concurrent-chains schedule, and I will then present the equations for each of the three models.

Figure 3 diagrams a concurrent-chains schedule. This procedure usually involves two schedules that operate during what are termed the initial links, each of which occasionally leads to its own terminal link. Each terminal link is another reinforcement schedule that leads to food. In this example, during the initial links, the response keys are white, and two identical VI 60-s schedules are in effect. Responses on the left key occasionally cause the key to turn green, and then food is delivered on a fixed-interval (FI) 10-s schedule. Responses on the right key occasionally cause that key to turn red, and then food is delivered on a FI 20-s schedule. After each food delivery, the keys turn white, and the initial links are again in effect. A concurrent-chains procedure therefore alternates between a choice period (the initial links) and a consequences period (the terminal links, during which it is not possible to switch to the other alternative).

A typical concurrent-chains schedule, with equal VI 60-s schedules as initial links, and two different FI schedules as terminal links.

The usual measure of preference is the subject's response proportions in the initial links. Not surprisingly, in this case, a pigeon will make more responses on the left key than on the right key, because the left key leads to the terminal link with the shorter FI schedule. However, it is well known that the schedules in the initial links, and not just those in the terminal links, affect preference. If the durations of two identical initial-link schedules are shortened, the terminal-link schedules have more affect on preference, and response proportions become more extreme (Fantino, 1969). This is called the initial-link effect. For instance, if the initial-link schedules in Figure 3 were decreased to VI 30-s schedules, the percentage of responses on the left key would increase.

To give a brief explanation of how CCM, DRT, and HVA make predictions for concurrent-chains schedules, the equations for these models will be presented in their most basic forms, without the free parameters that were used in the curve-fitting comparisons (Mazur, 2001). This will avoid unnecessary complexity, and it will help to focus on the fundamental differences among the models.

When considering mathematical models of behavior, it is useful to distinguish between descriptive equations and theoretical equations. A descriptive equation merely provides, in mathematical form, a convenient way to summarize the relation between independent variable and dependent variables, without giving any theoretical basis for using this specific equation. In contrast, a theoretical equation is derived from basic principles or assumptions about the psychological processes that give rise to the behaviors in question, and the form of the equation reflects these assumptions. CCM, DRT, and HVA all can be considered theoretical equations because they are based on a few basic assumptions about the psychological processes that govern choice behavior. As will be seen, some of these assumptions are shared by all three models, whereas other assumptions are different for each model.

Without free parameters, Grace's (1994) CCM can be written as follows:

6

B1 and B2 are response rates in the initial links of a concurrent-chains schedule, ri1 and ri2 are the rates of reinforcement in the initial links (i.e., the rates at which each of the two terminal links are entered), and rt1 and rt2 are the rates of reinforcement in the two terminal links (the rates at which the terminal links deliver food). Thus, according to CCM, choice responses in concurrent-chain schedules depend on both the schedules in the initial links and the schedules in the terminal links. The distinguishing feature of CCM is the ratio Tt/Ti. Tt is the average terminal-link duration, and Ti is the average initial-link duration. Because the ratio Tt/Ti is used as an exponent for the terminal-link reinforcement rates, CCM states that differences in the terminal links will have greater effect on preference when they are long relative to the sizes of the initial links, and the terminal links will have less effect on preference when they are relatively short. Therefore, in the example described above, when the initial-link schedules are shortened from VI 60 s to VI 30 s, Tt/Ti increases, and Equation 6 predicts more extreme preference for the green terminal link.

In developing CCM, Grace (1994) began with the basic assumption of Herrnstein's (1961) matching law—that the relative rate of behavior is proportional to the relative rate of reinforcement (Equation 1). The matching law was designed to account for choice on simple concurrent schedules, which do not have terminal links. CCM was developed to extend the framework of the matching law to concurrent-chains schedules. Grace assumed that terminal-link schedules are conditioned reinforcers whose values are a function of their rates of reinforcement (rt1 and rt2). Grace also was guided by the previous work of Baum and Rachlin (1969) who proposed that when reinforcers differ along two or more different dimensions (e.g., rate, delay, amount, quality), these different factors can be combined multiplicatively to obtain a measure of the overall values of the reinforcers. Grace reasoned that, in a similar way, the initial-link reinforcement rates (ri1 and ri2) could be multiplied by the terminal-link reinforcement rates (rt1 and rt2) to obtain the values of the two alternative schedules in concurrent-chains procedures. In addition, based on both theoretical considerations and research findings, Grace was convinced that the behavioral expression of the terminal-link values is dependent on the context in which they are presented (that is, on the durations of the terminal links compared to those of the initial links). Following Baum's (1974) work on the generalized matching law, which uses an exponent to reflect an animal's sensitivity to differences in reinforcement rates, Grace used the exponent Tt/Ti to express the fact that sensitivity to the reinforcement rates in the terminal links depends on the relative durations of the initial and terminal links. The end result of this set of assumptions was CCM. Notice that Equation 6 reduces to the simple matching law if Tt = 0, that is, when there is no terminal link.

DRT also uses the principle of matching as a basic assumption, and it, too, reduces to Herrnstein's (1961) matching law if there are no terminal links. The Squires and Fantino (1971) version of DRT can be written as follows:

7

As already explained, R1 and R2 are the rates of reinforcement, but in a concurrent-chains schedule they are the overall rates, including time in both the initial and terminal links. Ttotal is the mean total time to primary reinforcement from the start of the initial links. Tt1 and Tt2 are the mean times to primary reinforcement from the start of the two terminal links (i.e., the average durations of the two terminal links). The fundamental assumption of DRT is that the conditioned reinforcing value of a terminal link is determined by the amount of delay reduction that occurs when each terminal link is entered, compared to the average time to food from the start of the initial links. For the example in Figure 3, Ttotal is 45 s (because the average initial-link duration is 30 s and the average terminal-link duration is 15 s), Tt1 is 10 s, and Tt2 is 20. Therefore, the onset of the green key represents a 35-s delay reduction, but the onset of the red key represents only a 25-s delay reduction, which is why DRT predicts a preference for the green alternative.

Besides its assumption about the crucial role of delay reduction, DRT differs from CCM in its assumption that choice behavior is also a function of overall rates of reinforcement (R1 and R2), compared to CCM's assumption that it is a function of the initial-link reinforcement rates (ri1 and ri2). Squires and Fantino (1971) did not explain why they chose this approach, but the implication of Equation 7 is that the overall rates of primary reinforcement combine multiplicatively with the conditioned reinforcing values of the terminal links to determine choice behavior. This difference between DRT's use of overall reinforcement rates and CCM's use of initial-link reinforcement rates may seem subtle, but for certain choice situations it leads to very different predictions that can be tested with suitable research designs.

HVA was built around three fundamental assumptions. First, like CCM and DRT, HVA adopts the principle of matching as a basic assumption, and it, too, reduces to the matching law if there are no terminal links. Second, HVA is built on the assumption that reinforcer value declines with increasing delay according to a hyperbolic function (as in Equation 5). With its free parameters removed, HVA becomes:

8

The two leftmost expressions in the equation are identical to CCM. The rightmost parenthetical expression includes Vt1 and Vt2, the values of the two terminal links, and Vi, the value of the initial links, and all of these values are calculated using a variation of the hyperbolic function (Equation 5) that applies to cases where delays to reinforcement are variable (Mazur, 1984). In Equation 8, the value of the initial links is subtracted from the value of each terminal link, because the third assumption of HVA is that choice depends on the increase in conditioned reinforcement value (i.e., the value added) that occurs when the initial link ends and a terminal link begins. For the example in Figure 3, Vi, the value of the initial links, will be low because they are associated with a long delay to food. Vt2, the value of the red terminal link, is a bit higher because the red key signals a somewhat shorter (20 s) delay to food. Vt1, the value of the green terminal link, is the highest because the green key signals just a 10-s delay to food. Therefore, there is more value added when the left terminal link begins, and so HVA predicts a preference for the green alternative. When there are no terminal-link delays, as in simple concurrent VI schedules, and the amounts of reinforcement, A, are the same for both alternatives, Vt1 and Vt2 both equal A. In this case, the rightmost parenthetical expression in Equation 8 becomes

CCM, DRT, and HVA all predict preference for the shorter terminal link and the initial-link effect, as well as other results from concurrent-chains schedules, and they all reduce to the basic matching law when there are no terminal links. This should come as no surprise, because mathematical models are constrained by empirical results, and no model that did not account for these well-established behavioral phenomena would be taken seriously. However, as the preceding discussion showed, these three models are based on different assumptions about the nature of choice behavior (assumptions that can be comprehended by someone who has no interest in the equations). For CCM, the key factor is the context in which a choice is made (or, more specifically, the duration of the choice period). If the choice period is long relative to the duration of the terminal links, then differences between the terminal-link schedules will have relatively little effect on preference. The key factor for DRT is delay reduction: preference depends on the decrease in time to reinforcement that is signaled by the onset of a terminal link. For HVA, the key factor is value (the value of the conditioned reinforcer associated with each schedule), and preference depends on the increase in value that is signaled by the onset of a terminal link.

If there were no way to distinguish among these different assumptions, it would be pointless to argue about them. However, when the assumptions are translated into mathematical notation, the differences among them are easier to discern, and it becomes possible to find cases for which the models make distinctly different predictions. For example, Savastano and Fantino (1996) showed that if the two terminal links are lengthened by identical amounts, CCM and DRT predict little or no change in preference, whereas HVA predicts less extreme preference with the longer terminal links. Mazur (2000b) showed that if a third alternative is added to the typical two-alternative concurrent-chains schedule, then there are cases in which both DRT and HVA predict an increase in preference for whichever of the two original alternatives was previously favored, whereas CCM predicts no change or a decrease in preference. Nevin and Grace (2000) noted that as initial links become very long, both CCM and DRT predict that choice responses should approach indifference (equal responding for the two alternatives), whereas HVA predicts continued preference for the shorter terminal link (approaching an asymptote of Vt1/Vt2).

I will not evaluate the experiments designed to test these predictions because it is not the purpose of this article to advocate for any specific mathematical model, and in any case, more than two or three experiments will be needed to determine conclusively which model's predictions are supported and which are not. The point is that the mathematical form of these models leads to unambiguous predictions that can be rigorously tested and contrasted so that their different hypotheses about the determinants of choice behavior can be put to an empirical test.

Beyond the Operant Laboratory

Once solid evidence for a mathematical principle is obtained in laboratory research, it often is possible to use this principle to predict or control behavior in applied settings. Herrnstein's matching law (Equation 1) is a good example of a mathematical principle that has been used by psychologists in many real-world settings. A few examples will help to give some indication of how broadly this principle of choice has been applied.

The matching law can be viewed as a principle of reinforcement relativity. It states that we cannot predict how much behavior a reinforcer will generate without knowing what reinforcers are concurrently available for other activities. For example, if R1 is held constant at some specific value (e.g., 40 reinforcers per hour), B1 (the behavior that produces this reinforcer) will be greater if R2 (the rate of reinforcement for alternative behaviors) is low than if it is high. Bulow and Meller (1998) used this principle to predict levels of sexual activity and contraceptive use among teenage girls. They reasoned that, according to the matching law, high-risk sexual behavior would be more common among teenage girls for whom alternative sources of reinforcement were relatively scarce. They used the Adolescent Reinforcement Survey Schedule (Holmes, Heckel, Chestnut, Harris, & Cautela, 1987) to determine what activities each girl found particularly enjoyable, and how frequently she was able to engage in these activities. This provided a measure of the rate of reinforcement that was available for nonsexual behaviors for each girl. Bulow and Meller found that girls with fewer alternative sources of reinforcement tended to engage in more high-risk sexual activity and that the matching law did a good job of predicting the levels of sexual activity for individual girls within different age, ethnic, and socioeconomic groups. Applying similar reasoning, Correia, Simons, Carey, and Borsari (1998) used the matching law to predict drug use among college students by measuring the students' levels of reinforcement for drug-free activities.

The matching law also has been applied to students' behaviors in educational settings. We can think of students in a classroom as having a choice between doing their academic work and engaging in other, nonacademic behaviors. Martens and Houk (1989) monitored the behaviors of a girl with mental retardation in a classroom setting, along with the behaviors of the teacher and a teacher's aide. They recorded the girl's on-task and disruptive behaviors, as well as the reinforcers (e.g., instructions, praise, or other forms of attention) delivered by the teacher and the aide. They found that the matching law provided a good description of the relation between the rates of on-task and disruptive behaviors and the rates of reinforcement the girl received for these two classes of behaviors. Going a step further, Martens, Lochner, and Kelly (1992) used different VI schedules to increase appropriate academic behaviors in third and fourth graders who had been spending excessive amounts of time on off-task behaviors, and they found that the rates of on-task behavior depended on the rates of reinforcement, as the matching law predicted. A number of other studies have applied the matching law to classroom behavior (e.g., Billington & DiTommaso, 2003; Mace, McCurdy, & Quigley, 1990; Skinner, Robinson, Johns, Logan, & Belfiore, 1996).

Mathematical models of behavior that are developed through operant research also can be used by researchers in other areas of science, such as neuroscience and psychopharmacology. As one example, Kheramin et al. (2002) used the hyperbolic-decay equation for delayed reinforcers (Equation 5) to assess the effects of brain lesions on rats' self-control choices (choices between a smaller, more immediate reinforcer and a larger, more delayed reinforcer). There was evidence from humans that damage to the orbital region of the prefrontal cortex (OPFC) leads to an increased tendency to make impulsive choices (i.e., to choose the smaller, more immediate reinforcer). How would lesions in the OPFC affect self-control choices in rats? To answer this question, Kheramin et al. gave rats with OPFC lesions and control rats a series of choices between a small amount and a larger amount of a sucrose solution, each delivered after a delay. The delays for the small and large reinforcers were varied over trials so that the researchers could estimate a set of indifference points—pairs of delay-amount combinations that the rats chose about equally often. For example, from one series of choices the experimenters might find that the small reinforcer delivered after a 1-s delay was about equally preferred to the large reinforcer delivered after a 4-s delay. From another series of choices they might find that the small reinforcer delivered after a 5-s delay was about equally preferred to the larger reinforcer delivered after a 10-s delay. Their study was designed to obtain five such indifference points, both from the rats with OPFC lesions and from the control rats.

Based on previous research with similar procedures, Kheramin et al. (2002) expected that if the delays for the small reinforcers are plotted on the x axis and the delays for the equally preferred large reinforcers are plotted on the y axis, the result should be a linear function, such as the hypothetical examples illustrated in Figure 4. Mazur (1987) showed that the hyperbolic-decay model (Equation 5) predicts that this indifference function will have a slope greater than one and a y intercept greater than zero. But exactly how would the indifference functions from the rats with OFPC lesions differ from those of the control rats? Based on predictions derived from Equation 5, Kheramin and his colleagues considered three possibilities, as shown in the three panels of Figure 4. (The derivations for all of these predictions are shown in the Appendix.) Suppose that K, the discounting rate parameter in Equation 5, is greater for the rats with OFPC lesions, which means that for these rats, a reinforcer's value declines more rapidly as its delay increases. It follows from Equation 5 that an increase in K will lead to a decrease in the y intercept of the indifference function but no change in slope, as in the left panel of Figure 4. A second possibility is that OPFC lesions might alter the rats' sensitivity to the differences in the sizes of the two reinforcers, as represented by the parameter A in Equation 5. For example, the center panel in Figure 4 shows the predictions of Equation 5 if OPFC lesions increase a rat's sensitivity to differences in reinforcer amounts—the indifference functions should have steeper slopes and larger y intercepts. A third possibility is that OPFC lesions might alter both a rat's sensitivity to delay and its sensitivity to reinforcer amount. For instance, the right panel in Figure 4 shows the predictions of Equation 5 for the case where sensitivity to delay and sensitivity to amount both increase as a result of OPFC lesions. In this example, the slope is steeper for rats with OPFC lesions, but the y intercept is essentially unchanged because the effects of an increase in K (which should decrease the y intercept) and of an increase in sensitivity to amount (which should increase the y intercept) tend to cancel each other out.

Hypothetical indifference functions illustrating three possible effects of lesions to the orbital prefrontal cortex in rats.

These are different predictions derived from the hyperbolic-decay model (Equation 5), depending on whether the lesions produce an increase in K (left panel), an increase in sensitivity to reinforcer amount (center panel), or both (right panel).

In their experiment, Kheramin et al. (2002) obtained indifference functions very similar to those in the right panel of Figure 4. Consider how difficult it would be to interpret this pattern of results without a suitable mathematical analysis. The differences between OPFC-lesioned rats and control rats seem to be inconsistent: The OPFC-lesioned rats were similar to control rats with short delays, but they were less impulsive than control rats with longer delays. Why should this happen? The mathematical analysis provided by Equation 5 offers an explanation of this pattern of results: It suggests that the OPFC lesions increased both sensitivity to delay and sensitivity to amount of reinforcement, and this is what Kheramin et al. concluded. This mathematical analysis also suggests specific ways that this hypothesis could be tested in future research. For example, if reinforcer amounts are kept equal, the model predicts that rats with OPFC lesions should show a greater tendency to avoid long delays than control rats (a prediction that may seem counterintuitive in light of the Khermin et al. data, where the OPFC-lesioned rats generally accepted longer delays than the control rats). Conversely, if the alternatives differ in amount but not delay, then rats with OPFC lesions should show a stronger tendency to choose the larger amount than control rats.

Mathematical models sometimes can give researchers guidance about what patterns to look for at the cellular level. When Platt and Glimcher (1999) studied the neural control of monkeys' saccadic eye movements toward visual targets, they were prompted by research on the matching law (Equation 1) to examine how neural activity in a specific area of the posterior parietal cortex might be affected by the probability that a reinforcer (fruit juice) would be delivered following an eye movement. They found that the activity levels of individual neurons in this brain area were directly proportional to the probability that an eye movement toward the target would be followed by reinforcement. In another experiment, they found that the activity levels of these neurons also were proportional to the size of the reinforcer (the amount of juice that would later be delivered, after a response was made). Platt and Glimcher concluded that individual neurons in the parietal cortex encoded the potential values of the reinforcers that might follow different possible eye movements, and that this information ultimately affected the monkeys' choices (because at a behavioral level, the monkeys' choices of different possible eye movements were consistent with the matching law). They stated, “In our free-choice experiment, both monkeys and posterior parietal neurons behaved as if they had knowledge of the gains associated with different actions” (p. 238). It remains to be seen whether additional research will continue to support this interpretation, but this is a tantalizing finding because it suggests a direct parallel between reinforcer value as measured in behavioral research and the activity of individual neurons. This is a clear example of a study on brain mechanisms that was stimulated by a mathematical model derived from basic behavioral research.

Unifying Diverse Phenomena

At their best, mathematical models can provide a common framework for describing diverse behavioral phenomena. Over a period of many years, Peter Killeen (1975, 1994; Killeen, Hall, Reilly, & Kettle, 2002) has developed and extensively tested a set of mathematical principles of reinforcement (MPR), which are designed to describe the relation between reinforcement and operant behavior. At the heart of MPR are three concepts: activation, temporal constraint, and coupling, each of which is represented by a parameter in the model. The activation parameter represents the number of seconds of responding activated by each incentive (e.g., each food delivery), the temporal constraint parameter is the amount of time needed to complete one response, and the coupling parameter is based on the animal's memory for recent events. Loosely speaking, the coupling parameter measures the strength of association between responses and reinforcers.

Beginning with these basic concepts, Killeen (1994) used a series of equations to derive predictions about behavior under a variety of different reinforcement schedules, and he compared these predictions to the data from many different experiments. For instance, he showed that for ratio schedules, MPR correctly predicts that response rates should first increase to a maximum and then decrease as ratio size is increased. For VI schedules, it correctly predicts a curvilinear increase in response rates with increasing rates of reinforcement (see Figure 5, top right panel). For FI schedules, it correctly predicts an increase and then decrease in response rates with increasing rates of reinforcement.

The application of Killeen's mathematical principles of reinforcement (MPR) to a variety of different behavioral phenomena.

The theoretical curves in each panel show the best-fitting predictions of MPR. Left panels: Pigeons' overall response rates, postreinforcement pauses (PRP), and running response rates on FR schedules of different sizes (from Bizo & Killeen, 1997, along with some data from a previous study by Felton & Lyon, 1966). Top right panel: Pigeons' response rates on VI schedules with different rates of reinforcement under different levels of food deprivation (from Killeen, 1994, using data from Heyman & Monaghan, 1987). Bottom right panel: Pigeons' activity levels in the 60-s intervals between food presentations after being given different drugs (from Killeen, 1975).

The ability of a single model to predict response rates successfully on these different reinforcement schedules is no small achievement because the different schedules specify very different feedback functions between behavior and reinforcement. However, MPR can do much more than this. By sensible manipulation of the model's parameters, it can account for the effects of a variety of different experimental manipulations, such as variations in the amount of force required to make a response (Heyman & Monaghan, 1987), response topography (McSweeney, 1978), level of deprivation (Snyderman, 1983), and hypothalamic lesions (Kelsey & Allison, 1976). In addition, Bizo and Killeen (1997) showed that MPR can be used to predict, not only overall response rates, but the fine structure of responding on ratio and interval schedules. For example, they showed that the model can predict the durations of postreinforcement pauses and running response rates (response rates excluding the postreinforcement pause) on FR schedules of different sizes (see the left panels of Figure 5).

MPR also has been applied to a wide range of other results. In his early work on the activation parameter, Killeen (1975) demonstrated how his equations predicted beautifully the increases and decreases in the general activity levels of different species between individual food presentations. For example, the bottom right panel of Figure 5 shows activity levels of pigeons under different drugs when they received food deliveries every 60 s. The pigeons' activity levels (measured by movement sensors in the floor of the test chamber) first increased and then decreased as the 60-s interval progressed, and Killeen's model described the activity patterns very well. Besides predicting steady-state performance on different reinforcement schedules, MPR has been applied to the dynamic features of operant responding. That is, it has been used to predict operant responding during periods of acquisition, as when an animal is learning a new response or when it is adapting to a new reinforcement schedule (e.g., Killeen, 1994; Killeen & Bizo, 1998).

Killeen's MPR provides a good example of a mathematical model that can account for a wide range of results, including both the fine structure of responding on reinforcement schedules and overall response rates, acquisition and steady-state performance, the effects of reinforcers on general activity, and so on. Without such a model, we might have the intuition that these different aspects of operant behavior are related: It seems reasonable to suppose that principles that can predict the fine structure of operant responding should be able to yield predictions about overall response rates, and that the same variables that affect behavior during acquisition should continue to operate once behavior has stabilized. However, in the absence of a model that makes quantitative predictions, assertions such as “the same principles of reinforcement govern both acquisition and long-term performance” are little more than hand waving. It is only when a theory makes specific, quantitative predictions, and when these predictions are tested against actual data, that we can have any real confidence in such a general statement. As the examples throughout this paper have shown, one of the greatest benefits of using mathematical models is that they generate precise and testable predictions.

Conclusions

The purpose of this article has been to demonstrate some of the advantages of mathematical models in the experimental analysis of behavior. Mathematical models offer a number of benefits to psychologists:

1. Translating a verbal hypothesis into a mathematical model forces a theorist to be precise and unambiguous, and this can point to ways of testing competing theories that sound as if they make similar predictions when they are stated in words (as in the comparison of the negative law of effect and the avoidance theory of punishment).

2. Even when the shapes of two mathematical functions are quite similar (such as the hyperbolic and exponential delay-of-reinforcement functions in Figure 2), these functions may make distinctly different predictions about behavior with profound theoretical and applied implications. In this example, the hyperbolic function predicts that people will often exhibit preference reversals in self-control choice situations as time passes, whereas the exponential equation predicts no preference reversals (unless one makes additional assumptions about changing parameter values, assumptions that can then be tested to see whether they actually apply).

3. In some cases, competing mathematical models may account for large data sets about equally well. But unless they are actually mathematically equivalent, different mathematical models are based on different assumptions about the psychological processes underlying an observed behavior. For example, CCM, DRT, and HVA can each account for a wide range of results from concurrent-chains schedules, but they assume different principles of choice behavior (dependence on context, vs. delay-reduction, vs. value-addition). If we want to understand the determinants of choice behavior, it is important to know which of these three assumptions is most useful.

4. Mathematical models of behavior that are developed through basic behavioral research can be used to predict or control behavior in applied settings. The models have been used in neuroscience and psychopharmacology to help researchers identify the functions of different brain structures and to assess the behavioral effects of different drugs.

5. A mathematical model can provide a common framework that unites diverse behavioral phenomena. Killeen's (1994) MPR was presented as an example of a mathematical model that has been applied to a wide range of behavioral phenomena, but it is by no means the only one. Other examples include the Rescorla-Wagner model of classical conditioning (Rescorla & Wagner, 1972), Gibbon's (1977) scalar expectancy theory of timing, and Nevin's (1992) theory of behavioral momentum, to name just a few.

Although some of these models are relatively straightforward (e.g., those representing the negative law of effect and the avoidance theory of punishment), for others the equations are quite complex, as are the derivations that allow them to be applied to specific examples, and not everyone is able to follow them. This is one of the drawbacks of mathematical modeling, but it is a cost that is more than offset by the advantages. The mathematical precision of these theories allows them to be tested rigorously, and in testing these theories their strengths can be demonstrated and their weaknesses can be exposed. For instance, the Rescorla-Wagner model is a landmark in the field of classical conditioning, and it has stimulated a great deal of research. However, the Rescorla-Wagner model does have some well-documented limitations, and these have prompted the development of alternative models (e.g., Hall & Pearce, 1983; Mackintosh, 1975; Miller & Schachtman, 1985; Wagner, 1981). Because of the empirical and theoretical work that was stimulated by the Rescorla-Wagner model, we now have a much better understanding of the richness and complexity of classical conditioning than we did before this model was introduced.

In a commentary about some competing mathematical models of timing, Killeen (1999) wrote: “If you think models are about the truth, or that there is a best timing model, then you are in trouble. There is no best model, any more than there is a best car model or a best swimsuit model, even though each of us may have our favorites. It all depends on what you want to do with the model” (p. 275). Those who do not enjoy studying mathematical models might take this statement (from a preeminent mathematical modeler) as an excuse to avoid them. Why bother putting in the time and effort to understand current mathematical models of behavior when there is no best model, and when they all have their weaknesses and limitations? Killeen addresses this issue by asserting that “all understanding involves models—reference to systems that exist in a different domain than the thing studied. Loose models make vague reference to ambiguous and ad hoc causes. Tighter models are more careful about definitions and avoid gratuitous entities. Models of phenomena are not causes of phenomena; they are descriptions of hypothetical structures or functions that aid explanation, prediction, and control” (p. 276).

In this article, I have argued that mathematical models are often “tighter models” than verbal descriptions of behavioral principles, and that they are essential to continued progress in the experimental analysis of behavior. Mathematical models can make precise and important statements about behavioral processes that are relevant to anyone who is interested in explaining, predicting, or controlling behavior, either in the laboratory or in applied settings. Communication between those students of behavior who routinely use mathematical models and those who do not should be a two-way street. Those who do not enjoy mathematics can nevertheless benefit from the theoretical advances that result from the development and testing of mathematical models of behavior. And those who specialize in mathematical models can make an effort to find nonmathematical ways (such as verbal descriptions, analogies, diagrams, or concrete examples) to communicate the essential ideas that are contained in the mathematical notation to as wide an audience as possible.

These alternative ways of expressing mathematical concepts often can be very helpful. For example, science writers have found many ingenious ways to explain Einstein's theory of relativity to general readers by using thought experiments, diagrams, animations, analogies, and other devices. As a result, although I have never studied the mathematical equations of general relativity, I feel that I have a basic understanding of the theory and what it has to say about time, space, matter, energy, gravity, the speed of light, and so on. If it is possible to explain, to a general audience, concepts so far removed from everyday experience as travel near the speed of light or the conversion of matter into energy, then it certainly should be possible to explain a mathematical model of behavior to someone who is not comfortable with mathematics. Even though a nonmathematical translation may lose some of the precision of the mathematical model, it can still convey the essential point (e.g., from the matching law, the idea that the effects of reinforcement are relative, or from scalar expectancy theory, how accuracy in temporal discrimination depends on stimulus duration). If the value of a mathematical model depends on how it is used, then those who develop mathematical models in psychology should be eager to promote their use, not just by other specialists, but by anyone interested in the explanation, prediction, and control of behavior.

Acknowledgments

Preparation of this manuscript was supported by Grant MH 38357 from the National Institute of Mental Health.

Appendix

The hyperbolic model (Equation 5) makes specific predictions about how the shapes of indifference functions should change as a result of changes in K or in sensitivity to reinforcer amounts. Suppose that an animal is given a series of choices between a smaller, more immediate reinforcer and a larger, more delayed reinforcer. We can begin by assuming that at an indifference point (where the animal chooses the two alternatives equally often), VS = VL, where S refers to the smaller reinforcer and L to the larger reinforcer. It follows from Equation 5 that AL/(1 + KDL) = AS/(1 + KDS).

To obtain the model's predictions shown in Figure 4, in which DL (on the y axis) is plotted as a function of DS (on the x axis), we can solve this equation for DL, which yields DL = (AL/AS)DS + (AL/AS −1)/K. This is a linear equation with a slope of AL/AS and a y intercept of (AL/AS −1)/K.

Three different possible consequences of some procedural manipulation (e.g., brain lesions) are considered in Figure 4:

1. If the manipulation causes an increase in K (signifying more rapid decreases in reinforcer value as delay increases), this will cause a decrease in the y intercept but no change in slope, as shown in the left panel of Figure 4.

2. If the manipulation causes an increase in sensitivity to the differences in reinforcer amounts, this will lead to an increase in the ratio AL/AS (that is, the animal is now more sensitive to the differences in the sizes of the two reinforcers). Because the ratio appears in the expressions for both the slope and the y intercept, the model predicts that the result will be an increase in both the slope and the y intercept, as shown in the center panel of Figure 4.

3. If the manipulation causes increases in both K and in sensitivity to amount, then the equation predicts an increase in slope (for the same reason as in the previous case). However, it also predicts that the effects of the changes in K and in sensitivity to amount will tend to have offsetting effects on the y intercept because the increase in K should produce a decrease in the y intercept but the increase in AL/AS should produce an increase in the y intercept. This possibility is depicted in the right panel of Figure.

4. Because the indifference functions obtained by Kheramin et al. (2002) were very similar to those in the right panel, these researchers concluded that OPFC lesions cause increases in both K and in sensitivity to reinforcer amount.

References

• Ainslie G.W. Specious reward: A behavioral theory of impulsiveness and impulse control. Psychological Bulletin. 1975;82:463–496. [PubMed] [Google Scholar]
• Baum W.M. On two types of deviation from the matching law: Bias and undermatching. Journal of the Experimental Analysis of Behavior. 1974;22:231–242. [PMC free article] [PubMed] [Google Scholar]
• Baum W.M, Rachlin H. Choice as time allocation. Journal of the Experimental Analysis of Behavior. 1969;12:861–874. [PMC free article] [PubMed] [Google Scholar]
• Billington E.J, DiTommaso N.M. Demonstrations and applications of the matching law in education. Journal of Behavioral Education. 2003;12:91–104. [Google Scholar]
• Bizo L.A, Killeen P.R. Models of ratio schedule performance. Journal of Experimental Psychology: Animal Behavior Processes. 1997;23:351–367. [PMC free article] [PubMed] [Google Scholar]
• Bulow P.J, Meller P.J. Predicting teenage girls' sexual activity and contraception use: An application of the matching law. Journal of Community Psychology. 1998;26:581–596. [Google Scholar]
• Correia C.J, Simons J, Carey K.B, Borsari B.E. Predicting drug use: Application of behavioral theories of choice. Addictive Behaviors. 1998;23:705–710. [PubMed] [Google Scholar]
• Deluty M.Z. Choice and rate of punishment in concurrent schedules. Journal of the Experimental Analysis of Behavior. 1976;25:75–80. [PMC free article] [PubMed] [Google Scholar]
• de Villiers P.A. Choice in concurrent schedules and a quantitative formulation of the law of effect. In: Honig W.K, Staddon J.E.R, editors. Handbook of operant behavior. Englewood Cliffs, NJ: Prentice-Hall; 1977. pp. 233–287. [Google Scholar]
• de Villiers P.A. Toward a quantitative theory of punishment. Journal of the Experimental Analysis of Behavior. 1980;33:15–25. [PMC free article] [PubMed] [Google Scholar]
• Dinsmoor J.A. Punishment: I. The avoidance hypothesis. Psychological Review. 1954;61:34–46. [PubMed] [Google Scholar]
• Dinsmoor J.A. Escape, avoidance, punishment: Where do we stand? Journal of the Experimental Analysis of Behavior. 1977;28:83–95. [PMC free article] [PubMed] [Google Scholar]
• Fantino E. Choice and rate of reinforcement. Journal of the Experimental Analysis of Behavior. 1969;12:723–730. [PMC free article] [PubMed] [Google Scholar]
• Felton M, Lyon D.O. The post-reinforcement pause. Journal of the Experimental Analysis of Behavior. 1966;9:131–134. [PMC free article] [PubMed] [Google Scholar]
• Ferris T. Coming of age in the Milky Way. New York: Anchor Books; 1988. [Google Scholar]
• Gibbon J. Scalar expectancy theory and Weber's law in animal timing. Psychological Review. 1977;84:279–325. [Google Scholar]
• Grace R. A contextual model of concurrent-chains choice. Journal of the Experimental Analysis of Behavior. 1994;61:113–129. [PMC free article] [PubMed] [Google Scholar]
• Grace R.C. The matching law and amount-dependent exponential discounting as accounts of self-control choice. Journal of the Experimental Analysis of Behavior. 1999;71:27–44. [PMC free article] [PubMed] [Google Scholar]
• Green L, Fisher E.B, Perlow S, Sherman L. Preference reversal and self control: Choice as a function of reward amount and delay. Behaviour Analysis Letters. 1981;1:43–51. [Google Scholar]
• Green L, Fristoe N, Myerson J. Temporal discounting and preference reversals in choice between delayed outcomes. Psychonomic Bulletin and Review. 1994;1:383–389. [PubMed] [Google Scholar]
• Green L, Myerson J. Alternative frameworks for the analysis of self control. Behavior and Philosophy. 1993;21:37–47. [Google Scholar]
• Green L, Myerson J, Holt D.D, Slevin J.R, Estle S.J. Discounting of delayed food rewards in pigeons and rats: Is there a magnitude effect? Journal of the Experimental Analysis of Behavior. 2004;81:39–50. [PMC free article] [PubMed] [Google Scholar]
• Green L, Myerson J, McFadden E. Rate of temporal discounting decreases with amount of reward. Memory & Cognition. 1997;25:715–723. [PubMed] [Google Scholar]
• Hall G, Pearce J.M. Changes in stimulus associability during acquisition: Implications for theories of acquisition. In: Commons M.L, Herrnstein R.J, Wagner A.R, editors. Quantitative analyses of behavior: Vol. 3: Acquisition. Cambridge, MA: Ballinger; 1983. pp. 221–239. [Google Scholar]
• Herrnstein R.J. Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior. 1961;4:267–272. [PMC free article] [PubMed] [Google Scholar]
• Heyman G.M, Monaghan M.M. Effects of changes in response requirement and deprivation on the parameters of the matching law equation: New data and review. Journal of Experimental Psychology: Animal Behavior Processes. 1987;13:384–394. [Google Scholar]
• Holmes G.R, Heckel R.V, Chestnut E, Harris N, Cautela J. Factor analysis of the Adolescent Reinforcement Survey Schedule (ARSS) with college freshmen. Journal of Clinical Psychology. 1987;43:386–390. [PubMed] [Google Scholar]
• Hull C.L. Principles of behavior. New York: Appleton-Century-Crofts; 1943. [Google Scholar]
• Kelsey J.E, Allison J. Fixed-ratio lever pressing by VMH rats: Work vs. accessibility of sucrose reward. Physiology & Behavior. 1976;17:749–754. [PubMed] [Google Scholar]
• Kheramin S, Body S, Mobini S, Ho M.-Y, Velazquez-Martinez D.N, Bradshaw C.M, Szabadi E, Deakin J.F.W, Anderson I.M. Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: A quantitative analysis. Psychopharmacology. 2002;165:9–17. [PubMed] [Google Scholar]
• Killeen P.R. On the temporal control of behavior. Psychological Review. 1975;82:89–115. [Google Scholar]
• Killeen P.R. Mathematical principles of reinforcement. Behavioral and Brain Sciences. 1994;17:105–172. [Google Scholar]
• Killeen P.R. Modeling modeling. Journal of the Experimental Analysis of Behavior. 1999;71:275–280. [PMC free article] [PubMed] [Google Scholar]
• Killeen P.R, Bizo L.A. The mechanics of reinforcement. Psychonomic Bulletin and Review. 1998;5:221–238. [Google Scholar]
• Killeen P.R, Hall S.S, Reilly M.P, Kettle L.C. Molecular analyses of the principal components of response strength. Journal of the Experimental Analysis of Behavior. 2002;78:127–160. [PMC free article] [PubMed] [Google Scholar]
• Mace F.C, McCurdy B, Quigley E.A. A collateral effect of reward predicted by matching theory. Journal of Applied Behavior Analysis. 1990;23:197–205. [PMC free article] [PubMed] [Google Scholar]
• Mackintosh N.J. A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review. 1975;82:276–298. [Google Scholar]
• Martens B.K, Houk J.L. The application of Herrnstein's law of effect to disruptive and on-task behavior of a retarded adolescent girl. Journal of the Experimental Analysis of Behavior. 1989;51:17–27. [PMC free article] [PubMed] [Google Scholar]
• Martens B.K, Lochner D.G, Kelly S.Q. The effects of variable-interval reinforcement on academic engagement: A demonstration of matching theory. Journal of Applied Behavior Analysis. 1992;25:143–151. [PMC free article] [PubMed] [Google Scholar]
• Mazur J.E. Tests of an equivalence rule for fixed and variable reinforcer delays. Journal of Experimental Psychology: Animal Behavior Processes. 1984;10:426–436. [Google Scholar]
• Mazur J.E. An adjusting procedure for studying delayed reinforcement. In: Commons M.L, Mazur J.E, Nevin J.A, Rachlin H, editors. Quantitative analyses of behavior: Vol. 5. The effect of delay and of intervening events on reinforcement value. Hillsdale, NJ: Erlbaum; 1987. pp. 55–73. [Google Scholar]
• Mazur J.E. Tradeoffs among delay, rate and amount of reinforcement. Behavioural Processes. 2000a;49:1–10. [PubMed] [Google Scholar]
• Mazur J.E. Two- versus three-alternative concurrent-chain schedules: A test of three models. Journal of Experimental Psychology: Animal Behavior Processes. 2000b;26:286–293. [PubMed] [Google Scholar]
• Mazur J.E. Hyperbolic value addition and general models of animal choice. Psychological Review. 2001;108:96–112. [PubMed] [Google Scholar]
• McSweeney F.K. Prediction of concurrent keypeck treadle-press responding from simple schedule performance. Animal Learning & Behavior. 1978;6:444–450. [Google Scholar]
• Miller R.R, Schachtman T.R. The several roles of context at the time of retrieval. In: Balsam P.D, Tomie A, editors. Context and learning. Hillsdale, NJ: Erlbaum; 1985. [Google Scholar]
• Nevin J.A. An integrative model for the study of behavioral momentum. Journal of the Experimental Analysis of Behavior. 1992;57:301–316. [PMC free article] [PubMed] [Google Scholar]
• Nevin J.A, Grace R.A. Behavioral momentum and the law of effect. Behavioral and Brain Sciences. 2000;23:73–130. [PubMed] [Google Scholar]
• Ong E.L, White K.G. Amount-dependent temporal discounting? Behavioural Processes. 2004;66:201–212. [PubMed] [Google Scholar]
• Platt M.L, Glimcher P.W. Neural correlates of decision variables in the parietal cortex. Nature. 1999;400:233–238. [PubMed] [Google Scholar]
• Rachlin H, Herrnstein R.J. Hedonism revisited: On the negative law of effect. In: Campbell B.A, Church R.M, editors. Punishment and aversive behavior. New York: Appleton-Century-Crofts; 1969. [Google Scholar]
• Rescorla R.A, Wagner A.R. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Black A.H, Prokasy W.F, editors. Classical conditioning II: Current research and theory. New York: Appleton-Century-Crofts; 1972. pp. 64–99. [Google Scholar]
• Richards J.B, Mitchell S.H, de Wit H, Seiden L.S. Determination of discount functions in rats with an adjusting-amount procedure. Journal of the Experimental Analysis of Behavior. 1997;67:353–366. [PMC free article] [PubMed] [Google Scholar]
• Savastano H.I, Fantino E. Differences in delay, not ratios, control choice in concurrent chains. Journal of the Experimental Analysis of Behavior. 1996;66:97–116. [PMC free article] [PubMed] [Google Scholar]
• Skinner C.H, Robinson S.L, Johns G.A, Logan P, Belfiore P.J. Applying Herrnstein's matching law to influence students' choice to complete difficult academic tasks. Journal of Experimental Education. 1996;65:5–17. [Google Scholar]
• Snyderman M. Body weight and response strength. Behaviour Analysis Letters. 1983;3:255–265. [Google Scholar]
• Squires N, Fantino E. A model for choice in simple concurrent and concurrent-chains schedules. Journal of the Experimental Analysis of Behavior. 1971;15:27–38. [PMC free article] [PubMed] [Google Scholar]
• Wagner A.R. SOP: A model of automatic memory processing in animal behavior. In: Spear N.E, Miller R.R, editors. Information processing in animals: Memory mechanisms. Hillsdale, NJ: Erlbaum; 1981. [Google Scholar]

Articles from Journal of the Experimental Analysis of Behavior are provided here courtesy of Society for the Experimental Analysis of Behavior