EQUINE CLICKER TRAINING.....
using precision and positive reinforcement to teach horses and people
Clicker Expo 2013: Stamford, Connecticut
After missing Clicker Expo 2012, I was very excited that Clicker Expo 2013 was in Stamford, Connecticut, a measly 3+ hours from my home. We drove up on Thursday in light snow and Friday morning the hotel had a few inches of snow on the ground. I think it's the first time I went to Clicker Expo in the snow so this was a new experience and we worried about there being issues because of the weather. But it seems like everyone made it (eventually) and the Clicker Expo staff did a great job dealing with last minute changes due to the weather. This year's Expo had the most horse people I have seen yet and it was great to catch up with old friends and make new ones. I am excited that more horse people are going and hope this trend will continue.
This year I wrote separate reports for each lecture or lab I attended. There was so much information and some of the topics are getting quite complex so I thought this was the easiest way to do it. You can read the individual reports as well as my summary by going directly to the one of interest or just starting at the top and reading down.
Ken Ramirez: Missteps, Myths and Mantras
Myths and Mantras (Sayings): There is often a factual component to these, but they should not be taken literally or as gospel. Usually they are descriptive information that is intended to simplify or provide general guidelines, so they are helpful if taken in context. He does also point out that some are just not helpful at all, like breed prejudice.
Missteps (Common Trainer Mistakes):
Using behavior before it’s complete: If a behavior is not complete, don't use it. For simple behaviors, using them too soon creates sloppy behavior. For difficult behaviors, using them too soon can break down trust and make the behavior harder to complete. If all approximations are not done, don't use the behavior.
Notes from Katie:
Ken started this talk by explaining why he wanted to focus on "mistakes.' Positive reinforcement trainers emphasize what to do, both in their training and their teaching. This is great and we all need to focus on what works, but looking at things that can go wrong is also valuable. The idea here is to be forewarned about things that can go wrong (so you can avoid them), and provide a little perspective on how to interpret many of the stock phrases that are thrown around by animal trainers.
I think Ken's list clearly shows why animal training is so challenging to do well. There are a lot of contrasting pairs on his list. You don't want to be too predictable, but you do want to be consistent. You don't to take something too literally, but you do need to recognize that there is an element of truth to it. As trainers, there are a lot of things we need to observe and monitor. Finding the right balance is important and paying attention to details can make a big difference.
This list was geared a little bit toward dog trainers, but I think most of them apply to any kind of animal training. Even the sayings that are "dog specific" have parallels in the horse world. For example:
Dogs are wolves - > with horses, there are a lot of people who put emphasis on acting toward a horse as if you were another horse in the herd.
Leader of the pack - > same thing with there being an emphasis on the importance of being "dominant" or "getting respect."
Breed prejudice -> I see this with some trainers who expect certain breeds to act in certain ways and miss the horse's individual needs or abilities.
I would add a few other myths or sayings that horse people encounter:
Hand feeding makes horses bite
A horse moving into your space is disrespectful
Anyone want to add anything else?
Dr. Jesús Rosales-Ruiz: Respondent vs. Operant Conditioning
Dr. Jesús Rosales-Ruiz: Operant vs. Respondent
Jesús started with some history. B.F. Skinner first made the distinction between operant and respondent conditioning. In operant conditioning, the animal's behavior is determined by the consequences and there is a more active relationship between the animal and its environment. When we clicker train, we are using operant conditioning because the animal learns it needs to do something to get clicked. In respondent (also called classical) conditioning, the animal's role is more passive. Respondent conditioning is about making associations. The animal is not necessarily actively acting on the environment to earn reinforcement. An example of respondent conditioning is conditioning the clicker. If you condition the clicker by just clicking and feeding, regardless of what the animal is doing, you are using respondent conditioning. The animal learns that the click predicts food, but it does not know yet what it needs to do to get a click. This is why it is "passive." Counter-conditioning where you associate something good with something that has previously been a trigger for fear or aggression is also respondent conditioning.
Changing the behavior of aggressive or fearful animals has traditionally been the domain of respondent conditioning as respondent conditioning is thought to affect an animal's emotional response to the antecedent (trigger) and this change in emotion can then lead to changes in behavior. A scared animal will have a lot of unwanted behaviors that will go away on their own once the animal is no longer scared. The idea is that behavior is determined by the animal's emotional state. Change the animal's emotional state and the behavior will change.
When Jesús started working with aggressive dogs, he expected to use respondent training techniques to change their behavior. But at some point he realized that the dogs were acting more as if their aggression was an operant behavior. They were aggressive because it worked for them, meaning aggressive behaviors were being reinforced by something they wanted. In this case, the dogs were using aggressive behaviors to create distance. From this observation, he and Kellie Snider came up with the CAT protocol which is based on the idea that aggressive behavior is maintained by negative reinforcement. Understanding that a behavior is maintained by negative reinforcement changes your strategy for changing it. In this case, they found they could use negative reinforcement (removal of aversive by adding distance) to reinforce behaviors they did like (calm behaviors) and the change in behavior led to a change in the dog's emotional state.
This experience led him to re-evaluate his own thoughts about respondent vs. operant conditioning and this was the subject of this talk. He believes that most respondent conditioning is really operant conditioning and we just don't look close enough to see the consequences that are driving the behavior. He took another look at desensitization and counter-conditioning with more attention to what is really happening and looked at examples of respondent procedures to see if there was some element of operant conditioning that had been overlooked.
After explaining how he came to look more carefully at the distinction between respondent and operant conditioning, he spent a little time on the historical perspective. I am including a bit of this here because I think it's important for us to have some idea that when we read about these terms in books and articles, we need to keep in mind that this has been the subject of much discussion. P
As I said, the distinction between respondent and operant started with B.F. Skinner. Prior to that, there was no distinction. Ivan Pavlov and Edward Thorndike both studied behavior and how it changed as a result of the environment, but they did not believe there were two types of conditioning. I thought this was pretty funny considering we now think of Pavlov's work whenever we need an example of respondent conditioning. In Science, it is easy to end up with dual views as scientists often get caught up in seeing behavior from one perspective. Jesús showed some charts of behaviors events and environmental events (Donohoe and Vegas) which showed how you can interpret them as demonstrating respondent conditioning or operant conditioning depending upon where you put your focus. You can make arguments for either respondent or operant conditioning happening depending upon what behaviors and interactions you observe as well as where you enter the behavior stream. He suggested that rather than have a dual view of behavior, it would be better (and more practical) to have a "Unified Theory of Reinforcement."
He led us through some of the developments that started to erode the distinction between respondent and operant conditioning. Originally, the two types of responses were defined as follows. Respondent conditioning was autonomic, involuntary, elicited and did not change the environment. Operant conditioning was skeletal, voluntary, emitted (or permitted) and did change the environment. Skinner did not believe that operant conditioning could control things like blushing or other "automatic" body functions. But in 1957, scientists showed that gastric skin releases, blood pressure and intestinal contractions were all controlled by their consequences, which made them operant, not respondent. Most of these are controlled by negative reinforcement and he had some really interesting examples of how the consequences could significantly change what happens. He also pointed out, and this goes back to his beginning statement, that if you don't look for the consequences for a behavior, you can miss what is controlling the behavior and this can make it impossible to change it.
Ok, so this is all fun and interesting (at least to me), but what does it have to do with training? To answer this, Jesús shared some charts from an article about the two types of conditioning (Rescorla and Wagner - the article is titled "Pavlovian Conditioning: It's not what you think it is"). According to Rescorla, both respondent and operant conditioning require a contingency and the spacing and order of events are important. In both respondent and operant conditioning, the response has to occur first in order for conditioning to happen. And it must be predictable so that both the antecedent and the consequence can be associated with the behavior. In short, both operant and respondent conditioning require many of the same things so when we train, we need to keep both in mind.
He used the example of transferring cues. Once a behavior is happening reliably, most trainers like to put it on cue. Adding a cue can be done by inserting the new cue before the old cue in the behavior stream. The new cue becomes a predictor for the old cue and the animal will eventually anticipate and offer the behavior before you use the old cue. This is usually explained as happening through respondent conditioning. But if we look at the whole behavior stream, then we see that the consequence is just as important. If you add a new cue but don't reinforce the behavior, the animal will not learn the new cue. Writing this I am thinking that this is pretty obvious and why would anyone expect to add a new cue without reinforcing the behavior. But that's not the point. The point is that we talk about adding new cues as a respondent process, and it's not. It requires both respondent and operant conditioning in order to be successful.
Often the most challenging part of training is getting the behavior started. He has some video that showed how food can be used to change both the emotional state of the animal, and get some changes in movement. As clicker trainers we typically focus on adding reinforcement (often food) after the behavior has occurred, but perhaps we need to be more flexible about this. He has some video of Kay Laurence who starts a lot of behaviors with food and then switches to food as reinforcement once the animal has "gone operant."
This means that depending upon where you start in the process, you can be starting with respondent or operant conditioning. In order to change behavior, you have to get some variation and you can approach it from the operant or respondent angle. Once you have some variation, and reinforcement is happening, then the animal looks for information from its environment. The animal will be looking for a connection between its behavior, the consequences, and the antecedent.
This was really the take home message of the talk. It was about looking at all parts of the conditioning process to see where you could insert a change that would take you in the direction you want to go. People tend to get hung up on the importance of that first step. If you start with food (not attached to a behavior), then it is perceived that you are using respondent conditioning whereas if you wait for a behavior and then add food (contingent on the behavior), then it is operant conditioning. But once you are past the first step, both are happening because the animal is looking for information from its environment to tell it how to get more of what it wants. It is better to think of both types of conditioning happening at once, without a clear separation as that is more close to the reality of what is happening.
He did talk a little bit about the best order in which to present the US (unconditioned stimulus) , CS (conditioned stimulus) and UR (unconditioned response) to get optimal conditioning. CS- US- UR works best, but you get conditioning in all three. You can block conditioning which he talked about a little, but my notes on that are pretty brief. The article he referenced was Kamin, 1970 if you want to look it up. What I remember is that it was possible to prevent an association from being made depending upon the timing. We do see this with training. If I have a cue to ask a horse to back which is a hand signal and I want to add a verbal cue, I have to add the verbal cue before the hand signal. If I do them both at once, the hand signal will "block" the learning of the verbal cue because they are happening at the same time.
He had some video of trainers using counter-conditioning and desensitization with dogs and if you looked carefully, you could see that specific behaviors were being selected out, even though the trainer thought they were working on the emotional state of the dog. It takes a lot of work to divide emotion from behavior and in most cases we don't want to. He actually said you can't divide emotion from behavior, but I'm not so sure. I always remember Kathy Sdao's example of capturing ET's bell sound (ET was an aggressive walrus). The sound was one he made when he was mad and they reinforced it and put it on cue. Once it was on cue, he would still make the sound, but it no longer meant "I'm going to kill you."
I am still thinking about all this, and will have to see if it actually affects what I do. As trainers, we need to figure out how to get a behavior started, reinforce it so it occurs again, and put it under appropriate stimulus control.. If we approach most problems with the operant conditioning mindset, we will be providing reinforcement which is targeted toward increasing a specific behavior, and also contributes positively to the emotional state of the animal. Ken Ramirez told Jesús that most of the training (both training new behaviors and decreasing unwanted behaviors) at the Shedd Aquarium is done with operant conditioning. He added a section to his book on respondent conditioning after someone else pointed out that he had not included it. I personally have found that once an animal really understands how clicker training works, it is really hard to use respondent conditioning with food (or other reinforcement) without them attaching it to some behavior because that is essentially what we have taught them to do.
Bob Bailey says that when you are training "Pavlov is always on your shoulder." Perhaps to balance this out, when you are using respondent conditioning, you should probably tell yourself that "Skinner is always on your shoulder" too...
This lecture had a little bit of overlap with the respondent vs. operant lecture, or maybe it picked up where the first lecture left off. In any case, Jesús started by
One way to understand about cues a little better is to look at how the environment affects the reliability of cues. When we train any behavior, there is a set of c
Jesús had some video of Kay shaping a behavior and then asking for the same behavior while holding a broom. The dogs were clearly confused by this change in her body posture and the addition of another piece of equipment. The audience got a bit caught up in the idea that the broom was aversive, but I think Jesús's point was that any change can have an affect and you have to realize these things matter when you plan your training.
He spent some time talking about the difference between "emitted" and "permitted" behavior. Behavior analysts refer to behavior as being "emitted," meaning that there is no antecedent control and it just spontaneously happens. Skinner made the distinction between behavior being "emitted" vs. "elicited." The distinction he cared about was the idea that consequences could make an animal more likely to "emit" a behavior. You didn't have to do something ("elicit") to get the behavior. Jesús said he is now skeptical that behavior is ever just "emitted." Something has to happen to make you change your behavior, otherwise why would you change? Having read some of Kay's articles recently, I would say this is a case where the consequence becomes the antecedent for the next behavior.
Even the word "emitted" might not be accurate. Donohoe suggested that a better word would be "permitted," meaning the behavior is permitted by the environment. The current stimulus situation determines what is possible. It is actually pretty easy to see how this applies to training. We use this all the time in training when we set up the environment to limit the animal's options or make a specific behavior more likely. At chicken camp, they put the chickens up on tables so there are fewer behaviors they can do. Michele Pouliot uses platforms in her dog training to limit movement and make certain positions more likely.
The other idea that comes out of the distinction between "emitted" and "permitted" is that it changes the way the trainer works. If you believe behavior is emitted, then you are going to wait until something happens. If you believe behavior is permitted, then you are going to set up the environment so that behavior is more likely to happen. I thought this was a pretty interesting distinction and is consistent with what I see happening with clicker training as the training becomes more sophisticated. When I first started training, there was a lot of emphasis on just waiting for the animal to do something and the trainer was very passive. Now I see much more emphasis on setting up the environment to make the target behavior more likely. The includes adding antecedents (prompting, luring) which are faded after a few repetitions, but considered useful to get the initial behavior started.
The bottom line here is that it is best to assume that behavior is always under some kind of control by the environment. As a trainer, you need to ask yourself "what environment would permit the behavior I want?" This works for training new behaviors and for changing old behaviors. If you want to change an unwanted behavior, you need to find the antecedent. If it's something you can control, then removing it will change the environment so the unwanted behavior is not permitted. If you want to get a new behavior, you need to set up the correct environment to permit new behaviors. He had a few examples including Kay demonstrating how to teach a tail wag by having the dog lie on the floor where the tail's interaction with the floor becomes important feedback, and Alex's use of the "slouch" as a cue for the horse to offer behavior in a micro-shaping session. He had a line here which I loved. He said "this is not your mother's differential reinforcement." Everything is becoming more sophisticated.
He had some other interesting video clips showing horses being wormed and teaching a dog left and right. In the worming case, they were looking at what antecedents were cueing evasive behavior on the part of the horse. It turned out that there had to be number of conditions (presence of bucket, lead rope, handler's movement, etc...) and the actual wormer was not one of them. Worming with an "invisible" tube created the same problem. They had to look quite far back in chain (looking for antecedents) in order to find out what triggered the first evasive response. I see this with horses that are reactive to shots. They are often reactive to position and body language, more than a syringe in your hand. With the dog, he showed how the trainer inadvertently created a cue which was going to make her long term training project harder. The take home message was that problems with cue control are often about consequence control.
Another important point about cues is how understanding how to add a cue. A lot of what he had discussed before this was about a new way of looking at cues, how cues evolve or are created as part of the training process, what can interfere with correct responses to cues, and how cues can be accidentally created. Now he talked about the specific process of adding a cue to an existing behavior. This goes back a little bit to the talk on respondent vs. operant conditioning because cues are taught by association. The animal learns that doing a behavior under certain conditions (presence of the cue) means reinforcement is likely.
In order for an animal to learn a new cue, the cue must become associated with the behavior. In a finished behavior, we cue the animal which responds correctly and is then reinforced. There are different methods for adding the cue. Some trainers don't add the cue until they can predict when the animal is going to do the behavior. They can use the "new cue" "old cue" method or just add the cue once the animal is offering the behavior in a predictable pattern. Other trainers will add the cue while the animal is doing the behavior and slowly change the timing so that the cue precedes the behavior.
In both cases, the cue must be added in such a way that it is salient to the animal and becomes attached to the behavior AND the consequences. Sounds simple? Well, it turns out the timing matters. He used the Kamin experiment on blocking to show that when you add the cue, you need to make sure that there is not another cue in the environment that will prevent the animal from learning the new cue. In the Kamin experiment the rats were conditioned to freeze (yes, yet another experiment shocking rats...), and they looked at whether or not a new cue could be conditioned if an existing cue was in place. If the new cue (light) is presented at the same time as the old cue (tone), no conditioning occurs. They did several combinations (differences in timing of light and tone), but conditioning only occurred if there was a discrepancy between the presentation of the new cue and the old cue. It may appear as if both are part of the stimulus condition for the behavior, but unless you try them separately, you don't know. This is why you need to identify possible stimuli that are part of a context cue and also why you need to add a cue by doing "new cue," pause, "old cue" so that the animal pays attention to each one individually.
The last part of his talk was perhaps the most interesting as it drew together a lot of these separate threads. And it led to his new "JRR Principle of Learning." The JRR Principle of Learning says that in order for learning to occur, there has to be something in the environment to which the behavior can become attached. If you can't predict when something is going to happen, then you can't learn from it.
He showed this with some video of rats being trained to take vodka out of syringes. The rats were trained to move between two stations on a desk and take apple juice out of a syringe. Once they were doing this reliably, the syringe was sometimes filled with vodka instead. The experimenters replaced the apple juice with vodka so that it was about 4 syringes of juice to 1 syringe of vodka. The rats hated the vodka and their body language clearly showed their disgust, but they continued to take it because they could not tell when a syringe would contain vodka vs. when it would contain apple juice. There was no antecedent to indicate when it was vodka, so they could not learn to avoid it. Over time their body language did change so that they seemed to be "getting used to" the vodka and it seemed less aversive.
Then they made it so there was a "cue" for vodka which was a paper plate around the syringe. Within 2 or 3 trials the rats no longer drank from the syringe with the vodka. So even though their body language suggested they didn't mind the vodka, they still chose not to drink it. And even if they were reinforced for drinking vodka (with a syringe of apple juice), this was not enough to maintain the behavior. The reinforcement value of the apple juice was not enough to counter the unpleasantness of the vodka.
Jesús's conclusion from this was that it's not enough to just read the body language of an animal that might be asked to do something it doesn't like. You don't know how the animal really feels about until you give them a choice with a clear cue to indicate what they are choosing. I think this has a lot of ramifications for some of the husbandry work we do with horses. I realize that some things are not optional, but I also think that we should be careful about assuming a horse is ok with something based on its body language alone. In some cases, it might be a good idea to set up a little experiment to see how the animal really feels about it.
Kay Laurence: Connected Walking
Kay Laurence: Connected Walking
Most people walk their dogs using some kind of equipment. There is nothing wrong with this (safety is important), but you should recognize that all equipment is inherently punishing. Dogs are not born to wear anything so even mild equipment like a flat collar takes getting used to. Also, a lot of equipment is designed to suppress behavior, not to teach the dog what to do. The wrong choice of equipment can prevent you from building connection. Equipment should be chosen carefully and used in such a way that it doesn't have a negative effect on the experience of being together. Along those lines she talked a little bit about how dogs can learn to make associations such as being on leash = bad stuff is going to happen vs. being off lead = freedom and fun. If you want your dog to enjoy being on leash, then you need to make sure fun stuff happens when the dog is on leash.
How does equipment suppress behavior? Some of the ways this happens are obvious, as many items are marketed as preventing certain behaviors (no pull harnesses, head halters, etc..). Other ways are just a by-product of what the equipment is intended to do. Equipment can make dogs feel trapped and frustrated. It can prevent them from behaving in normal ways and force them to put up with unwanted advances by strangers (both people and dogs). It can create behavioral resistance by actually strengthening behaviors you don't want. And if the equipment is aversive enough, it can make the shared experience of walking together so unpleasant that this spills over into other areas of your relationship.
In addition to looking at the effect of equipment on walking with our dog, we need to look at the physics of how we walk when we walk with our dog. Kay has spent some time studying dog/human pairs that can walk comfortably together and comparing them to dog/human pairs where the dog has to do a lot of stopping and starting, or ends up pacing. It turns out that if you compare normal walking speed (and stride length) for most people and the normal speed (and stride length) of dogs at the walk and trot, you can see that there are only certain combinations that are going to mean that both the person and the dog are moving at a comfortable pace.
If the person's walking speed is between the dog's walk and trot speeds, then the dog is going to have to start and stop to keep pace, or it is going to have to find some in-between speed which often ends up causing the dog to pace. Pacing is very hard on a dog and can lead to physical issues so it is not something you want your dog to be doing. This means the person has to adjust by either walking slower so the dog can also walk, or walking faster so the dog can trot. Otherwise the dog has to compensate and you get pulling, starting and stopping or unnatural movement. She believes that a lot of pulling happens when the dog's trot is faster than the person's walk and the dog doesn't compensate by starting and stopping or pacing. If you can the dog can find a comfortable speed for both of you, then the leash should be able to be loose or have a soft connection.
When I got home I walked our dogs to see how we are doing. It turns out that my casual walk is a nice speed for the shi-tzu to trot. If I take the border collies out, I have to walk slower to accommodate their walk or walk faster so they can trot. I thought it was pretty funny that the most comfortable dog for me to walk is actually the shi-tzu, considering I am about 5'7" and have long legs.
er connected walking protocol has three basic components which are
connected walking, transport, and parking. I am going to start by
describing connected walking because the goal is to work towards doing connected
walking most of the time. Connected walking is about walking together, but
in a more companionable sense than the usual idea of "taking the dog for a
walk." It is more about enjoying being outside moving through the environment in
a protocol where both you and the dog are allowed input in what you choose to
do. The dog is allowed to sniff, look at things, stop, etc.. within
reasonable bounds that you can set. The way you establish these boundaries is by
using the leash to limit the dog's behavior and teaching the dog to reconnect to
you so you can move on.
er connected walking protocol has three basic components which are connected walking, transport, and parking. I am going to start by describing connected walking because the goal is to work towards doing connected walking most of the time. Connected walking is about walking together, but in a more companionable sense than the usual idea of "taking the dog for a walk." It is more about enjoying being outside moving through the environment in a protocol where both you and the dog are allowed input in what you choose to do. The dog is allowed to sniff, look at things, stop, etc.. within reasonable bounds that you can set. The way you establish these boundaries is by using the leash to limit the dog's behavior and teaching the dog to reconnect to you so you can move on.
T his element of the dog choosing to reconnect with you is central to the
idea of connected walking. You can teach it by using a clicker and treats
(or some other reinforce), but the clicker and additional reinforcement should
become less necessary and eventually faded out as the connection between you and
your dog builds. With many dogs, she doesn't use a clicker and treats at all.
The moment when the dog reconnects is a "visual click" and continuing the walk
functions as a reinforcer too. She had some video of a woman walking
her dog with a nice connection. They were moving together at a relaxed pace.
If the dog wanted to stop and look at something, the woman waited until the dog
reconnected with her, which in this case was when the dog looked at her and
moved toward her. If the dog took the slack out of the leash to pull in a
direction she didn't want to go, then she just waited holding the leash steady
at the point of contact (sound familiar?).
his element of the dog choosing to reconnect with you is central to the idea of connected walking. You can teach it by using a clicker and treats (or some other reinforce), but the clicker and additional reinforcement should become less necessary and eventually faded out as the connection between you and your dog builds. With many dogs, she doesn't use a clicker and treats at all. The moment when the dog reconnects is a "visual click" and continuing the walk functions as a reinforcer too. She had some video of a woman walking her dog with a nice connection. They were moving together at a relaxed pace. If the dog wanted to stop and look at something, the woman waited until the dog reconnected with her, which in this case was when the dog looked at her and moved toward her. If the dog took the slack out of the leash to pull in a direction she didn't want to go, then she just waited holding the leash steady at the point of contact (sound familiar?).
Connected walking does take some skills. The person walking the dog
has to learn how to hold the leash (she recommends a trapeze hold), adjust the
length of the leash as needed, and wait in a stable position if using the leash
to set a boundary. They have to learn to assess the environment to see
what's coming and make changes as necessary. They have to learn to walk
with the right speed, energy, and rhythm. They also have to make decisions
about knowing what and when to reinforce and when to change strategies,
especially if they need to go into "transport."
Connected walking does take some skills. The person walking the dog has to learn how to hold the leash (she recommends a trapeze hold), adjust the length of the leash as needed, and wait in a stable position if using the leash to set a boundary. They have to learn to assess the environment to see what's coming and make changes as necessary. They have to learn to walk with the right speed, energy, and rhythm. They also have to make decisions about knowing what and when to reinforce and when to change strategies, especially if they need to go into "transport."
"Transport" is the second component of connected walking and it is pretty much what the word implies. It is a method of walking where you are moving the dog from one place to another with efficiency. She recommends that you use transport in situations where you just need to move the dog quickly either away from something or to a new location. In transport the leash is shortened and you are using pressure on the collar to direct the dog in the direction you want. Dogs pick up on our intent so if you walk with a sense of purpose, the dog will too. Most dogs learn the difference between connected walking (loose leash or soft connection) and transport. If the dog is uncomfortable with transport, you can click and treat every few steps until the dog understands the protocol.
When you are out and about with your dog, you are going to be using a mixture of connected walking and transport. For younger dogs, or dogs new to training, or in some environments, you are going to use a lot of transport. Over time you will be able to add in more and more periods of connected walking until transport is only used when something unexpected happens.
The third component of connected walking is "parking." When you park your dog, you step on the leash so that the dog has limited room to move (more on this below). Parking is about teaching the dog what to do when you are not moving. It's a useful behavior to do in dog training class if you need to stop working and listen to the instructor. It's a useful behavior to do if you want to stop and chat with someone while walking your dog. It's also a useful behavior to do if you want to prevent someone else from having access to your dog. Kay is a big believer that our dogs do not need to be touched by strangers. She spent part of the lab showing people how to teach their dogs to park and also had them practice "repelling" strangers by stepping forward and blocking access to the dog.
So how do you park your dog? While the dog is near you, you are going to take the dog's collar and hold it, then carefully step on the leash so that the dog has less freedom to move. The dog should have enough length of leash to sit or lie down next to you and still have slack in the line. With small dogs, the dog may have enough room to stand or move around a little, but there should be a slight downward feel on the collar if the dog moves too much. This will encourage the dog to choose a more stationary (and lower to the ground) position. Kay does not want the handler to interact at all with the dog when it is in park. This is "down time" for the dog, so no talking, no treats, no patting. If you are consistent about ignoring the dog, it will learn that it is just supposed to take a break and settle down. When you want to move off again, you carefully step off the leash by stepping back, wait for the dog to connect and move off.
In the lab, Kay demonstrated parking, repelling strangers, connected walking, and transport. She showed how you can move easily between connected walking and transport. The dogs and their people went for a little walk through the lobby and around the mezzanine and learned how to move at a pace that was appropriate for the dog. In one area where there was a lot of activity, the dogs needed a little time to think before moving on and in other areas Kay had them use transport to get past an obstacle.
At the end of the lecture, Kay shared these thoughts. Connection is about investing quality time into a relationship and enjoying your dog for being a dog. A lot of dog training puts the emphasis on "fixing" your dog. But part of what we like about dogs is who they are. Rather than "fixing them," dog owners should learn to dismiss the need for external social approval and do what is right for them and their dog. This means having realistic expectations, being together in the now, and enjoying the reinforcement that comes from connection.
The lab was designed to show how this actually works. She started off by talking a bit about the advantages of using targets this way. She finds you get less variability in the behavior and this makes it more reliable long term. This is partly because in addition to adding the target later, you also don't start clicking until later. You don't start clicking until the behavior you want (for that step in the process) is happening on a consistent basis. This means you have only clicked responses that are closer to your final criteria.
Kay likes to keep things simple. She doesn't use body parts (hers) as targets. She said if you use your hand as a target, the dog will sometimes become confused about what to target and this can create a "stutter" in the behavior. You don't need to use modifiers (left, right etc..) as the target object alone tells the animal what to do. If you want to use the same target to create different effects such as a high five vs. wave with a paw target, you can change how you present the target to get the different behaviors.
The last behavior she worked on was "trotting out" which is getting the dog to follow a target in motion. She uses a "scent target" which is a dog treat in a cup on the end of a stick. The dog orients toward the cup and this encourages a certain body posture. The "scent target" also creates focus. At home she will reinforce the dog by throwing the treat that is in the cup. I think she just tosses it by tipping the cup slightly and swinging the stick. This didn't work very well here with the set-up she was using, so she just threw a treat as if it had come from the cup.
This led into a bit of a discussion about why one needs to use a clicker at all? What is the purpose of the click? I think we all know the standard answers to this question which are that it adds precision and allows you time to deliver the reinforcement. The click marks a specific behavior so by selecting it out of a stream of behavior. This adds precision and removes uncertainty and increases focus. The click also allows you to separate out the click from the reinforcement.
This is the part Kay was interested in. She wanted to show that this gives us a lot of flexibility in how we handle the reinforcement. If you always do the click -> reinforcement part in the same way, you are missing out on opportunities to increase the power of the reinforcement or the clarity of the click. She did a little demo of the difference between holding the food and throwing it right after the click as compared to getting the food from the table right after the click. The little delay as she walked, from where she was standing when she clicked, to the table made the dog focus back on her and built anticipation. This also had the benefit of teaching the dog to go away from the reinforcement (on the table) so that it could do the behavior that would earn the next reinforcement. This builds power and reliability.
Perhaps it depends upon how cold you are? This is kind of a silly example, but there are lots of situations where you can argue that something is positive reinforcement or negative reinforcement, depending upon how you look at it.
Negative reinforcement results in an increase in one behavior (or maybe a related set of behaviors), but this can manifest itself in different ways. They can be broadly classified into either "escape" or "avoidance," but Jesús broke them down farther into the categories listed below:
5. Reduction of intensity. People diet to reduce their blood pressure. If that is why you are dieting, then dieting is maintained by negative reinforcement.
An example he shared was training loose leash walking. A trainer who is focused on negative reinforcement in a constructive way is going to be focused on using negative reinforcement to increase the likelihood of a slack leash. They will not be focused on using it to make the dog do less of something (pulling). This may seem like a subtle difference, but in training the intent of the trainer is very important.
He talked a little bit about the importance of choosing your reinforcer and having a good training plan. To make a good training plan you need to look at what you have (component behaviors), decide where you want to go, decide how to get there, and choose the type of reinforcement you are going to use. You are better off with a good training plan and mediocre reinforcers than you are with excellent reinforcers and a poor training plan. He also does not recommend mixing types of reinforcement. If you use both positive reinforcement and negative reinforcement, then you don't really know what is motivating the animal.
A constructive approach to training, including how you handle errors, is very different than the more traditional ways to handle errors. He had a list of 9 ways to deal with errors that is published by a traditional dog trainer. Every one of the nine involved some kind of punishment. I can't remember exactly why he shared the list, but I think it was partly to show how negative reinforcement has been used by dog trainers as part of punishment and that this is why many trainers choose not to use it at all.
At this point he talked a little bit about jargon. He had already mentioned how negative reinforcement and punishment are often confused, but now he expanded upon the idea that the jargon that is used by behavior analysts actually makes it harder to understand what the quadrants do. He shared some other examples of commonly held beliefs about the quadrants, and suggested that perhaps we need to rename the quadrants to get rid of the confusion. Other behavior analysts have suggested the same thing. Lindsely suggested the following alternative names: positive reinforcement= reward, negative punishment = penalty, positive punishment = punishment, and negative reinforcement = relief.
While renaming the quadrants might be one step toward removing some confusion, the reality is that what really matters is your procedure. Your procedure needs to creates a positive learning experience for the animal. You also need to be using your reinforcement (positive or negative) to build behavior by focusing on what you want. If both of these are true, then you are using your reinforcement in a constructive way and that is consistent with the principles of clicker training.
That led to the next part of the lecture which was a closer look at different ways that negative reinforcement is affecting our behavior on a daily basis, whether we know it or not. To understand this, it helps to remember that negative reinforcement can result in escape or avoidance. Jesús mentioned earlier when he was listing the ways that negative reinforcement can operate. In escape, the animal learns to that it can remove (or get away from) an aversive by doing another behavior. In avoidance, the animal learns that it can avoid the aversive by doing another behavior.
Whether negative reinforcement leads to escape vs. avoidance can change over time. In many situations, negative reinforcement starts as escape and then becomes avoidance. This can happen if there is a stimulus that precedes the addition of the aversive. This stimulus becomes a cue and the animal can avoid the aversive by responding to the cue. If there is no cue, the animal can't learn to avoid the aversive and is stuck in escape (remember the rats drinking vodka in the "Cues and Context" lecture?).
Once you recognize the many ways that negative reinforcement can operate, you realize that it is everywhere. A lot of the information we get about our world is in the form of cues that allow us to avoid aversives. Many of the behaviors that we do when driving a car are maintained by negative reinforcement and they are cued by things like traffic signs (stop, yield, merge, etc...), traffic lights, speed limit signs, warning signs ("bridge freezes before road," "windy road," etc..).
A lot of our "choices" are maintained by negative reinforcement. Why do students study? Some students study because they are afraid to fail. This is a different kind of motivation than studying because you are interested in the material. Why do people to work? Do they go because they like it, or because they are afraid of getting fired, not having money, or some other reason that is more about avoiding a consequence?
Why does this matter? It matters because motivation matters. If you are avoiding, escaping, or running away from something, you are going to be in a different mental state than if you are moving towards something.
Addictions are maintained by negative reinforcement. Withdrawal feels bad so even if you have some desire to stop, you might be unable to do so because the urge to do it again is too strong. Incidentally, Jesús pointed out that if you do give in when you have the "urge," you are making things worse than if you gave in at some other time.
These examples may make it seem like negative reinforcement is always bad, but this is not the case. If you think back to the lecture on operant vs. respondent conditioning, Jesús noted that a lot of biological feedback loops (in our body and other places) are controlled by negative reinforcement. I always remember Kathy Sdao's example of urinating as being controlled by negative reinforcement. When we urinate, we feel a sense of relief. Jesús used the example of a crying baby. Babies cry when they need something and we respond because the sound is aversive. This is a good thing! We need to take care of crying babies.
So if negative reinforcement is prevalent in the environment and so effective, why is its use in training controversial among animal behavior analysts? There are a number of reasons including:
1. motivation - why you are
doing something is important as it affects your emotional state
Jesús quoted Iwata (1987) as saying "In at least one sense, -R might be considered to be more intrusive than punishment because with -R, presentation of the aversive stimulus is contingent on the presence rather than the absence of behavior." He also quoted B.F. Skinner who said "If we knew as much about -R as we do about +R, we might find it good for shaping, but at the moment it's ineffective and has negative by-products."
So where does this leave us with regard to training with negative reinforcement? On one hand, it is in the environment and clearly affecting our behavior so it could be an effective training tool. On the other hand, there are clearly disadvantages to using it because of its close association with aversives and punishment. He spent the last part of the lecture showing how trainers can use negative reinforcement in a positive or "constructional way." This is really an important piece. It is about recognizing what is controlling the existing behavior and setting up conditions where reinforcement works best (whichever you choose). If you start with negative reinforcement, you have the additional step of switching over to positive reinforcement.
He shared two examples. One was the CAT (constructional aggression treatment) protocol which uses negative reinforcement to shape friendly behavior in dogs. For a fearful dog, the best reinforcer is distance. Distance is more important to them than food, toys, or other things that could be added with positive reinforcement. He showed a video clip of a shelter dog who was fearful about being approached. The trainer would approach the dog and retreat when the dog showed some sign of calmly accepting his presence. This could be a change in posture or movement. They look for behaviors that are seen in "friendly" dogs.
Over a series of approaches, the dog becomes less and less fearful until the dog is comfortable with the trainer approaching. Once the trainer is close, the dog will indicate when it is ready for an interaction that involves positive reinforcement. So they start with negative reinforcement, but then change over to positive reinforcement. (If you've never heard of CAT and want to learn more about it, google CAT + Jesús Rosales-Ruiz).
An important point here is that aggressive behavior in dogs is often maintained by negative reinforcement. The dog is using aggression to get the aversive (other dog, person, etc..) to move away. By using negative reinforcement to shape the dogs behavior to from aggressive to friendly, Jesús is using the same motivation. He finds that behaviors that are maintained by negative reinforcement are often easier to solve by using negative reinforcement.
His other example was the rope handling that Alexandra Kurland uses. In her rope handling and use of pressure, Alex teaches the horse to accept physical contact (either through a rope or body contact) as information. She starts out with adding a small amount of pressure and waits for the horse to respond. There is no escalating of pressure and the amount used is individual to each horse. The release of pressure is marked with a click and she reinforces the horse for the correct response. Over time she can use less and less pressure and the contact becomes more of a cue.
This is the same idea of starting with negative reinforcement and then moving toward cues and positive reinforcement. I'm not sure one can say that a cue taught this way does not have some component of negative reinforcement, but you can always add a new cue once you have the behavior. The poisoned cue video showed that the behavior is not poisoned, it is the cue that is poisoned. Adding new cues is one way to remove any emotional fallout from using negative reinforcement. If you do that, then you don't have to wonder if your cue is a conditioned aversive stimulus or a cue with positive associations.
There was a little discussion here about if the animal is "learning" when it is being directed by negative reinforcement. Jesús indicated that in order for real "learning" to happen, the animal needs to learn to do the behavior without guidance. I have to say that I'm not sure I agree with such a narrow definition of learning. I think animals are learning all the time and one thing I want my horses to learn is to accept various kinds of guidance from me. Yes, if I want them to do the behavior independently, I need to slowly remove the guidance (like taking training wheels off a bike), but that doesn't mean that they are not learning important things while they are still in the "guidance phase."
These examples show that negative reinforcement can be used in a "positive" way if applied with discretion in a positive training environment. Jesús said that if negative reinforcement is already controlling a behavior, you can often change it by using negative reinforcement (as in CAT). If an animal is not offering behavior, you can use negative reinforcement as information to help the animal find the right answer (as in Alex's work). He added that even with these examples, the goal is to move towards positive reinforcement because for most animals positive reinforcement is more fun.
Panel Discussion with Karen, Kay, Alex, Julie, Kathy, Ken, Jesus, and Michele
Every year they have a panel discussion with some of the faculty members. I always enjoy this as it's a chance to listen to different viewpoints on the same topics. Th
There was a little discussion here about how clicker training changes the relationship between people and their animals. Karen said that by teaching animals about cues and other concepts, we teach them to think and make choices. This allows us to have a different type of conversation that is different than working with non-clicker trained animals. Kathy talked about encouraging creativity and how it changed the trainer's perception of what the animal could do. Kay thinks clicker training changes relationships because people are listening to animals instead of just telling them what to do. There was general agreement that clicker training should empower the animal.
Aaron posed this as a question from a frustrated student who didn't see why the animal had to be given such precise cues. Why can't the animal just figure it out even if the cue is a little sloppy?
Dr. Susan Friedman: What the word PARROT reveals about teaching dogs
Dr. Friedman also talked during the closing ceremony on Sunday.
I hope you enjoyed reading the Clicker Expo reports. If you have any questions or comments, feel free to email me at email@example.com.
Katie Bartlett, 2013 - please do not copy or distribute without my permission