Exactly what is the DTA4 validation process?
I have read most of the important sections of the DTA4 test about three times now and I have several reservations about the test.
My first reaction is to question whether this is truly science or a primitive belief system, unworthy of being classified as science. Science demands the highest standards of accountability and integrity of methodology. To claim findings as science when there is no real evidence presented is to discredit the whole concept of science.
To me this test appears to be non-scientific hokum, a quick and easy way to deal with animals and to probably eliminate the ones who do poorly on the test.
The DTA4 test writers repeatedly proclaim that their tests are valid, but saying they are valid requires a lot of facts. Where are the data on record? Have peers or academic supervisors had access to the data? Or is it just unreviewed work done for an organization?
THE UK BASED RSPCA DOES NOT NOW USE A SIMILAR TEST
The forerunner of the DTA 4 was a test conducted in England. We are just told that the results were valid, but no data is presented..
Apparently the English version of test has been abandoned in favour of a 10 day in-shelter assessment. For details of this test see the previous post on Animal Advocates Message Board- http://www.animaladvocates.com/RSPCAdogTest.pdf
To quote a sentence from the DTA4 manual, “Since then DTA has been performed and validated throughout the UK and at two BC SPCA shelters, more than 1000 dogs have now gone through the DTA validation process.” We need more details than this. What definition was used of “valid?” Just where were the tests done? Were any stray dogs included in the English study? Are conditions comparable between the dogs in UK shelters and the ones here?
The BC SPCA validation procedures were assessed using 211 dogs. 106 were eventually adopted by new owners, who filled in questionnaires which were compared to their DTA 4 assessments. What happened to the other 105 dogs? Were the other 105 (almost half of the sample) dogs nice ones or were they vicious ones who were killed? Considering that thousands of dogs' lives depend on it, this is a very inadequate sample.
THE INTRODUCTION TO THE TEST
The first page that I have read (p.35) is problematic. The distinction between the supposedly less useful “Behaviour” tests and the supposedly more scientific “Temperament tests” is far-fetched and not clearly drawn. The distinctions sound scientific when they are not. How consistent across situations or how consistent over time are these “Temperament tests”? The tests are done on the second day that a dog is in the “shelter” and it is claimed that the eight tests “can predict how a dog will behave in many situations.” Judging from Cheech, who escaped his fate, and the four Nanaimo dogs, who all passed assessments and were killed, they are not consistent over time and therefore do not predict how a dog will behave in many situations. Many more dogs are killed after having passed the initial assessment, such as the recent case of Delta SPCA's Tina. DTA4 more resembles the behaviour tests, which are like a snapshot of a particular point in time, which are claimed to be inferior.
ACCURACY WAS MEASURED USING THE THREE CRITERIA BELOW
Predictability(how often the dogs aggressive on the DTA 4 aggressive in their new homes?) Predictability ranged from 50 to 100%
The predictability across the tests ranged from fair to excellent. The claim is made that no non-aggressive dogs are mistakenly profiled to be aggressive.
Specificity (how often dogs that were not aggressive in their new homes not aggressive in DTA 4?
Specificity ranged from 70 to 100% predictive value for aggression. This means that dogs described by their new owners as not aggressive showed no aggression on the DTA 4. This sample is so small here that it is difficult to make definitive statements.
Sensitivity (How often are dogs that are aggressive in their new homes not aggressive in DTA4?)
These sensitivity figures are disappointing. A maximum of 44%.
Sadly this test misses lots of dogs who are aggressive in their new homes.
The test protocols start on p. 69. The tests take 15 minutes. The fate of a dog comes down to a short, quick and easy test which can result in the killing of the animal. The test can take one minute for a dog who reacts badly to being stared at. After each test, the assessor fills in a worksheet, which itself would take hours to review, as there are probably as many interpretations of behaviour as there are assessors.
TEST 1 Stare at the dog from outside the kennel, hands at you side or behind your back
The dog must wonder what these dumb people are up to! Only if the dog shows no aggression can the examiner proceed to Test 2. This test is predictive of aggression. In 100% of cases the dogs are highly aggressive and cannot be rehomed if they show signs of aggression so there is no point in continuing with the assessment, according to the DTA course. To do this on the second day in the shelter is unfair. They have seen and experienced many strange things for them and then to presented with assessors doing strange things must be terrifying for many dogs. Would you, the reader, or your own dog feel comfortable being stared at?
TEST 1 can be the end of the line for dog- in just one minute. This could happen after Test 2 as well- or any subsequent test.
TEST 2 Enter kennel and ignore dog. You can’t look directly at the dog or talk to him.
At least you don't have to stare some more!
TEST 3 Groom and handle the dog.
Many dogs have never been groomed before. I wonder how they cope. Many matted dogs could be uncomfortable.
TEST 4 Show and jangle the leash for 30 seconds
TEST 5 Put the leash on the dog. The tester puts the collar on and sees how the dog reacts
TEST 6 Walk the dog outdoors for a 2 minute walk
All the three above tests can be difficult for many rescue dogs as many of them have no experience at all on leashes. A lot of them have been chained, tethered or running free for years. Many have never been on a walk before. How will they fare? Leash walking is a learned skill.
TEST 7 Watch the dog for 2 minutes and then play tug of war for 2-3 minutes
Playing tug of war with an unknown dog is a test? Not every dog knows the unwritten rules of a tug-of-war
TEST 8 Meet other dogs – Walk the dog by other kennelled dogs. Assess the threshold for aggression
Some dogs have experience with other dogs and others don't.
ASSESSMENT OF THE DOGS
When the form is filled in, the dog is placed in one of four categories- red, orange, yellow or green. Green is the top category for dogs likely to be rehomed quickly.
Each colour is discussed used as a determinant for assessing suitability for adoption, with greens and yellows being better candidates for adoption than the orange or red dogs.
I feel that the method of scoring is somewhat unfair to a dog. - 2 greens scores, 2 yellows and 1 orange, gets the dog rated orange, which can eventually be a death sentence, just as it was with the Nanaimo Four. Of course a red score will usually be the end, and the future of an orange dog is highly uncertain.
Writing all these scores on a fancy-looking form does not constitute science. Results depend on so many factors that it can be totally unreliable. So many assessors are incapable of making the judgments demanded or cannot decide just where to put a particular animal on the scale. Some of them are scared of big dogs. Some dogs react to different people in a very different way. The training course offered by the SPCA is much too short to ensure that the test is properly conducted, even assuming that something meaningful can be learned .
SHELTER REHABILITATION
Under the section “Shelter rehabilitation” it is stated that the BC SPCA protocols for dealing with aggression, either directed towards people or other dogs, are yet to be developed and validated. The most aggressive response presented by a dog during testing is the one that the test uses. If a dog reacts badly to one other dog, or flunks one test, it can mean death or a problematic future. If the dog fears men and reacts aggressively towards them, he will fare badly on the test.
In my opinion, to pretend that DTA 4 is science does an injustice to every real scientist who ever lived. The criteria of objectivity are impossible to meet. 15 minute observations serve our dogs badly. They deserve better. Personally I believe that close observation would be much fairer on a dog. They need time, good food and a chance to settle and relax before we condemn them to death. So many of them have come from awful backgrounds and they need every break from their human guardians.
Let's consider our dogs with compassion and watchful vigilance instead of putting them through tests on their second day in the shelter!