Comparing Group Design and Single Case Design

One of the claims that I hear from the more scientifically oriented readers or other commenters in the blogosphere is that the way to be really certain of an answer in science is to use some variant of the blinded methodology with random assignment.

My thoughts are “Well…. not exactly”. The group designs are the right tool to assess certain scientific questions, but not others. It really depends on the nature of the question. Below I have created a brief list of some of the differences between single case design and group design research logics. My objective is not to thoroughly explore the differences, but merely to be a brief introduction.

Group Design

-Tests the null hypothesis

-Uses deductive logic

-Uses inferential statistics

-Washes out the individual variance in data

-Effect significance is assessed in objective terms

-Compares the experimental group to the

-Control group

-Uses reliable and valid assessment


Single Case Design

-Uses inductive logic

-Answers the question of to what extent the change in the independent variable caused the change in the dependent variable

-Uses graphical analysis

-Can detect patterns in the data that might otherwise be missed

-Effect significance is determined in a more subjective manner, gray area is possible

-Systematically alters the level/presence of the independent variable

-Uses objective behavioral definitions and inter-rater reliability checks

Education Technique #2: Taking a Break

Starting about the 1st grade (ages 6-7) the general pattern in a classroom is work….. then recess. This often works well for both typically developing student and student with autism. Sometimes however, it just doesn’t shake out. The student does alright for the first little bit, but eventually they start to go down hill. Maybe this happens during a particularly hard activity for the student, or a particularly long one.

One of the ways I think is beneficial to work with this situation is to re-think my approach to breaks. I find that allowing the student frequent small breaks, is often more meaningful in terms of preventing tantrums and maintaining the student’s contentedness, than infrequent large breaks, such as recess.

If a teacher or parent decides that this plan is appropriate in a specific child’s case, then the first step would be to determine when the problem occurs. Is it perhaps that Sue doesn’t like her reading program where she has to speak out loud? Or maybe John starts to head downhill when he has to sit more than five minutes?

If we know when the problems occur, we can make a plan. For a stressful program, brief breaks where the student can get up, walk around a bit, maybe go get a drink of water in the hall, both before and after the program, often can prevent the issue from arising. Sometimes if the aversive program is highly aversive, or if it is really long, breaks can be added within the program itself. Another strategy would be to break up the aversive program into very small chunks and intersperse them into other activities.

Dealing with a longevity based issue is usually easier. If you know that John has issues if he has to work for more than 5 minutes. Then the solution is usually is to pick a point a bit before the 5 minutes and give the student a break. In this case I would probably choose 4 minutes. But if that didn’t work, then I might bump it down to three minutes.

I might deliver the above non-contingently. In other words no matter what John was doing when the three minutes were up, I would give him the break. Or I might make it contingent. I would start the timer only when John was working. I would pause the timer if he went off task, and restart it when he went back to work.

There is a third option and this is the one I typically use. It is also the easiest to use, but it involves careful monitoring and awareness of how the student is holding up. I informally keep track of time. I make sure that roughly every five minutes John gets a break. However, I observe if John is showing signs of fatigue, loosing interest, or stress. When I observe this, I ask for one more correct response, and when I get it I offer the break. But I would be careful to always get a correct response before I give the break.

Using non-contingent breaks are usually done if the student has a sever tantrums, or if the break program is first starting out, or if the teacher does not know much about the student yet. Using contingent formally timed breaks are useful when the student is doing independent seat work. If a teacher is roaming between several students and can’t give total attention to the student. The third option works very well for 1:1 work, where the teacher is very familiar with the student.

What A Beard Can Teach About Autism

The fallacy of the beard (fallacy of the spectrum) is to claim that because two points exist on a continuum, that they are indistinguishable, due to the fact that at some point they both blend. The famous example is that of a man with a beard is indistinguishable from a man with a clean shave, because it can be hard to tell at some point what exactly is still a clean shave and what is a beard.

With that in mind I like the reader to consider autism. Persons meeting criteria for Autistic Disorder seem distinct from persons who are typically developing. Many of us, on different sides of the debate would go so far as to consider autism and typical development, entities in and of themselves.

However, the criteria for autism and the categories have not been static. These have changed and expanded over the years. One of the things we have learned from this is that there are many people who fit some of the criteria for autism, but are in other ways more typically developing.

It seems that while autism is distinct from typical development, it is still part of the same continuum. This of course is no deep revelation at all. Most of us recognize that autism is a spectrum. And that many behavior are in fact shared, even if they differ is frequency, topography, and function.

What I would like to argue that might be a bit unusual is that we should not place Autism and typical development on one single continuum. I would argue that these things exist as smaller distinct behaviors on hundreds or maybe even thousands of continuums. I would argue that while the general patterns of autistics or typically developing persons might be similar across these continuums that they are inevitably individualized and distinct to the person.

Making Fun of Homeopaths

The 72nd Skeptic's Circle is up. The circle is mostly focused on Homeopathy this time.
It makes an informative read about alternative medicine though.

Do Old Teachers Claim an Epidemic

One of the more interesting claims from our friends in the biomedical advocacy camp is “That if one talks to a veteran special education teacher then they will tell you that the rise in autism is real and not due to changing criteria”.

I certainly have seen this claim a number of times and I suspect that many of my readers will have as well. It seems fair to put this to a test. After all, the way the point is usually written tries to reflect the opinion of most/all the special educators and not just one or two special educators.

Recently I attended a meeting with 20 or so special education teachers. A total of 6 of them had 25+ or more years of experience. The opportunity as not lost on me and I took a moment to ask: “Is the prevalence of autism increasing, decreasing, or staying static and why do you think this is the case?” The answers are summarized below.

Teacher 1: The prevalence seems to be increasing. She had no ideas why, and didn’t care to speculate.

Teacher 2: The prevalence is truly increasing. Unspecified environmental toxins are most likely to blame

Teacher 3: Prevalence seems to be increasing. No ideas why, claimed not be very familiar with the issues. Didn’t want to guess.

Teacher 4: Prevalence is static. Political issues related to special education law, allow parents to choose a more desirable autism label compared to what the testing says. Claimed she had seen it before with ADHD in the late 80s. She also mentioned that placement is a IEP category does not constitute diagnosis.

Teacher 5: Prevalence is static. Better diagnostics and broader definitions are the cause. Claimed that no one heard of Asperger’s Disorder before 5 years or so ago.

Teacher 6: Prevalence is increasing. No ideas why and didn’t care to speculate.

So the total is 4 who didn’t know why and/or refused to try to guess.
1 person who thought that the increase was due to IEP issues.
1 person who thought that the increase was due to better diagnostics and definitions.
And 1 person who thought that environmental toxins were the cause.

So, what does all this prove? Well, very little to be honest. This type of informal survey can tell us almost nothing about popularity or what the majority veteran special educators think. The only thing is proves, is that bowling up to a special education teacher and asking if the autism increase is real or not, is not for sure, going to be met with a certain answer. This is contrary to claim or myth as it is usually given. The morale here is: “Don’t buy it!”

Education Technique #1

This is the first installment of a series I will run every Saturday through November. The series will be on educational techniques that have used in the past.

Education Technique #1: High Probability Response

I am philosophically committed to trying to minimize the use of exclusionary or aversive techniques to the extent reasonable in a child’s life. I offer this technique as an alternative to “time-out”.

The use of a high probability response uses the idea of behavioral momentum. This is the general observation that the types of behavior we engage in tends to continue in the same direction for a time. Off task or inappropriate behavior tends to precede more off task behaviors, and on task behavior tends to precede more on task behavior.

The technique here, involves the attempt to disrupt the chain or hierarchy of inappropriate behaviors and start a new chain of appropriate behaviors. The teacher/parent would deliver rapid instructions using a lesson or modality the child has already shown mastery of. The moment the child in only engaging in appropriate behavior, they are redirected back to the task. Skillful users of this technique can usually have a child back into the regular activity in less than a minute, although that is not always the case.

The activity should be something the child has mastered and that can be done quickly. But it should not be a highly preferred activity. It not supposed to be reinforcing or soothing, or aversive for that matter; it is supposed to be neutral. The idea is redirection. The idea is catch the child as they head “downhill” in terms of behavior and quickly build up some momentum in a positive direction. And then to get them back into their regular activity as quickly as possible.

My Answer to the Charge

One of the most interesting aspects of being a blogger is the feedback I receive. Take for example my previous post “Are Hub Science Blogs are Unfair”. The feedback of course, showed many different opinions. Some commenters felt that the autism science bloggers, myself included, may be quick to jump on the logical errors or bad science of others, but showed a reluctance to do so for theories that we support. And others thought that our pattern of analysis was reasonable, within the confines of our interests.

Probably the most interesting question that emerged from the comments concerned if what the authors write about should be determined in part or whole by the readership’s interest. I think there are some good reasons to do so, the most obvious reason is, that if you don’t produce posts interesting or relevant to your readership, then they won’t visit and your ideas won’t be promulgated.

However, there are also some reasons not to. The first and foremost, would be that it may not mesh with the purpose of the blog. A second is reason would be that the desired theme has little of worth that can be said about it at the moment.

I think that one of the things I learned when considering the above, is that there is a time and place for writing a requested article. One of the times I have found it appropriate to do so, was when the readers request closely matched my own interest. When the issue involved a strong accusation against a theory I advocate for. And also when answering a challenge.

I have traditionally stayed shy of issues like education and advocating for what I think evidence suggests works. Yet, I find some of commenters asking for just that. My reasons for avoiding these issues include the fact that to address these points is beyond the scope of this blog.

Other issues include the fact that advice is always necessarily very individual, and should be in my view confidential, neither of which is realistic on the internet. I would prefer that readers deal with their local education teams or visit specific advocacy sites to learn what to do.

I think one of the effects of this has been to somewhat isolate this blog, into a more academic niche. This is a place were one could go to dig into meaty issue of autism science and statistics. This also means that my blog may not be right for new parents or those first learning about autism.

I think however, since there is interest in education, that there may be room for an occasional discussion of some educational technique or issue. To this end I am creating a series to be run every Saturday through November. It will begin tomorrow.

Are Hub Science Blogs Unfair

Some helpful anonymous person expressed the idea to me that Autism Hub bloggers are very unfair. We only tackle mercury and ignore other autism science. This person proposes that we should criticize all autism science.

I put up this thread in the hopes that this person would have a place to have his/her/it’s concerns discussed. And that if so, then the so called “autism science” blogs will really be shown to be totally one sided and really un-critical in the nature of their reasoning.

So, how ‘bout it, anyone want to agree/disagree that the hub blogs that focus on science are not critical of bad science that supports what they think?

Any takers?

CDDS Quarterly Report

(CLICK TO MAKE LARGER)

Well its that time again. Let's have a look at the incidence data in the 3-5 age group in the California Deparment of Disability Services. These "incidence" data (the CDDS says you can't use their data to calculate incidence, but who pays attention to little things like that) are from the CDDS system which some people like to say is the "gold standard" in autism epidemiology (its not) and only measures Autistic Disorder (it doesn't). Anyway, let's look at the most recent datum point on the graph.

So..... the expected autism decrease has not appeared. You know it has been a while since thimerosal was removed from childhood vaccines. The only thing really left is flu shots for pregnant mothers. So, when are we going to see the numbers drop? I mean if you take a way the cause, then the numbers should drop right?

Of course this fact might be lost on the advocates of the thimerosal theory. Because when it is Jenny McCarthy vs science....I suspect I know who is going to be seen as more important in woo woo circles.

Goodbye LB/RB

Kev is closing his excellent blog Left Brain/ Right Brain. This is a blow, because not only was LB/RB an interesting and very readable blog, but it was by far the most popular autism blog. The really sad part however is why it is closing. An individual named John Best runs a blog called “Hating Autism” (I refuse to link to it). He has over the course of the last several days written several articles from the imagined pro- quackery viewpoint of Kev’s autistic daughter.

The health of one’s children is obviously an extremely important and emotive issue. Some folks, put great stock or faith is questionable theories like the vaccine etiology of autism or quacky therapies like HBOT. Pointing out that these may not be good things, even when done to the nth degree of politeness does not always go over well. Ask any autism science blogger and they will tell you that sometimes people can’t separate the issue from their personhood. These people get mean, they get incredibly personally insulting.

What seems to have inspired John Best to write these articles, is Kev’s and others, quoting the most outrageous, ridiculous, or horrific statement from certain pro-quackery groups. Quoting these folks is direct proof for what many of us have suggested, that certain treatments are stupid and dangerous. Further, many of the most “outstanding” quotes show that instead of biomed becoming more popular with the masses, it ahs instead become more popular with folks much deeper into other forms of quackery.

John Best seems to argue that because Kev and others do this, sometimes even naming names of the kids exposed to such, we are mocking the children. Well no….. Saying that a treatment given to a child is dangerous or unscientific is not equivalent to mocking the child. Notice how these involve different matters.

Kev, I am sorry you have left us. However, you are doing this to protect your daughter and not only is that a good reason, it is the very best reason. I am proud to have been associated with you.

Allyship: My Story

Returning visitors here know that I usually write objective reviews about science in autism. Today’s post will be quite different. This will be a subjective article about experiences with autism advocacy. Specifically this is my journey. It is the brief tale about learning to be an ally and my experiences with the philosophy of nuerodiversity. In the post I try to define some difficult concepts. Others may disagree with the way I have defined such. I should say that my objective here is not to offend or to speak for the group, but to speak for myself.

My story begins as university Sophomore. A friend talked me into… well actually she made me… be a volunteer in a local special education school to get some “resume building” done. It wouldn’t be so bad she suggested, it was just two hours a day and we would get to work together. My friend was placed in classroom for students with a cognitive impairment. It turns out though that they had enough help there. They said they could use me more with in a pre-school classroom for children with autism. And thus did my journey with autism begin.

I had already worked a little bit in my last semester in High School as a teacher’s aid with students with autism. I didn’t know very much however, except that these folks were an interesting mix of strength and weakness. But now, I was completely fascinated. I spent my nights studying the late, but much missed “Oops Wrong Planet Syndrome” and various related autism websites.

I was also in a tricky time in my own life. I was trying to decide whether I should zig and stick with psychology, or zag into pre- med. To help me reach this decision I elected to take a Philosophy of biology course in my second semester. The course was tremendously hard and tremendously fascinating. It was a revelation. I remember being caught up in Gould’s clear gift for prose, Dawkin’s lucidity, and Oyama’s complexity. And the topics that were drawn into the class were amazing; red shift, cryogenic earth, the central dogmatic fallacy. For the first time in this class I was told what a “strawman” was. Up to this point I had figured that college had nothing to teach, that my undergrad years were here to “fill in the gaps” from High School.

This one class taught me more new information than the sum of my Freshman year. It also left me three world views that have lasted since. It directed me (not on purpose I think) into behavior analysis and into skepticism. It also humbled my newly found behavior analytic bent by having me read “As Nature Made Him”, a powerful reminder that no animal is a tabula rasa (blank slate), and thus directed me into Oyama’s developmental systems theory.

I spent my Junior year learning the basics of behavior analysis and I continued to read books old and new about autism and visit autism based websites. Then late in my Junior year I began to stumble across sites very critical of ABA. Some of these criticisms seemed vacuous at the time, and years later, I am even more certain that they are vacuous. However a few sites made good points. I began to interact and debate with these better sites.

This all went on several years. It was a tremendous education. Suddenly I had to go back, check references, and explain things in a logical manner. If I couldn’t do so, I had to cede the point and modify my worldview in small ways. I honestly think that I understand behavior analysis on the level I do compared to my peers, because I spent so much time debating the basics of it on the internet.

But more than that changed, I met people who didn’t talk about autism just as a collection of behavioral excesses and deficits. The reminded me what psychology often teaches, but sometimes forgets, that normalcy is a statistical matter, not a yardstick to which others must be bent. The autistic persons and their allies, who held these views, were passionate, strongly opinionated, and vocal. But they were also like all groups of people. They were not homogenous. Some argued using science and logic, some argued using ad hominems.

I was turned off. Many of the points weren’t just too tart; they were plain not very well thought out in the first place. I strongly disagreed with a lot of what I read or heard from these folks. Being called an “ally” (no one said “ND” yet) wasn’t something I saw a good thing, at this time. I told myself that “I drink no ones cool-aid”, whether it be behavior analytic or otherwise. I had a real set to with a few such people over certain issues. I came very close a few times to stopping my discussion with them at one point. I think that it was the presence of a few solid science based arguments that kept me around.

In September 2005, I started this blog. Initially most of my work was graphing or discussing the autism epidemiology. My blog was not so much pro ND as it was against the argument for an epidemic. It wasn’t even anti- quackery or pro- skepticism at first. These are changes that happened later.

However, this blog was relatively popular (in a small way) with autistic self advocates and their allies. But even at this time I wouldn’t have strongly identified with them. It was inclusion in the Autism Hub that finally caused the change in my view. The hub became a locus for other excellent science and logic based blogs on autism. Not mercury, Do’C, Prometheus, Autism Diva, Joseph, Caitlin, the gang at LB/RB, and Michelle (Love ‘em or hate, their work and criticisms are science based).

I think I learned that Neurodiversity is partly, but best summed up as remembering the old psychology lesson that the average alone does not = what everyone should be.

I think that I have learned that being an ally is very similar to being a true friend. Meaning that an ally both supports those with differences in terms of maximizing their ability to self- determine and being able to tell them when you think they are wrong, without hesitation.

The Parable of the Shoe

We usually require that children tie their shoes, as to prevent possible tripping. No research needed here in my opinion. Maybe kids who do not tie their shoes don't actually trip more often than kids who do; however, the issue is so petty, that it doesn't seem worth the effort it would take to research it.

But now let's say that I was a concerned advocate. I noticed a massive increase in ankle and knee injuries among students. I notice a trend among kids to wear their shoes untied. I suspect based on my own knowledge and experiences that the two are linked. Being a concerned person I advocate a law be passed that would require kids to have tied shoes. To enforce this law, a small fine can be assessed if a child is caught with untied shoes.

Maybe my observation that when kids have untied shoes they trip more isn't a good enough standard. I have taken a petty issue and made it a more serious one. Now it is time for research.

When the research comes back we find that "yes" lots of kids do have untied shoes, as is fashionable, but the style requires short laces that can not possibly trip the child. Also, there has been an impressive increase in both sandlot and organized soccer, which upon investigation seems to be the cause of the increase in injuries.

What I am hinting at, is that there are all sorts of things that quickly modify a petty issue, into suddenly requiring real research. This includes:

1. The possible danger of treatment

2. The expense of the treatment

3. The time involved in the treatment

4. The required participation of others, in the treatment

5. The legislation of some issue related to the treatment

6. And the claim that an anecdote is sufficient proof for broader acceptance or usage



The last claim in particular is what we see so often in alternative medical circles. Unfortunately, the advice of our myth busting friend Adam Savage to “reject your reality and substitute my own” doesn’t work in real science. When you advertise or give a testimonial about the effect of a treatment, you intrude into the realm of science and epistemology. What you say will be put under scrutiny, to see how well it holds up.

Behavioral Progression Paradox

It has been a while since I have done a behavior analytic post. So, below I will dabble with a concept that really deserves much more than a brief blog entry. I hope that readers will find it at least mildly interesting.

I have previously written about a well known paradox in behavior science. That whenever a reinforcement contingency occurs, a shadow contingency of aversive control is simultaneously established. This theory implies that there can be no purely positive interventions or education.

And this theory is broader than just ABA or some other application of behavior analysis. Anytime a consequence is given (whether appreciated or not) and this consequence affects the rate or likelihood of a behavior, then the above paradox is put into play. There doesn’t have to be behavior analysts and gaggle of therapist, or even someone who knows the first thing about behavior analysis; there just has to be a consequence.

But there is another related paradox, that I wish to discuss. This is what I would call the “progression paradox”.

As any child, including one with autism, begins with relatively few contingencies that are overtly aversive in nature. Work is brief and closely monitored by others. But then, progressively, the number and type of interactions multiplies. If one analyzed what maintains these contingencies, then largely (but not exclusively) they are aversive in nature, they are usually avoidance or loss or avoidance of the presentation of an aversive outcome contingencies. For example, homework must be turned in by a certain day, the alarm clock annoyingly goes off, chores must be done by supper time, friends expect you to be with them at inconvenient times, the computer keyboard has an “K” key that is jammed and extremely irritating, you must correctly discriminate between the voice level that is appropriate in the gym and voice level that is appropriate in the class or home.

And so it goes…… But herein sits the paradox. We expect that education not only be done in a way that minimizes the unpleasant or aversive, but even that it moves a child towards learning to avoid the unpleasant or unfortunate. If this theory is true, then the further a child moves in their education, then the greater the amount of aversive contingencies they contact.

So if this theory is true what does it change? And the answer is “not much”…..
This theory doesn’t change the ethical value many of us place on avoiding aversive contingencies in teaching children to the extent possible. Or for that matter, on education that maximizes independence and development interests.

The only thing this paradox could really alter is the way we talk about aversives. We can’t say that we only use reinforcement. We can only say that we try to minimize aversives.

A Review of the Challenge- Dechallenge- Rechallenge Design

Most of the readers here are probably familiar with testimonials that claim a given treatment was implemented for a young child and then caused drastic improvement. Many of us on the skeptical side of the argument would correctly note that this is a post hoc, ergo propter hoc fallacy and that just because a change was noted after the treatment doesn’t mean the treatment caused the change.

I think it goes it without saying that pointing this out doesn’t always go over well. These issues deal with the core of what different people consider to be acceptable evidence. It also inevitably ties into an extremely emotive issue; their children’s health and wellbeing. Sometimes though, the people the people who have been accused of using such a fallacy do take the criticism to heart. They want to know what they could do to offer proof and be convincing.

I have been asked this several times. Frankly it is a frustrating question for me to answer, because I know the answer itself is unfortunate and unsatisfactory. And the answer is: “Not much”. The very limited list of possibilities includes:

1. Volunteering their child to be a participant in trials of their preferred treatments.

2. Conducting group based research like a double blind, placebo controlled, crossover design.

3. Advocate for studies to be undertaken concerning their preferred treatment.

4. Conducting a well controlled single case design.

The first option is unlikely because such trials are rare and the logistics of the thing make having the opportunity difficult. The second option is exceptionally hard because it requires significant technical expertise and well as considerable time and money. The third option is very indirect and often feels unsatisfying. It takes the ball out of the advocate’s court and places it into the hands of some third party.

This leaves the fourth option. This seems to be the option that many of the caregivers who accept the criticism of the post hoc fallacy, attempt to use. Specifically they attempt to make use of the Challenge- Dechallenge- Rechallenge Design. Said another way, this design has three phases, a treatment phase, followed by a baseline (non- treatment) phase, followed by another treatment phase. The advocates or caregivers claim this is a scientific design, which proves that a treatment was effective for their child.

To such advocates it may look like a good plan. This design is well considered in some mainstream scientific circles; it doesn’t involve the expense and difficulty recruiting for a group design or the detailed knowledge of inferential statistics. The basic mechanics of the design are easy enough to understand. It puts the ball back in the caregiver’s court. And maybe best of all, it seems to provide a direct answer about the effectiveness of a treatment, for someone very dear to the advocate or caregiver.

Unfortunately, a certain cliché is apt here: nothing is ever easy. The treatment- baseline- treatment design, even in the best possible case is a quasi- experimental design. It has reduced validity to answer if the treatment is what caused the improvement for the child. This has nothing to do with the fact that there is only one child. In fact there are several excellent single subject designs that allow us to be quite confident that treatment caused the improvement. Unfortunately, this design isn’t one of them.

What worse is that even though this design when conducted under excellent conditions and high control might have some validity, this still requires years of study, guided practice, and hard work to understand. Single subject has its own quirks and trip falls just like any other type of research from epidemiology to double blind designs. The point is you are not going to be competent in this type of design from reading a book or researching the topic on the internet. They could become so, but it takes years of hard study and practice.

The concerned caregiver or advocates who claim that they administered a treatment, withdrew, it administered it again, are not drawing level with trained researchers who use this design. They don’t have the same safeguards and controls. A good research project isn’t just the basal design, it is the level and appropriateness of the controls in the design. A trained researcher’s design might have:

1. More than one participant being investigated at the same time or in sequence to control for maturation or outside factors.

2. Specific targeted behaviors, with objective definitions, as opposed to someone’s general impression of “wellness” or competency.

3. Appropriate reliable and valid assesment.

4. Inter- observer agreement assessment, to make sure that those who give the assessment or record the behaviors are doing so in a valid and reliable manner.

5. Careful notation of possible confounds that emerge over the conditions.

6. Appropriate graphical analysis to observe trend, level, variability, and possible patterns indicative of additional or outside concerns e.g. (cyclical behavior).

7. Careful control of the context and environment in which the child is exposed to the treatment.

8. Submission and analysis in peer reviewed publications, where ones knowledgeable peers can point out problems, weakness, or graphical shenanigans.

9. Knowledge of what type of questions the design can and can not answer in the first place.

10. Knowledge that the conditions can not be arbitrarily switched, that there are rules for when we can transition to the next condition.

If one doesn’t have at least some of the above and especially 2, 3, 8, 9, and 10 intact, then they have no real design to speak of.

Look at Figure 1. This a graph of a hypothetical treatment- baseline- treatment design. There are only two points of comparison in this design; the first treatment phase to the baseline, and then the baseline to the second treatment phase. We have absolutely no idea what was going on before the first treatment phase implemented. And even if there was a dip in the baseline condition we have no idea if this was an artifact from an outside event or maybe based on a cyclical pattern of behavior. If you only have two points of comparison it is very hard to know.



This is figure 2. This is what happens when someone tries to use a treatment- baseline- treatment design, for a task that involved learning. You get stability where we should see variability between the phases. So, even if the treatment was biomedical and the measure was on something not easily lost like acquisition of language, the negative effect would not be observed during the language tests.

The next time someone says they used a Challenge- Dechallenge- Rechallenge which is a appropriate scientific research design, ask them if they can send you the write up so you can see if they used the same controls the trained scientists do.