Monday, October 23, 2006

A Review of Mr. Rollens’ Latest Interpretation of CDDS Data.

Rick Rollens periodically reviews the autism data from the California Department of Disability Services (CDDS). He often issues these interpretations in the Schafer Autism Report.

Mr. Rollens is not alone in using the CDDS data. Geier & Geier (2006); and Croen, Grether, Hoogstrate, & Selvin (2002) have also made use of these data. Many bloggers have also analyzed these data, including
myself.

This issue also pops up in the journalistic sense from time to
time. David Kirby has made specific claims about a decrease that must occur by 2007. His goose might soon be cooked, unless he shifts his goal posts.

CDDS has noticed this and gone on record mentioning that their data are
not to be used for incidence and prevalence studies as there would be numerous random and systematic errors that are not controlled for in their data,which are really only meant to be an executive database, something to summarize for state legislatures and other interested parties who State funds are being used for more or less.

This is not a problem unique to the CDDS. It also present in the US special education data (Laidler, 2005; and Shattuck, 2006) and pointed on this blog
here, here, here, and here.

In the past I have reviewed Mr. Rollens’
comments. His latest commentary has just come to my attention. He makes several specific interpretations that I think are worth discussion. In this post I will review these claims and talk about fallacies and data errors that were committed within his analysis.

Mr. Rollen’s writes:

“According to data just released by the California Department of Developmental Services (DDS), during the 3rd. Quarter of 2006 (August through September), California added 841 new cases of autism to it's developmental services system, a number that represents the second highest quarterly reported number of new cases in the system's 37 year history.”

That position should be contrasted to his position in January of this year which asserted that CDDS autism cases were at a four year low. This is a rapid shift from first to third quarter. This is more problematic when one considers his arguments at that time that CDDS autism cases were in general, headed down.

However, a problem emerges here in that Mr. Rollen’s uses all the autism data across ages, not merely the youngest children ages 3-5 which the average age of diagnosis (Jick, Beach, & Kaye, 2006). Autism must be fist evidenced by age three, even if diagnosis occurs later (APA, 1994). So, any older persons who enter the CDDS system, must also have met criteria by age 3 and were simply missed, or moved, or were already in the system and receiving services for another
category.


Of course, since the CDDS is service based system, even if someone was diagnosed and entered the system in one quarter it is entirely realistic that because of paper work traffic that they might not be included on any official counts for some time.
This means that there are inherent threats to the accuracy of any attempt to use these data for incidence or prevalence calculation. Specifically a previous analysis on this blog found that the CDDS data were susceptible to all six types of random and systematic statistical
errors.






(click on graph to make larger)

Figure 1. shows what the 3-4 year old CDDS autistic data looks like. Although I give all the data going back to 1992, I calculated the mean of the quarterly reports beginning in the first quarter of 2000 and extending to the Lanterman Act revisions in that took effect in August 2003. I did this to keep the years relatively recent.I repeated this process for
The data following the Lanterman Act revisions which happens to coincide with a supposed downturn in the number of new cases.

The data are (133; 95% Confidence Interval = +/- 31) for the former and (139; 95% Confidence Interval = +/- 29) for the latter. The mean differences are minor indeed and are well within confidence intervals.

Mr. Rollens writes:

“In 1987 there were 2,778 cases in the system, by the end of 2002 the number had increased to 20,377, and today there are 31,853 persons with autism in the system.”

By which he means persons as young as 0 years of age or as old as 80+. But again, when one looks very broadly across age groups, it is difficult to know if a increase is meaningful in terms of people newly developing autism. There is an excellent graph of “prevalence” to be found
here.

Mr. Rollens writes:

“Over 84% of all persons with autism in California's system are between the age of 3 and 21. 88% of the autism population currently live at home.”

Which is of course rather intuitive, many people under 21, do in fact live at home. This might be nearly all of people 18 years of age or younger. However, this is an interesting point for another reason. In informal conversation, I report often hearing that some parent would be forced to give their child up due to their autism. And that if a child is not cured this is the inevitable result. If these data have any validity, then apparently that is false.

Mr. Rollens writes:

“INCREASING RATE SLOWS DRAMATICALLY

In 1999, DDS released it's now famous and historic autism caseload report that documented a 273% increase in the number of new cases of autism entering California's developmental services system from 1987 through 1998.

In 2003, DDS followed up with an updated report that documented a 97% increase in the autism caseload over the 48 month period from December 1998 through December 2002.

ACCORDING TO DDS, DURING THE MOST RECENT 45 MONTH REPORTING PERIOD FROM JANUARY 2003 THROUGH SEPTEMBER 2006, THERE HAS BEEN A 50% INCREASE IN THE AUTISM CASELOAD. THE RATE OF INCREASE HAS DECLINED BY NEARLY HALF OVER THE PREVIOUS LIKE REPORTING PERIOD. THE RATE OF INCREASE HAS SLOWED BY CLOSE TO HALF FROM 97% TO 50% DURING THE PAST 45 MONTHS.”

Simply said; Mr. Rollen’s analysis is discrepant from the data in the 3-5 age cohort.

Mr. Rollens writes:

“while the younger, 3-9 year old cohort reflects the substantial, declining rate of increase as noted above.”

Again, this is not what the data show. Returning to Figure 1., the pattern if anything, is one of instability.

Mr. Rollens writes:

“Reasons for this phenomenon could include the lessening burden of mercury in vaccines slowing the numbers of new young children entering the system, a tightening of eligibility criteria that took effect in July 2003 (see last paragraph), and Regional Centers responding to the pressure to qualify more older persons with higher functioning autism spectrum conditions.”

An yet a third time, the 3-5 year old cohort is not showing a decrease. This seemingly invalidates any theories for a cause of such. I am reminded of an excellent comment from Gernsbacher, Dawson, and Goldsmith (2005) "Epidemics solicit causes; false epidemics solicit false causes." With apologies to the authors I would like to offer a corollary: The end of false epidemics solicit causes; the false ends of false epidemics solicit false causes.

Mr. Rollens writes:

“One thing is for sure, the hidden hordes of adults with autism that needs to be accounted for in order to discredit the existence of an autism epidemic and an increasing incidence of autism have yet to come forward or be discovered.”

Mr. Rollens is partially correct. He is right in the sense that we have not found a “horde”, but
oft and again we find older person 40+ years of age who we now diagnose as autistic.

Also, I am concerned how strongly the
burden is now shifted to those who would ask for better proof to accept an “epidemic of autism”. This is in fact, shifting the burden of proof.

All this said, I would argue that evidence is
amassing that would refute the theory of an thimerosal based epidemic. I note the equivelance of of the prevalence rate of autism for the US (Bertrand, Mars, Boyle, Bove, Yeargin-Allsop, & Decoufle, 2001), the UK (Chakrabarti & Fombonne, 2005), Canada (Fombonne, Zakarian, Bennett, Meng, & McLean-Heywood, 2006), and the Faroe Islands (Ellefsen, Kampmann, Billstedt, Gillberg, & Gillberg, 2006), this is despite the differences of vaccination schedule and type in those nations.

Also, Chakrabarti & Fombonne (2005) showed that 2 cohorts were equivalent; one of the cohorts was born the early 1990s and the other was born in the mid 1990s. This is significant as these groups should have split the difference we see in Figure 1., and yet they were equivalent. In the past I have gone so far as to collate a list of research which does not support an autism epidemic.

At the close of my last response to Mr. Rollens I posed a question. I will pose it again now: What is the validity of using the California DDS data when the CDDS has stated that they should not be used for this purpose and when they do not resemble other existing epidemiology?

References

American Psychiatric Association. (1980). Diagnostic and Statistical Manual ofMental Disorders, Third Edition. Washington, DC: American Psychiatric Association; 1980.

American Psychiatric Association. (1987). Diagnostic and StatisticalManual ofMental Disorders, Third Edition, Revised. Washington, DC: American Psychiatric Association; 1980.

American Psychiatric Association. (1994). Diagnostic and StatisticalManual ofMental Disorders, Fourth Edition. Washington, DC:American Psychiatric Association; 1994.

American Psychiatric Association. (2000). Diagnostic and Statistical Manual ofMental Disorders, Fourth Edition, Text Revision. Washington, DC:American Psychiatric Association; 1994.

Bertrand, J., Mars, A., Boyle, C., Bove, F., Yeargin-Allsop, M., & Decoufle, P. (2001). Prevalence of autism in a United States population: the Brick Township, New Jersey, investigation. Pediatrics, 108, 1155-161.

Chakrabarti, S., & Fombonne, E. (2001). Pervasive developmental disorders in preschool children. Journal of the American Medical Association, 285,3093-3099.

Chakrabarti, S., Fombonne, E., (2005). Pervasive developmental disorders in preschool children: confirmation of high prevalence. American Journal ofPsychiatry, 162(6), 1133-1141.

Croen, L. A., Grether, J. K., Hoogstrate, J. and Selvin, S. (2002). The Changing Prevalence of Autism in California. Journal of Autism and Developmental Disorders. 32, (3), 207-215.


Department of Developmental Services (2006). Quarterly Client Characteristics Reports.
http://www.dds.ca.gov/FactsStats/quarterly.cfmAccessed Friday October 20, 2005.

Department of Developmental Services (2005). Data Interpretation Considerations andLimitations.http://www.dds.ca.gov/FactsStats/pdf/CDER_QtrlyReport_
Consideration_ Limitations.pdfAccessed Friday January 13, 2005.

Ellefsen, A., Kampmann, H., Billstedt, E., Gillberg, I. C., Gillberg, C. (2006).
Autism in the Faroe Islands. An Epidemiological Study. Journal of Autism and Developmental Disorders. [Electronically published ahead of print]

Fombonne, E. (2002). Prevalence of childhood disintegrative disorder (CDD). Autism 6, 2, 147-155.

Fombonne, E. (2003). Epidemiological surveys of autism and other pervasive developmental disorders: an update. Journal of Autism and Developmental Disorders. 33, 365-382.

Fombonne, E. (2001). Is there an epidemic of autism? Pediatrics.Vol 107 (2), 411-412.

Fombonne, E., Zakarian, R., Bennett, A., Meng, L., McLean-Heywood, D. (2006). Pervasive developmental disorders in Montreal, Quebec, Canada: Prevalence and links with immunizations. Pediatrics. 118(1) 139-150.

Friis, R. H., Seller, T. A. (2004). Epidemiology for public health practice, 3rd ed. Sundbury, MA: Jones and Bartlett Publishers.

Gernsbacher, M.A., Dawson, M, & Goldsmith, H. H. (2005).Three reasons not to believe in an autism epidemic. Current directions in psychological science, 14 (2), 55-58.Honda, H., Shimizu,

Y., Imai, M., & Nitto, Y. (2005). Cumulative incidence of childhood autism: a total population study of better accuracy and precision. Developmental Medicine And Child Neurology. 47(1), 10-8.

Jick H, Beach KJ, Kaye JA. Incidence of autism over time.Epidemiology. (2006). Epidemiology, 17(1), 120-121.

Laidler, J. (2005). US Department of Education data on "autism" are not reliable for tracking autism prevalence. Pediatrics, 116 (1), 120-124.

Mandall, D. S., Novak, M. M., Zubritsky, C. D. (2005). Factors associated with age of diagnosis among children with autism spectrum disorders. Pediatrics,Vol 116 (6), 1480-6.

Shattuck, P,T. (2006). The Contribution of Diagnostic Substitution to the Growing Administrative Prevalence of Autism in US Special Education. Pediatrics, (117) 1028-1037.

Schafer Autism Report, 8, 165. Wednesday, October 20, 2004,
http://www.sarnet.org/Accessed Accessed Friday February 4, 2005.

United States Census Bureau. (2006). Personal Communication.





11 Comments:

Blogger Autism Diva said...

Nice work. Thanks, Interverbal.

12:41 PM  
Blogger Joseph said...

Great analysis, very well referenced, Jonathan.

It's particularly interesting that Rollens has shifted from his usual claim that "new cases" (caseload growth) is declining, to the newly developed claim that it's somehow significant that growth expressed as a percentage is declining. It's a new sort of cherry-picking as I see it. If caseload is not declining, let's check caseload growth. If that's not declining, let's check the percentage change. What's next? See if the growth of the growth of the growth is declining?

For one, if caseload (the denominator) is growing, obviously the rate of growth expressed as a percentage of the caseload will decline, provided absolute caseload growth is fairly constant, as it is.

Second, it is impossible for caseload growth expressed as a percentage not to decline eventually. This is not a possibility, but a mathematical certainty.

12:46 PM  
Blogger Do'C said...

Excellent summary Interverbal. It's interesting that the burden of proof for an epidemic does (at times) seem to have shifted to "prove that there isn't one" - I'm glad you pointed that out.

I, for one, acknowledge that it's certainly possible, but where is the scientific evidence of such?

12:47 PM  
Blogger notmercury said...

Wow. That was a thorough review Interverbal. Thanks

I think Rollens is in the unenviable position of thinking he should just shut up about the whole thing and knowing how it will appear if he doesn't comment at all.

He's painted himself into a corner but he's in good company there.

2:22 PM  
Anonymous chattahoochee said...

Once again the aphorism comes to mind: You can't reason a man out of a position he didn't reason himself into. The toxic-cause hypothesis of autism is seductive enough that it doesn't matter to its devotees that there's no data to support it.

8:45 PM  
Anonymous anonimouse said...

Rick Rollens needs there to be an autism epidemic caused by vaccines. He's staked his personal reputation on it (as well as some $$$) and will do whatever he needs to do to ensure the public THINKS that an autism epidemic has happened.

In other words, he's an intentionally lying piece of crap. I know he knows that he's wrong, but continues to promote falsehoods anyway.

9:33 AM  
Blogger Fore Sam said...

The only way you can deny a thimerosal induced epidemic is to show us autistics who never received thimerosal at the rate of 1 in 166. Good luck.

6:00 PM  
Blogger Interverbal said...

Hi Fore Sam,
I can think of a few more ways besides that to refute that theory.

As you can see the current topic is problems with the CDDS. Any comments on the problems discussed in this post?

6:41 PM  
Anonymous Anonymous said...

I wonder if Rollens can be reasoned with to see that his son never had measles induced autism.

12:23 AM  
Anonymous anonimouse said...

Fore Sam,

The next cogent statement you make will be the first. Leave the scientific discussion to real adults with post-grade school thinking skills, kay? Or else go back to the JB Handley Refresher Course On All Autism Is Mercury.

1:10 PM  
Blogger Interverbal said...

A spammer seems to be hitting some of my earlier posts.

All posts must be approved now. Sorry if this causes any delays. Hopefully we will be back to usual once this person goes away.

4:26 PM  

Post a Comment

Links to this post:

Create a Link

<< Home