A Review of the California Department of Disability Services: The December, 2005 Report
(Click on graph to make larger)
Introduction
The Department of Disability Services (DDS) has published their quarterly report (DDS, 2006). In that report the numbers of service categories the DDS provides services for are listed. These reports are issued once every quarter. It has become habit for various persons to review these reports for autism and to chart the number of supposed new cases and compare to the number of supposed new cases at the same time, the last year (Rollens, 2006). Various blogs have also featured this topic at different points (Autism Diva, 2005)
The purpose of the current review is to provide a graphical display of these number and justification for this presentation. It is also to provide criticism for the way these numbers have been displayed in the past. It is finally, to provide a discussion for what these numbers indicate.
Analysis
Figure 1. Shows the prevalence per 10,000 for ages 3-5 according to the DDS. The trend is an increase. This is relevant as it shows that the numbers of autistic children ages 3-5 is still increase according the DDS. This was calculated by comparing the number of children ages 3-5 receiving services for autism from the DDS to the 2000 census for California for ages 0-4. This does not match up perfectly with the DDS group 0f 3-5, but is the best that can be done for the moment. However, any variation in this would not alter the fact that the numbers of 3-5 years old receiving services for autism are still on the rise, both between quarters and years.
Figure 2. Shows the change in the increase from the past quarter from December, 2002-December, 2005. The pattern here is primarily one of instability. There is no immediately recognizable trend, nor does this appear to be a cyclical pattern. I have added an additional line to show the mean, (Mean = 134; Standard Deviation = 50).
Criticisms
The DDS data have been presented and analyzed in a manner that I argue, is erroneous. It has been the practice of some to use the total data presented in these reports as opposed to the data specific to the youngest children ages 3-5 (Rollens, 2006). I will note that some data exists for autism for ages 0-3 in the DDS reports, but very few children seem to receive services at that age.
The problem in using the total data is that only a portion constitutes the 3-5 age groups. When one presents the total data, they are also presenting on adults receiving services. This is an inaccurate practice. Also, some, such as (Rollens, 2006) analyze this as increase by year. This is unfortunate as is glosses over the instability that exists between quarters within a year. By missing this, some have failed to observe that the observation of a slow, downward trend is inaccurate. In addition the claim by Rollens (2006) that change in numbers are new intakes explicitly violate the DDS recommendations that the change between periods, not be counted as new cases (DDS, 2005).
Others have informally asked that the 6-9 ages groups be analyzed as this group may receive booster shots and regress. I criticize this; because to meet criteria for Autistic Disorder (American Psychiatric Society, 1994) the child’s must have shown the same traits by age 3. This means we can diagnose a person who is older than 3, if there history suggest that these traits were present by that time. In fact, the only spectrum category that will allow for this is Childhood Disintegrative Disorder (CDD), in which the child has typical development for the first two years following birth followed by a regression no later than age 10 (APA, 1994). CDD, has a stable prevalence rate at .2 per 10,000 (Fombonne, 2003).
The concept that a child could regress (possibly following a booster shot) anecdotally, does seem to occur. However, such children I have hard of, had been diagnosed as meeting criteria for Autistic Disorder. The concept that a child could suddenly turn autistic from a booster shot has no precedent in the literature. I would guess that such a regression would merit the categorization of CDD (as they would be excluded from any other diagnostic label), yet that prevalence rate is stable.
Discussion
By calculating the autism prevalence post year 2000, from data taken from (Fombonne, 2003) I find a mean prevalence of 23 per 10,000. To calculate prevalence based on the most recent DDS quarterly report (December 2005) and the 2000 census projections for ages 0-4, I find a prevalence of 22 per 10,000. This is an insignificant difference. This implies that the DDS who only provide services for those who meet criteria for Autistic Disorder seem to be servicing the number of children in the 3-5 age range we would expect. It is important to note that this is not the case during other recent quarters where we should have expected that rate as well.
It is quite possible that this increase in children receiving services will continue to go up and will exceed the mean prevalence rate. This is, because the DDS are gatekeepers for services in California and individuals must meet criteria for Autistic Disorder, if they will receive services. It is quite possible who meet criteria for PDD-NOS are deliberately mis-diagnosed by DDS diagnosticians so that they are eligible to receive services.
To conclude; Rollens (2006) asks “For those who continue to believe in the fantasy that we have NOT experienced an epidemic of autism, might I ask one simple question: If the incidence of autism hasn't increased dramatically over the past 20 plus years, then where are all the adults with full syndrome autism? Surely if there is no real increase then we should see roughly the same number of adults with autism as we do children. I am sure it is about as easy today, as it has been in the past, to somehow misplace or not recognize thousands of adults with full syndrome autism...about as easy as missing a train wreck. Sorry but no Ph.D. or MD required to recognize either one.)”
I note that the above argument has an illogical statement by stating that any counter statement is “fantasy” before the argument is ever given. This technique provides ridicule in an attempt to distract from any legitimate argument that may be given. To answer the question itself one can review the DDS data and observe an increase in the older age groups as well. While this may be in part provided by movements into California, I note that the 3-5 year old bracket is also not controlled for such. This means there may be some parity between them in this regard. These adults were very likely receiving services under another category of the DDS. I propose that many persons who are autistic are still receiving services under such categories. This is logical considering that before 1980 autism was not a formal category in the DSM and that in 1994 the diagnostic criteria were altered. This is analogous to what is seen with the IDEA numbers, as has been observed (Laidler, 2005).
To equate Autistic Disorder to a train wreck in terms of being unmistakable is a false equation and is an error of logic. For such equations, even a single counter example can show the inaccuracy of the argument. I have personally helped diagnose a teenage boy with Autistic Disorder, who was formerly diagnosed only with Mental Retardation. Perhaps, even the MDs and PhDs are not always the diagnosticians we would wish for. How such as these would be expected to make a diagnosis that was not yet a formalized category would also be a question that merits an answer.
I would expect that portion of these older persons would be re-diagnosed into the autism category in spite of the earlier category they would have been placed in. I note, that an increase of older persons is in fact noted.
Notes: A special thanks to Ginger at Adventures in Autism for generously providing a portion of the data used in this analysis. Ginger’s blog can be found at http://adventuresinautism.blogspot.com/
References
American Psychiatric Association. (1994). Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition. Washington, DC: American Psychiatric Association; 1994.
Autism Diva. Monday, July 11, 2005. http://autismdiva.blogspot.com/ Accessed October 6, 2005Autism Diva. Friday, April 22, 2005. http://autismdiva.blogspot.com/ Accessed October 6, 2005
Department of Disability Services (2006). Quarterly Client Characteristics Reports.
http://www.dds.ca.gov/FactsStats/quarterly.cfm
Accessed Friday January 13, 2005
Department of Disability Services (2005). Data Interpretation Considerations and Limitations.http://www.dds.ca.gov/FactsStats/pdf/CDER_QtrlyReport_
Consideration_Limitations.pdf
Accessed Friday January 13, 2005
Fombonne, E. (2003). Epidemiological surveys of autism and other pervasive developmental disorders: an update. Journal of Autism and Developmental Disorders. 33, 365-382.
Laidler, J. (2005). US Department of Education data on "autism" are not reliable for tracking autism prevalence. Pediatrics, 116 (1), 120-124.
Rollens, R. (2006) California Reports: New Autism Cases at 4 Year Low. Schafer Autism Report, 10 (7). Thursday, January 12, 2006, http://www.sarnet.org/
Accessed Accessed Friday January 13, 2005
State of California, Department of Finance, Population Projections by Race/Ethnicity, Gender and Age for California and Its Counties 2000-2050, Sacramento, California, May 2004.
Taylor, G. (2006). Adventures in Autism. California Autism Numbers 4th Quarter 2005
Thursday Januray 12, 2006 http://tinyurl.com/88b6z.
Accessed Friday January 13, 2006
posted by Interverbal at 8:21 PM
29 Comments:
*Applause*
Nice work. You are a true asset to this discussion.
What Cubbins said.
Really excellent work here: Concise, succinct, polite and accurate.
*more applause*
Thanks you sirs
Rollens' analysis of DDS data is also erroneous because he disregards politics as a factor affecting rates of increase of the numbers of individuals receiving services. If there is, in fact, any reduction in the rate of increase of autistic individuals receiving services, that reduction coincides with the July 2003 institution of more restrictive thresholds for qualification for services. Here's what Rollens himself publicized in the October 20, 2004 Schafer Autism Report:
In July 2003 California adopted a new additional "substantial disability" criteria for eligibility into California's developmental services system. Not only must persons with mental retardation, epilepsy, cerebral palsy, and autism be professionally diagnosed (and in the case of autism receive a diagnosis of full syndrome DSM IV autism, not including any other autism spectrum disorder such as PDD, NOS, or Asperger's Syndrome), they now must demonstrate "significant functional limitations in three or more of the following areas of major life activity:"
1. Self-care
2. Receptive and expressive language
3. Learning
4. Mobility
5. Self-direction
6. Capacity for independent living
7. Economic self-sufficiency
Since the implementation of the new law in July 2003 there has been a decrease in the number of new intakes in all four categories of disabilities in California's system. In some categories the decrease in the number of new intakes has been substantial.
A reduction in the rate of DDS intakes is part of the bureaucratic program. This is one reason why statistics from social services agencies subject to political influence are useless for epidemiology.
Hi Kathleen,
Really good point.
The politics of this issue were something I just brushed on in this post.
I got into why the DDS reports are not epidemiology here:
http://interverbal.blogspot.com/2005/10/reviewing-autism-prevalenc_112863776623559236.html
Well done John. It never fails to impress me how unfailingly polite you are, even when disagreeing with someone. Folks - here's a role model:)
Cheers
Al
In fact, Kathleen's point is probably the explanation for why the quarterly data represent, as interverbal noted, an unstable process (a stable Poisson process with a mean of 134 should have a standard deviation of only 12 rather than 50; thus the data are "overdispersed").
Almost all the overdispersion can be accounted for by the extreme values found in 2003; the peak occurs right before the new criteria went into effect, suggesting a scramble to get as many clients as possible qualified before the change, and there's a sudden crash as the new criteria go into effect; part of this is probably the result of cases being rushed earlier on.
Month-to-month variations could also result from anything that creates a bottleneck in the eligibility certification process, such as the absence of key employees, whether for vacations, sick leave, compassionate leave, sudden promotion, etc. Life happens, people take off, and there's not always someone immediately available to pick up the slack. Think of how many times you've left your post for a short break, and come back to a desk full of work that you wish someone else could have done for you! Take a look at that graph and see how intake rates don't change that much between June and September. I'm inclined to think that that has more to do with summer vacations than it has to do with autism. ebohlman points out the steep rise in intakes over the winter of 2002-2003, before more restrictive eligibility criteria went into effect; once you look at that, check out what happens over the Christmas season the following two years -- not a whole lot.
Thank you Alyric,
I have had the advantage of really good role models in this regard e.g.(Michelle).
Hi ebohlman,
For a lack of more eloquence; I strongly agree with your analysis.
I think you nailed it on the head.
Speaking of means and standard deviations you may be interested in this post:
http://interverbal.blogspot.com/2005/10/reviewing-autism-prevalence_07.html
Hi Kathleen,
I agree that the analysis you offered.
In the first draft of this post, I had more data interpretation and discussion. I removed these parts, to keep the focus on just presenting the data.
I would like to put these things in their own post down the road. I am also re-comparing the MR rate according to the DDS, to what is known from the epidemiology and racial percentages.
I had some of this graphed in post #2 on this blog, but I recently found a methods flaw, so I am having to double check some of my work
It might take me a month or so to get this all settled.
Thanks Interverbal. Autism Diva tried to leave some applause yesterday but couldn't get the comment posted, some glitch...
Autism Diva agrees with Alyric that you are unfailingly polite. Autism Diva thinks you are being wasted on the world of ABA, but then, she's not that polite. :-)
Once again Kathleen brings up in important point. In broad terms, there are two possible reasons why the counts will vary from month to month:
1) Common causes of variation. These are the ordinary influences that are always present in the process that generates the counts. The vacations, leaves, variable backlogs and the like that Kathleen mentioned are examples of common causes. Common causes cause the results to fluctuate randomly around the mean in a predictable fashion. You can think of them as creating "random noise." A process is called stable if common causes are the only source of variation.
2) Special causes of variation. These are influences that are not normally present in the process; the appearance of a special cause means that the process has somehow changed. Examples would be a change in the eligibility criterion or an actual increase or decrease in the underlying incidence of autism. Special causes result in unpredictable variation.
In a process that generates count data like we have here, a number of things will be true if the process is stable:
1) The standard deviation of the counts will be approximately equal to the square root of the mean.
2) All, or almost all, the counts will lie within three square-root intervals of the mean.
3) Approximately half the counts will be below the mean and half above.
4) There will be no "runs" of 7 or more consecutive increasing or decreasing counts.
5) There will be no runs of 7 or more consecutive counts below the mean, or 7 or more consecutive counts above the mean (this is really just a formalization of #3).
The data we have violate the first two conditions, so we can conclude that the process isn't stable; something changed along the way. Note that the data do comply with the last three conditions, which would likely have been violated if the special cause was a long-term shift in the underlying incidence. Thus it seems very likely that the special cause was in fact the change in criteria.
Hi Diva,
Thanks for the words and the link.
Ebohlman,
Wow, thank you for that additional analytic information. It kind of puts things in perspective for what is going on with that graph.
Interverbal,
Have you, or could you or would you, track the intakes of a couple other categories or the total to see if there has been a slowing down in the rate of intakes period at the DDS. Would knowing that be helpful? MR is complicated because they have different levels of MR and MR is more likely to be comorbid with epilepsy or CP, I think, than autism is.
Also, have you read
The Changing Prevalence of Autism in California Lisa by A. Croen, et al?
Hi Diva,
I did a review of those data in brief, at this time, last year, but I found a methods flaw in my analysis (this is when I was first learning, after all.)
I do plan to review those data (two projects down the line).
I have not read Croen et al., yet, but it is on the official list for tonight.
In other news, I talked to a data person at the DDS who very generously gave me access to the 3-5 age, autism data from 1992 onwards.
We also had an interesting discussion on data usage.
J,
I got the 2001 and 2002 numbers emailed to me, but I have not looked at them yet. They are posted hereand here.
I will finally get the chance to start looking at this tonight.
Thanks,
Ginger
HA!
Just saw the note about getting the data back to '92. I am to slow.
So I read this tonight and I have several questions and a couple of comments, but since you have the numbers going back to '92 I will wait until you have the chance to run this for the whole data set.
What I am more interested in, rather than just are they rising or falling, is to find out if there is a drastic change (or any change) in in the tragectory of the graph starting around 2003 that continues on (there for not completely attributable to the change in inclusion criteria).
If it would be more helpful to have my comments before hand so that you can address them in your next post rather than just going back and forth in the comments section, let me know and I will try to make time for it this week.
Ok, one comment, in determining the prevalence, I am assuming that you are using the same 2000 population number for each year if I am reading right. Because we know that the population is growing each year, will you not always see an increase in the prevalence if you are using denominator? Is there any estimate out there of how much the state population is increasing per year that can be used to estimate a more accurate year to year population number?
Any way, I know that this is just one more case of 'the best we have to go on'.
Thanks for doing all the work on this.
Also, when you do the next round, could you also show a year to year graph? I did both quartely and yearly graphs because it is good to see the quarterly graph, but the yearly drowns out the "Betty missed all of June because of maternity leave, paper piling up confounder" noise that has been commented on.
also... FYI
Wanted to make sure you knew that in second quarter 2002 they dumped a bunch of "lost" cases of autism (and other disorders) into the numbers, so it will be artificially inflated. I am not sure how to allow for this, but you may want to get in touch with the office and see what numbers they suggest to use for this quarter. I spent last week trying to figure out what the hell was going on in that quarter and wanted to make sure you didn't waste time with that too.
Another issue/question: Is there any possible way to find out from your contact if they track, in any way, the number of clients they drop per quarter or per year?
Jonathan,
I made a first pass at this and I wante to know if you want me to just post it here or email it to you. It is kinda long for the comments section.
I thought you might just want to have it to try to account for the points I make in your next pass at it going back to 1992 rather than filling up the comments section.
Email me at ginger@adventuresinautism.com
Hi Ginger,
I used the 2000 census data. They used to have projections, but removed them. I have located another source, and this should be resolved on the next graphs
I would like your feedback now, so I can scour it for merit and possibly improve my analysis before next time.
Thank you for the heads up on the data in 2002.
I would have to know what you meant by "yearly" for doing these graphs. I disagree with the logic that has been used for some of the yearly graphs I have seen.
Hi Hollywoodjaded,
The attrition rate?
Hmmm.... Well, all I can do is ask.
I will check this out.
Yes, Jonathan, the attrition rate -- as to whether or not they even track it and if so, then it would be great if you could get those numbers. It would be much appreciated. Thank-you.
Hi, Jonathan: I would like to contact you off-list if that's OK with you. I have some updated information in re: the Reg Ctrs -- tracking of numbers and administration matters, to name a few. ~HJ
Hi Hollywoodjaded,
Definately,
katahajime74(at)hotmail(dot)com
Post a Comment
<< Home