A Review of the Parent Designed Report Form

The Parent Designed Report Form is an assessment method designed to record the potential change in autistic child undergoing therapy and specifically methyl-B12 injections. This tool, while embracing some of the qualities of objective assessments does not embrace them all. The threats to the validity and reliability of this method are numerous and have the potential to mislead the decision makers in a child’s life. Methods of alleviating these problems are discussed.


Introduction

The Parent Designed Report Form is an assessment tool created by parents, for parents to measure the progress of their autistic children under specific treatments. It is advertised as the most specific and sensitive measure of change via methyl-B12, by its advocates. It’s creation and use seems closely, but not exclusively tied to the treatment protocol of methyl-B12 injections for autistic children.

The purpose of this paper is to 1) give a loose overview of the Parent Designed Report Form (PDRF). 2) Point out the problems present in within the PDRF and the justification of this tool. 3) Suggest specific changes that might help validate this tool.

Overview of PDRF

The PDRF is an assessment tool. It is periodically filled out by the parents who are often using the vitamin B12 injection treatment for their children’s autism. The tool is a retrospective Likert based scale assessment. This means that the child is not necessarily being directly observed by the parents at the time the scale is being filled out. The scales are to be filled out based on parent impression of child behavior within the recent period (6 weeks). The question include a second portion where parents are asked for specific examples of the change.

The Likert scale is based on 1-5, 1 being “Mild”, 5 being “Significant”. The questions typically refer to a “behavior” and a “change”.

For example:

# 125 “Tasks: New things never done before or never accepted before: Give the before and after examples necessary to explain you observation”.

#76 “Laughter, more normal amounts and ore normal types of laughter: Why are you saying this? Support your observation as to what is was before and now.”

An additional part of this assessment tool involves the Parents Specifics Documentation Letter. In this portion the parents are invited to write down in detail the changes they have seen in their child in what can accurately be called an emotional format.
Criticisms

There seems to be no evidence in support of the claim that the PDRF is the most subtle measure of a child’s progress. This claim is offered by the creators or advocates of this measure. It is seemingly based on the types and number of questions asked. Moreover there are no data that shows any sort of inter-observer-agreement on the test. We have no idea if the test is reliable between, two caregivers who were with the child over the same period. Further, we have no idea what the validity of this test might be. It has not been compared meaningfully to other established scales or to quantitative behavior assessments.

I report that some advocates of this measure claim that an affect is noticed following treatment via this measure, but not on other measures and that this constitutes a proof that this measure can detect smaller changes than the larger and better established tools. However, that is not necessarily the case, these could be false positives; I note again, that there has never been an attempt to validate this measure, the validity is assumed.

One of the great advantages claimed by proponents of this tool is that objective diagnosticians, using a reliable/valid observation based tool, only watch for a limited amount of time. They might miss the rare but increasing utterances of child first beginning to use words. The parents, they argue, who are always with the child, can observe this, remember it, and regurgitate it on the form. This seems acceptable, accept that parents like all people, are prone to the fallacy of confirmation bias, or only identifying those data, which confirm their current position possibly overlooking those data which could refute it, such as the child already increasing her utterances before the treatment began. I am reminded in this situation, of the claims of ESP advocates; it seems the really great evidence is never present when the objective skeptic is looking (Sagan, 1996).

The wording within the questions also seems to invite subjective opinion rather than objective observations on numerous occasions. For example:

#84 “Mood swings, fewer: Describe.”

#47 “Flexible, less rigid, can be redirected: Describe why you feel this to be a true statement. Be specific and give as many examples as you can think of”

#141 “Improvement in the quality of life of others e.g. therapists, etc. (indicate a title or position of the person)”

#82: Loving, cuddly, more so now than prior to the shots even though your child may always have been loving.

I identify particular problems with the Parents Specifics Documentation Letter.
It would not be an error to say that the description of this letter is a remarkable invitation to anti-science.

“Please note that a piece of paper with your “facts”, no matter how well written, will not convey some of the most important information for us to determine how well methyl b-12 is working for your child- that information being the “emotion of the moment”, that you felt when something your child did for the first time or at a special moment.”

“The specifics Documentation Letter is so much more powerful, and so much more believable, when you let us into your hearts and feelings by describing those items that made an impact or created a special memory.”

“We only want you to include comments on the responses that “blew you away”, had you utterly amazed!”

“it is this that MAKES MUSIC AND GIVES BEAUTY to being alive and working together with your child and you”.

The above being an excellent description of a post hoc, ergo propter hoc fallacy and a non-sequitur, the feeling of the parents, has absolutely no ability to establish the accuracy of a change caused by methyl B12.

Next, I wish to take issue with the claim via advocates of this measure, that researchers must find a way to validate parental experiences that show improvement via certain treatments in autism. This assertions seems to be assuming what it should first be proving, that parental observations in such situations are in fact “valid”. We know from research that parent’s description of child behavior changes based on what they are told, irregardless of the reality of the situation. This was demonstrated by Hoover & Millch (1994) who told parents to rate their child’s behavior as much worse following the children being given a sweet drink that the parents were told contained sugar (It contained an artificial sweetener).

My final problem with this assessment is that, it the comparison is not to a normed distribution, but to the child’s previous assessment. These assessments are typically done every six weeks for up to two years. The problems presented here are the concept of regression to the mean and maturation for eh child. Pre-school aged can and do mature rapidly (if unevenly at times). Moreover several of the questions in the form seem to focus on behaviors very closely tied to a developmental schedule as opposed to learning.
For example:

#92 (Concerning the child having opinions)

#81 (Listening better)

#78 (Use of language)

#110 (self confidence is greater)

#111 (Better structuring of sentences)

#112 (Less silliness)

#114 (New level of singing)

#100 (Reading at higher levels)

#115 (Improved sitting behavior)

#124 (Can multi-task)

#25 (Less clumsy)

#26 (More intelligent)

#27 (Improved compliance)

#45 (Able to better express feelings)

#1 (Abstract thinking)

#23 (Understands cause and effect)

There is not a single one of the above that is unlikely to improve, perhaps even significantly on a pre-school aged child, and especially a younger pre-school aged child over the course of 12 weeks. As matters stand the PDRF seems to be focusing on developmental tasks that are very likely to increase on just about any child.

Recommendation

I propose that in attempt to gain an understanding of the reliability of this measure that the following be investigated by those who took the time and effort to create this measure.

1) Typically developing children be assessed and their average scores compared to autistic children both in and out of the methyl-B12 program.

2) That objective data counts by observers (observing for a prolonged time) be compared to parent’s outcomes for this test.

3) That an attempt be made to assess this tool against other tools whose reliability validity is already established

4) That inter-observer reliability of this measure be assessed by comparing scores between caregivers of the same child.

References

Deprey LJ, et al "Double-Blind, Placebo-Controlled, Cross-Over Trial of Subcutaneous Methylcobalamin in Autism: Preliminary Results" AACAP 2006; 33:F47.

Hoover, D. W., Milich, R. (1994). Effects of sugar ingestion expectancies on mother-child interactions. Journal of Abnormal Psychology. 22(4), 501-15.