Observed growth is less than the standard error (SE) for growth

Document created by Community User on Oct 26, 2017
Version 1Show Document
  • View in full screen mode
Question
How can I accurately assess student observed growth compared to projected growth when the typical growth is less than the observed growth standard error?

Answer
There are two distinct topics regarding growth and SE for growth (observed growth SE):
  • Projected growth (typical growth) is less than the SE for growth.
  • Observed growth is less than the SE for growth.

Regarding the first topic - projected growth is less than the SE for growth - the standard error (SE) for growth does not apply directly to the projected growth or to the growth index, but to the actual growth - the observed growth - a student achieves between to valid test events.

This article will address the 2nd observation, when observed growth is less than the observed growth SE. For more information on SE, see Making Sense of Standard Error of Measurement on NWEA.org.

 
Definitions:

Observed growth SE is the amount of measurement error associated with the term-to-term growth between two test events. The equation to determine the observed growth SE between any two test events is (SEM1^2 + SEM2^2)^1/2 and it is typically around 4.3 for two valid tests.

Projected growth is the mean growth that was observed in the latest NWEA norms study for students who had the same starting RIT score. The SEM for projected growth is typically very small and can be disregarded for the purpose of this question.

Growth index is the RIT points by which a student exceeded (positive values) or fell short of (negative values) their projected growth.

 
Example:

Student Diamonte Cymbola, a 4th grader, achieved the following two scores on his fall 2014 and spring 2014 language assessments, respectively: 159 with a SEM of 3.0 and a 163 with a SEM of 3.2. The SEM for growth between these two events is 4.4, the growth projection was 11, and the growth index is -7. Given these values, the observed growth for this student is 4 RIT points between his fall and spring assessments. 

Question: With an observed growth of 4 and an SEM for growth of 4.4, the actual growth for this example student ranges from -0.6 to 8.4 RIT points. Is it possible this student did not grow at all? The answer is yes, but it’s unlikely. The following is true:
  • An actual RIT growth of 4 points is the best estimate for this example student. This is the mean. The probability that Diamonte’s actual growth falls within the range determined by this student’s SEM of growth (within ±1 standard deviation) is about 68%. The confidence interval grows as we include larger areas around the observed score. For example, the probability that Diamonte’s actual growth falls within the range -4.8 to 12.8 (within ±2 standard deviations) is about 95%.
  • An actual growth of -0.6 RIT is one full SEM below the mean for this example student, which means there is about a 15.9% likelihood that Diamonte’s actual growth is -0.6 RIT points or less. By comparison, the probability that Diamonte’s actual growth is greater than -0.6 RIT is about 84.1%.
  • An actual growth of 8.4 RIT is one full SEM above the mean for this student student, which means there is about a 15.9% likelihood that Diamonte’s actual growth is 8.4 RIT points or higher. The probability that Diamonte’s actual growth is less than 8.4 is 84.1%.
  • If this student could be tested again over the same period with comparable tests, there would be about a 68% (2/3) chance that his growth would fall within the same range.

The purpose of standard error of measurement is to communicate the amount of precision one can expect from a score. In the case above and in similar cases, the best response is to proceed under the assumption that the actual growth achieved is positive and that the student’s actual growth is more likely to be near the mean of the range than the extremes. As with all data, our conclusions about this student would be strengthened if they were supported by corroborating evidence. Keep in mind that it is not wrong or irrelevant to see actual growth that is less than typical growth.

Note: Observed growth has a SE associated with it. The term "possible observed growth," which is sometimes used by partners, refers to the observed growth, plus or minus the SE. As above, this means that when we include the SE, we should question whether we can definitely say that the student met their projected growth. In other words, the "met projected growth" flag should be carefully evaluated.

See also:

Article Number
000001914

Attachments

    Outcomes