Product Update

2020 MAP Growth Norms Overview and FAQ

Blog Post created by Product Update on Apr 10, 2020

Introduction

The latest NWEA MAP Growth norms are scheduled to be released in July 2020. NWEA conducts norming studies every three to five years to provide the best and most up-to-date information we can about student achievement and growth to better support educational decision-making. It is an important part of our commitment to our partners.

 

The 2020 norms describe achievement and growth trends for the U.S. public school student population in language usage, mathematics, reading, and science over the 2015–16, 2016–17, and 2017–18 school years. The methodological approach to the 2020 norms is generally consistent with the approach taken in the 2015 study, though the new sets of norms benefits from an increased sample of students and schools.

 

There are several key enhancements to the 2020 norms that will be described in more detail below, including norms in new grade/subject areas, and growth norms that describe changes in student RIT scores over the summer months (from spring-to-fall).

 

Below are some answers to frequently asked questions (FAQs) that will help provide an introduction and overview of the 2020 norms. Partners can also access the 2020 MAP Growth Norms Overview document in the MAP Growth Help Center here. Additional information will be continue to be provided to partners prior to the July 2020 norms update.

 

Watch for the monthly MAP Growth Partner updates and visit the Product Update section of NWEA Connection, our online community for the latest on MAP Suite Back-to-School 2020.

 

COVID-19 Updates

Are timelines for the release of the 2020 Norms changed as a result of the impact of COVID-19 on NWEA and/or partners?

No. Timelines for the release of the 2020 Norms at this point have not been affected. The release of the norms is still scheduled for July 24, 2020, and additional norming information and data overviews will be made available in the coming weeks/months. Data from the 2019-20 school year was not included in the norming study, so there should be no effect of this absence on the estimation of norms, And, because of this absence, NWEA believes that up-to-date norming information will be critical to help interpret student achievement and growth when students return to school (updated April 8, 2020).

 

How will future norms studies be impacted by the potential absence?

This question is one of many questions that will be investigated by NWEA in preparation for future norms work. At this time, it is too early to say to what extent this absence will affect student achievement, and in turn, how this prolonged absence should be considered in future norms releases.

 

Key Norms Updates – Coverage and Trends

What grades and subjects are included in the 2020 MAP Growth norms—are there any changes from the 2015 norms?

There are a number of new grade/subject areas that now have achievement and growth norms. The table below shows the grades/subject areas in which norms are available and includes information on where norming information has been added with the release of the 2020 MAP Growth norms.

 

Subject

Overall Grades Covered

New Grade Coverage Added

Math

(K-12 General)

Achievement: K-12

Growth: K-12

Achievement: 12th Grade

Growth: 11th & 12th Grade

Reading

Achievement: K-12

Growth: K-12

Achievement: 12th Grade

Growth: 11th & 12th Grade

Language Usage

Achievement: 2-11

Growth: 2-11

Achievement: No Change

Growth: 11th Grade

Science (General)

Achievement: 2-10

Growth: 2-10

Achievement: 2nd Grade

Growth: 2nd, 9th, & 10th Grade

 

Are there any new growth norm term pairs in the 2020 norms?

Yes, one additional term pair will be available in the 2020 MAP Growth Norms Study – Spring-to-Fall – which will help with understanding changes in student RIT scores over the summer months (often referred to as “summer loss”).

 

2015

2020

Fall-to-Winter

Fall-to-Winter

Fall-to-Spring

Fall-to-Spring

Fall-to-Fall

Fall-to-Fall

Winter-to-Spring

Winter-to-Spring

Winter-to-Winter

Winter-to-Winter

Spring-to-Spring

Spring-to-Spring

 

Spring-to-Fall (NEW!)

 

Will the new Spring-to-Fall term pairing be included in MAP Growth reports?

No. The new Spring-to-Fall term pair will not be available in MAP Growth reports with the July 2020 norms release. NWEA is exploring other ways to make Spring-to-Fall growth norms available to partners.

 

Where can partners get more information related to the addition of the Spring-To-Fall term pair?

Spring-to-Fall growth norm information will be included in the 2020 MAP Growth Norming Study, the comprehensive research document that is scheduled to be released in July 2020.

 

How should partners utilize the new Spring-to-Fall term pair data?

Student achievement in the fall is typically lower than it was before summer break. This phenomenon is often called “summer loss” or “summer slide.” With the inclusion of Spring-to-Fall growth norms in the 2020 MAP Growth Norms Study, partners will now be able to contextualize students’ summer loss to understand if observed changes in scores over the summer are more or less than the average changes in scores for students across the U.S. 

 

What changes in student achievement did you find while developing the 2020 norms?

The general trend observed in the 2020 norms is that student achievement has declined since 2015.

The trends in 4th and 8th grade student achievement observed in the 2020 MAP Growth norms are consistent with those observed on other external benchmarks including the National Assessment of Educational Progress (NAEP).

 

How does the downward trend in student achievement affect the status percentiles associated with RIT scores?

The average RIT score associated with a particular grade/subject tends to be lower in 2020 than in 2015.  This means that the same RIT score for the same grade level, for the same testing season, and in the same subject area will generally be associated with a higher status percentile rank in the 2020 norms than in the 2015 norms.

 

What changes in student growth did you find while developing the 2020 norms?

In general, there aren’t significant differences in mean growth across grades, subjects, and term pairs. For example, in mathematics and reading, the growth norms on average tend to reflect growth that is slightly lower in magnitude for students in the earlier grades, and slightly greater for students in the upper grades. This pattern is generally consistent for students across the achievement distribution. The following tables show how the growth norms differ from 2015 to 2020, expressed as the difference in average RIT growth in mathematics and reading, for students with a starting achievement level at the 50th percentile.

 

 

Study Data & Methodology

What terms and academic years were used in the 2020 norms study?

The terms used for the 2020 norms are the fall, winter, and spring terms of the 2015–16, 2016–17, and 2017–18 academic years. 2016–17 is the “focal” year — the primary year used for the estimation of achievement norms. Data from all three years were used in modeling the growth norms.

 

What sample size is the 2020 norms study based on and how does that compare to the 2015 norms study?

The study’s results are based on K–12 grade level samples. Records are sampled from between 3.6 and 5.5 million test scores from 500,000 to 700,000 students attending over 24,500 public schools in 5,800 districts spread across all 50 states.

 

Rigorous procedures were used to ensure that the 2020 norms were representative of the U.S. public school student population, consistent with the approach taken with the 2015 MAP Growth Norms. More information on these procedures will be made available upon the release of the 2020 MAP Growth Norms Study in July 2020.

 

Were there any methodology changes applied in the 2020 norms study?

The methodological approach to the 2020 norms is generally consistent with the approach taken in the 2015 study.

One significant change is that an additional filter is applied in the 2020 norms to remove construct irrelevant variances due to student test-taking effort. In the 2020 norms, only effortful test scores (i.e. 10% or fewer rapid guesses) were utilized. By removing non-effortful scores from the sample, “noise” is removed from the data which leads to a more clearly defined target population for achievement and growth norms. This is an important change, given that high rates of student rapid guessing can bias student achievement estimates, often underestimating what students know or have learned.

 

MAP Growth and MAP Growth K-2 are both often taken at second grade. Does that factor into how the norms are reported at that grade?

The norms describe the achievement and growth of a nationally representative group of students in the subjects tested. The achievement and growth norms do not distinguish between second graders who take the MAP Growth K-2 and 2-5 tests. The 2020 norming sample reflects the fact that the majority of second-grade students take the MAP Growth 2-5 test. For more information on which test to administer to your students, please consult the following MAP Growth grade-level test guidance documents:

 

What testing windows for fall, winter, and spring will be used for the norms in the MAP Growth reporting systems? Are the weeks used the same as the 2015 Norms Study?

The 2020 Norms Study will show achievement and growth norms tables for and between fall, winter, and spring testing by subject and grade. These tables assume that a fixed number of weeks of instruction have occurred between each testing season and present norming information associated with the default instructional weeks (i.e., 4th week for fall, 20th week for winter, and 32nd week for spring). The default instructional weeks used for the 2020 Norms Study tables are the same as those used in the 2015 Norms Study tables.

 

Default Instructional Weeks for Norms Study Tables

Fall

4th Week

Winter

20th week

Spring

32nd week

 

Will the MAP Growth reporting system always use default weeks to calculate scores, or will they take into account actual district testing calendar?

[Updated July 2020] When the 2020 norms are implemented in July 2020, the MAP Growth reporting system will adjust these instructional weeks assumptions based on the testing schedules specified for each district or school, so the printed tables in the norms document may differ from what are shown on school/district reports, depending on district/school inputted testing schedules. As of the August 2020 update, the reporting system specified instructional weeks can range from week 0 to the 8th week in the fall, the 16th to 24th week in the winter, and the 26th week to the 36th week in the spring. If no changes are specified, the default instructional weeks will be used (4th, 20th, and 32nd instructional weeks for fall, winter, and spring respectively).

 

General FAQs

When will the new 2020 norms show in the MAP Growth system and reports?

The new norms will be implemented in MAP Growth as part of the July 2020 software release currently scheduled for July 24, 2020. We are providing norms information and resources ahead of the release to assist partners with planning for the update.

 

Will Premium Reports be updated to reflect the 2020 norms data?

Premium Reports can be generated using either the 2015 or the 2020 norms after the July 2020 release date. Information on which norms to apply will need to be specified when ordering a Premium Report.

 

Will linking studies, and therefore related projected proficiency data in reports, be updated to reflect the new norms data?

[Update May 2020]: Due to COVID-19 and related school closures, some linking studies may not be updated as originally planned in time for the July 2020 norms release. NWEA will update and publish as many linking studies as possible with the new norming information. The updated studies that will be published in July 2020 include those where the necessary data was available or newly gathered prior to the COVID closures. We will work with our partners to secure the necessary data to complete the remaining linking study updates as soon as possible after the July release with the majority anticipated by calendar year end.

 

Will the Achievement Status and Growth (ASG) and School Norms calculators be updated to reflect the 2020 norms data?

Yes. NWEA calculators will be updated with the July 2020 release. With the 2020 norms calculator update, the ASG and School Norms calculators will be combined into a single file. Partners can continue to access these resources in the MAP Growth Help Center.

 

Will the 2015 norms information be available in any reports after release of the 2020 norms?

Yes, after the 2020 norms release in July 2020, a toggle option will be available in the Comprehensive and Combined Data File (CDF) exports, similar to the 2011 to 2015 toggle previously available. This toggle option is only available for CDF export and is not available for any other MAP Growth reports.

 

Since the changes in the norms, several students who did not qualify for gifted placement last spring under the old norms now have percentile rankings associated with that same score that would qualify them for gifted placement. What do we do?

Whenever we release new norms, it is likely that the overall distribution of RIT scores will change, meaning that percentile ranks for some students will change as well. Changes in the norms reflect a change in the normative frame of reference, including what a particular RIT score means in comparison to other students across the nation in the same grade and subject area. In general, NWEA recommends that placement decisions be made based on a student attaining a particular RIT score instead of a percentile. If decisions are made on the basis of points on the score scale (e.g., RIT) instead of percentiles, changes in percentiles that almost invariably accompany changes in the norms will be far easier to explain to stakeholders. We would recommend treating all students who achieve a qualifying score, whether that score was gained under the old norms or the new norms, as crossing the qualifying threshold. And of course, we’d recommend that the district consider multiple criteria in addition to MAP Growth scores to determine any program eligibility.

 

Why are achievement status norms at the high school level relatively flat across the grades?

Many high school students show relatively little change in observed MAP Growth reading and mathematics scores over the course of a typical school year. This is particularly true for students with high MAP Growth scores at the beginning of the school year, since, presumably, most of these students may not receive instruction in general reading and general mathematics but take more specialized coursework.

 

How can we use the status and growth norms to help track performance and improvement of high school students?

It is perfectly appropriate to measure the growth of high school students with the MAP Growth assessment. Individual student growth scores can be valuable for the purposes of evaluating progress for subsets of students, goal setting, and predictions to external benchmarks. To the latter point, growth norms for high school students can be used for predictions about likely performance on external measures of achievement such as state summative tests or measures of college readiness like ACT and SAT.

One way to track performance and improvement at the high school level is to focus on the mean score at the group level rather than individual student gains. The measurement error associated with group averages is far smaller than that for individual students, meaning that very small changes in group averages over time can be observed with far greater precision than is possible for individual students.

 

Does the 2020 MAP Growth Norms Study include percentile ranks at the district level?

No, the 2020 Norms Study only includes student and grade-within-school level percentile ranks and does not provide summary district level percentile ranks.

 

How can I get an idea of how my district is performing in comparison to other districts across the nation?

It would be informative to look at a picture/graph of individual schools (and specifically, individual grade levels within schools) within a district in terms of their school Conditional Growth Index (CGI) or school Conditional Growth Percentile (CGP) and their school achievement status information. This would not be a summarization of the entire district as a whole per se, but would give some indication of how the district, as a collection of schools, performed relative to the norms, and identify grade-levels within schools across the school district where, for example, students have demonstrated above-average achievement and growth.

 

Has NWEA taken any new steps to ensure the integrity of test responses the norms are based on?

Yes, the 2020 norms exclude students who rapidly guessed on more than 10% of their test items (i.e. non-effortful test events).

This is important given that high levels of rapid guessing can bias student test results (generally underestimating what students know or have learned). As such, the 2020 norms reflect the achievement and growth patterns for engaged test takers in public schools across the US.

 

What additional resources are available for partners to learn more about the 2020 norms release, changes to achievement and growth norms, and more?

There are several resources available and in development to provide additional information about the 2020 norms release for partners.  In addition to this FAQ, which will be expanded between now and the July 2020 release, the following resources may be helpful.

 

Be sure to watch the monthly MAP Growth Partner Update and visit the Product Update section of NWEA Connection, our online community for the latest on MAP Suite Back-to-School 2020.

Outcomes