Frequently asked questions about how NWEA plans to address test invalidations due to rapid-guessing concerns.
What happened during the recent fall test season related to rapid-guessing and test invalidation?MAP Growth partners saw a higher number of invalidated tests because of excessive rapid-guessing after the July 2018 MAP Growth product update. This update introduced a new invalidation rule that automatically invalidated the test if a student rapid-guessed on 30 percent or more of the total number of questions received. This caused confusion and frustration to many partners during fall testing. We know this was not rolled out effectively to our partners. We did not provide adequate supporting documents, guidance, and necessary time for partners to prepare for retesting and train proctors accordingly.
While we continue to alert proctors to rapid-guessing, starting October 31, 2018, MAP Growth will not invalidate tests due to excessive rapid-guessing. We are updating help documents, training support staff, and providing early communication to partners starting with an email on October 17 from our CEO, Chris Minnich, with the goal of re-introducing invalidation at a future date.
Your partnership and feedback are essential in the process as we work to improve our assessments and make the information they provide timelier and more useful to you, your teachers, and your students.
Starting October 31, 2018, MAP Growth will not invalidate tests due to excessive rapid-guessing. All other test invalidation rules remain unchanged.
What changes are being made for the upcoming winter test season?
The following reports will not display invalidated tests due to excessive rapid-guessing:
These changes will not, however, affect the proctor alerts due to rapid-guessing during testing and will not change previous MAP Growth scores.
Starting November 12, 2018, the Grade Breakdown Report will display the percent of questions rapid-guessed for each student. The Grade Breakdown Report can only be accessed by the Administrator, School Assessment Coordinator, or District Assessment Coordinator roles in MAP Growth. This information will be shown under the header of % of Disengaged Responses in column K.
What if districts want to see which students rapid-guessed on 30% or more of the total number of questions received?
Yes, the Student Profile Report will continue to display Percentage of Disengaged Responses and Estimated Impact of Disengagement on RIT.
Will the Student Profile Report display disengagement data?
Yes. Proctors will receive an alert when students rapid-guess. Note, many proctors are not at their computers at all times watching this screen during testing. The proctor screen refreshes every 60 seconds, indicated in the image below, but students can rapid-guess before the proctor is alerted because of the 60 second refresh window. In short, the alert is not real time, but delayed.
Will proctors still receive an alert when students rapid-guess during the live testing session?
Proctors can click the orange Student Disengagement box to expand the information.
Full plans for re-introduction are not finalized but will include more robust proctor tools such as the ability for a proctor to auto-pause a student test.
What improvements are being made after winter testing?
Additionally, we will update our in-product help to be more thorough and provide professional learning and onsite training on the topic of student disengagement so that you have full support for a successful testing session across your schools.
Look for an update from NWEA on plans in January.
Please contact NWEA Partner Support.
Who can I contact for advanced inquiries?
We believe that the invalidation rule is important in that it provides a more valid and reliable test experience and ensures you have the highest quality data possible to make decisions that impact students and their learning.
Why is NWEA turning it off, then back on again in the future? Is NWEA considering these tests valid now? Is there any impact on data validity?
We are reviewing the impact of adjusting the 60 second refresh but do not plan to adjust it by the winter test season.
Is NWEA going to change how often the proctor screen refreshes?
We are reviewing this request but do not plan to add this request by the winter test season.
Will students who are rapid-guessing receive a warning in MAP Growth?
MAP Growth data comparison from fall to winter continues to provide you valuable information about student growth to inform instruction at the district, classroom and student levels.
Is it meaningful to compare fall to winter scores due to the changes coming out October 31, 2018?
While, there is no change to those students that did not rapid-guess, we recommend you use caution when evaluating students who did rapid-guess. As you compare individual student scores to inform instruction, it is always best to use the valid score as the basis for instruction. Using a score that was generated through rapid-guessing may provide lesser quality information.
Starting November 12, 2018, the Grade Breakdown Report will display the percent of questions rapid-guessed for each student. This report will allow you to identify students who reached or exceeded the 30 percent threshold.