FAQ: Improving the test engagement capability in MAP Growth

Document created by Community User on Oct 16, 2018Last modified by Community User on Jan 15, 2019
Version 36Show Document
  • View in full screen mode
Introduction
Responses to frequently asked questions on the test engagement capability currently in MAP Growth and updates for fall 2019 testing.

Information

This FAQ is divided into three sections for your convenience. Quickly access the content most relevant to you.

 
Test engagement overall

What happened during fall 2018 testing related to rapid-guessing and test invalidation?
MAP Growth partners saw a higher number of invalidated tests because of excessive rapid-guessing after the July 2018 MAP Growth product update. This update introduced a new invalidation rule that automatically invalidated the test if a student rapid-guessed on 30 percent or more of the total number of questions received. This caused confusion and frustration to partners. We know this was not rolled out effectively to our partners. We did not provide adequate supporting documents, guidance, and necessary time for partners to prepare for retesting and train proctors accordingly.

While we continue to alert proctors to rapid-guessing, since October 31, 2018, MAP Growth does not invalidate tests due to excessive rapid-guessing. We’ve updated help documents, training support staff, and provided early communication to partners that started with an October 17 email from Chris Minnich, CEO.

Your partnership and feedback are essential in the process as we work to improve our assessments and make the information they provide timelier and more useful to you, your teachers, and your students.

Who can I contact for advanced inquiries on the test engagement capability in MAP Growth?
Please contact NWEA Partner Support.

 
Winter and spring 2019 testing

What changes were made for winter testing?
Since October 31, 2018, MAP Growth does not invalidate tests due to excessive rapid-guessing. All other test invalidation rules remain unchanged.

The following reports do not display invalidated tests due to excessive rapid-guessing:
These changes do not affect the proctor notifications due to rapid-guessing during testing and will not change previous MAP Growth scores.

Since November 12, 2018, the Grade Breakdown Report displays the percent of questions rapid-guessed for each student. The Grade Breakdown Report is available for users with the Administrator, School Assessment Coordinator, or District Assessment Coordinator roles in MAP Growth. This information is shown under the header of % of Disengaged Responses in column K.

Screenshot of the Grade Breakdown Report spreadsheet, with column K, labeled % of Disengaged Responses, highlighted.

Will the Student Profile Report continue to display disengagement information?
Yes, the Student Profile Report will continue to display Percentage of Disengaged Responses and Estimated Impact of Disengagement on RIT.
A screenshot of the header tab for the Student Profile report. It shows the subject, RIT, standard error, date, duration, % of disengaged responses, and estimated impact of disengagement on RIT.

Are proctors notified when students rapid-guess during testing?
Yes. Proctors receive a notification when students rapid-guess. However, many proctors are not at their computers at all times watching this screen during testing. The proctor screen refreshes every 60 seconds, indicated in the image below, but students can rapid-guess before the proctor is notified because of the 60-second refresh window. In short, the notification is not real time but delayed.

Screenshot of the Students Disengaged alert on the testing session page. The alert appears near the Refresh Status button, and shows the number of students currently disengaged.

Proctors can click the orange Student(s) Disengaged box to expand the information.

Screenshot showing how to dismiss a disengagement alert. Each disengaged student has an alert icon under the Disengaged column of the test session page. You can click this alert to dismiss it. Alternately, you can click the Students Disengaged button towards the top and then select Dismiss All.

Is NWEA going to change how often the proctor screen refreshes?
We are reviewing the impact of adjusting the 60-second refresh but do not plan to adjust it by the winter test season.

Is it meaningful to compare fall to winter scores due to the changes released on October 31, 2018?
MAP Growth data comparison from fall to winter continues to provide you with valuable information about student growth to inform instruction at the district, classroom and student levels.

While there is no change to those students that did not rapid-guess, we recommend you use caution when evaluating students who did rapid-guess. As you compare individual student scores to inform instruction, it is always best to use the valid score as the basis for instruction. Using a score that was generated through rapid-guessing may provide lesser quality information.

Since November 12, 2018, the Grade Breakdown Report displays the percent of questions rapid-guessed for each student. This report will allow you to identify students who reached or exceeded the 30 percent threshold.

 
Fall 2019 testing

What improvements are being made for fall 2019 testing?
We will re-introduce the test engagement capability for fall 2019 testing. The capabilities listed are under development. Please note that some minor adjustments to the described capabilities may occur leading up to July 2019.
  • Students will receive a message at the start of the test encouraging them to remain engaged.
  • When students rapid-guess, proctors will be notified and the test will auto-pause so the proctor can re-engage the student and resume the test.
  • MAP Growth will invalidate tests when students rapid-guess on 30 percent of the total number of test questions, at which point the test will end in order to protect instructional time.
  • To better support retesting processes, educators, including proctors, will have access to reports showing students with invalidated tests due to excessive rapid guessing.
  • New in-product training and content at NWEA Connection will provide detailed guidance on the rapid-guessing test invalidation rule, the associated functionality and reports, and guidelines for communicating the relevance of test engagement to students.
  • New videos will help parents, teachers, proctors, and students understand the relevance of student engagement on tests and the reasons for the rapid-guessing invalidation rule.

How do I get more details on test engagement for fall 2019 testing?
We are planning to update this article with details by March 4th. A key step in developing the test engagement capability is to gather input from users to validate and refine the capability. We will share additional details on March 4th, please check back at that time.

What do you mean when you say, “MAP Growth will invalidate tests when students rapid-guesses on 30 percent of the total number of test questions, at which point the test will end in order to protect instructional time?”
As we re-introduce the test engagement capability for fall 2019 testing, user feedback indicated students that rapidly guess at the beginning of the test were frustrated they had to complete the entire MAP Growth test even though they already met the 30 percent threshold.

For fall 2019 testing, we want to prevent this frustration so that when a student rapidly guesses at the beginning of the test and meets the 30 percent invalidation threshold, he/she will not have to complete the entire test.

 
See also:


Article Number
3688

Attachments

    Outcomes