FAQ: Improving test engagement in MAP Growth

Document created by Community User on Oct 16, 2018Last modified by Community User on Mar 14, 2019
Version 63Show Document
  • View in full screen mode
Introduction
Responses to frequently asked questions on the test engagement capability currently in MAP Growth and updates for fall 2019 testing. The fall 2019 section was updated on 3/14/19.

Information

This FAQ is divided into three sections for your convenience. Quickly access the content most relevant to you.

 
Test engagement overall

What happened during fall 2018 testing related to rapid-guessing and test invalidation?
MAP Growth partners saw a higher number of invalidated tests because of excessive rapid-guessing after the July 2018 MAP Growth product update. This update introduced a new invalidation rule that automatically invalidated the test if a student rapid-guessed on 30 percent or more of the total number of questions received. This caused confusion and frustration to partners. We know this was not rolled out effectively to our partners. We did not provide adequate supporting documents, guidance, and necessary time for partners to prepare for retesting and train proctors accordingly.

While we continue to alert proctors to rapid-guessing, since October 31, 2018, MAP Growth does not invalidate tests due to excessive rapid-guessing. We’ve updated help documents, training support staff, and provided early communication to partners that started with an October 17 email from Chris Minnich, CEO.

Your partnership and feedback are essential in the process as we work to improve our assessments and make the information they provide timelier and more useful to you, your teachers, and your students.

Who can I contact for advanced inquiries on the test engagement capability in MAP Growth?
Please contact NWEA Partner Support.

 
Winter and spring 2019 testing

What changes were made for winter testing?
Since October 31, 2018, MAP Growth does not invalidate tests due to excessive rapid-guessing. All other test invalidation rules remain unchanged.

The following reports do not display invalidated tests due to excessive rapid-guessing:
These changes will not affect the proctor alerts due to rapid-guessing during testing and will not change previous MAP Growth scores.

Since November 12, 2018, the Grade Breakdown Report displays the percent of questions rapid-guessed for each student. The Grade Breakdown Report can only be accessed by the Administrator, School Assessment Coordinator, and District Assessment Coordinator roles in MAP Growth.

This information is shown under the header of % of Disengaged Responses in column K.

Screenshot of the Grade Breakdown Report spreadsheet, with column K, labeled % of Disengaged Responses, highlighted.

Will the Student Profile Report continue to display disengagement information?
Yes, the Student Profile Report will continue to display Percentage of Disengaged Responses and Estimated Impact of Disengagement on RIT.

A screenshot of the header tab for the Student Profile report. It shows the subject, RIT, standard error, date, duration, % of disengaged responses, and estimated impact of disengagement on RIT.

Will proctors be notified when students rapid-guess during testing?
Yes. Proctors will receive a notification when students rapid-guess. Note, many proctors are not at their computers at all times watching this screen during testing. The proctor screen refreshes every 60 seconds, indicated in the image below, but students can rapid-guess before the proctor is alerted because of the 60-second refresh window. In short, the alert is not real time but delayed.

Screenshot of the Students Disengaged alert on the testing session page. The alert appears near the Refresh Status button, and shows the number of students currently disengaged.

Proctors can click the orange Student(s) Disengaged box to expand the information.

Screenshot showing how to dismiss a disengagement alert. Each disengaged student has an alert icon under the Disengaged column of the test session page. You can click this alert to dismiss it. Alternately, you can click the Students Disengaged button towards the top and then select Dismiss All.

Is NWEA going to change how often the proctor screen refreshes?
We are reviewing the impact of adjusting the 60-second refresh but we do not have immediate plans to adjust it.

Is it meaningful to compare fall to winter scores due to the changes released on October 31, 2018?
MAP Growth data comparison from fall to winter continues to provide you with valuable information about student growth to inform instruction at the district, classroom and student levels.

While there is no change to those students that did not rapid-guess, we recommend you use caution when evaluating students who did rapid-guess. As you compare individual student scores to inform instruction, it is always best to use the valid score as the basis for instruction. Using a score that was generated through rapid-guessing may provide lesser quality information.

Since November 12, 2018, the Grade Breakdown Report displays the percent of questions rapid-guessed for each student. This report will allow you to identify students who reached or exceeded the 30 percent threshold.

 
 Updates for fall 2019 testing

What improvements are being made for fall 2019 testing?
Below are details of the test engagement capability planned for fall 2019 testing. Functionality will focus on keeping students engaged with the MAP Growth test and supporting optional retesting. 

A key shift to note: We will not be reintroducing the rule that invalidates tests when students reach a rapid guessing threshold in fall 2019. We'd first like to get your feedback on the impact of other fall 2019 features that we expect to increase student test engagement. This will help us determine how we can continue to enhance the student test engagement capability in a way that is responsive to your needs. 

The capabilities listed below are under development. Please note that some minor adjustments to the described capabilities may occur leading up to July 2019.
  • MAP Growth will not terminate or invalidate tests when students rapid-guess on 30 percent of the total number of test questions.
  • Students will receive a message at the start of the test encouraging them to remain engaged.
  • When students rapid-guess, proctors will be notified and the test will auto-pause so the proctor can re-engage the student and resume the test.
  • New in-product training and content at NWEA Connection will provide detailed guidance on the auto-pause feature and guidelines for communicating the relevance of test engagement to students.
  • New resources will help parents, teachers, proctors, and students understand the relevance of student engagement on tests.
  • To support optional retesting:
    • A new report will be available to help districts identify students that rapid-guessed on 30 percent of the total test questions.
    • Districts will be able to work with NWEA to configure the system to allow, or not allow, retesting for students who rapid-guessed on 30 percent of the total test questions.
    • Proctors and administrators will be able to use the test session management page to easily find students who rapid guessed on 30 percent of the total test questions. If the district configuration allows, they can assign these students to a new test in a group.
    • The Student Profile and Grade Breakdown reports will continue to display the percent of questions rapid-guessed for each student.

How do I get more details on test engagement for fall 2019 testing?
We will share additional details in the months leading up to fall 2019 testing through the monthly partner update emails and this FAQ. Please refer to this FAQ for additional information.

What constitutes excessive rapid guessing?
Our research shows that when students rapid guess on 30 percent or more of the total number of questions on a test, their results may not accurately reflect their performance and we recommend that these students be retested. 

Will MAP Growth invalidate tests when students rapid-guess on 30 percent of the total number of test questions?
MAP Growth will not invalidate tests when students rapid-guess on 30 percent of the total number of test questions in fall 2019. We'd first like to get your feedback on the impact of other fall 2019 features that we expect to increase student test engagement. This will help us determine how we can continue to enhance the student test engagement capability in a way that is responsive to your needs.

 
See also:

About rapid-guessing and test disengagement

Article Number
3688

Attachments

    Outcomes