Table of Contents
existing SamplesError Report
To simplify analysis, the Error Report examines data from student first attempts only.
In the Error Report, you can view information aggregated by problem step or knowledge component.
The Error Report provides:
- summaries of student performance by step or knowledge component
- actual values students entered and the feedback they received
- at-a-glance information on problem coverage—the number of students exposed to a particular step or knowledge component, the knowledge components associated with each step, and the the problems that contain each knowledge component.
Viewing by problem
When viewing by problem, you can view only one problem at a time. Change the problem by clicking a problem's name from the scrollable list in the navigation sidebar.
The current problem is broken down by step, and lists the various attempts students made.
At the top of the page, the full problem name is shown.
Steps
A problem is commonly broken down into multiple steps. For each step, a list of details about that step are displayed.
- Step Name
- The first line of step information is the step name. The format of the step name varies across tutor technologies and logging implementations. CTAT tutors, for example, characterize step name as the widget the student interacted with (ie, the 'selection') and the action the student performed on that widget. (Student input—the third component of the selection-action-input format—is not displayed in this portion of the table since it varies by student response. Instead, if it exists, it appears under the Answer column of each row in the Attempts table.)
- Number of Students
- The total number of students that attempted this step of the problem.
- Knowledge Component(s)
- The knowledge component(s) that are associated with this step of the problem.
- Sample
- The name of the sample(s) from which the shown data comes. The sample name here will one of the selected samples shown in the upper-left corner of the sidebar.
Attempts
Figure 1: Evaluation Types
- Evaluation
- Student attempts are classified into three evaluation types: correct, incorrect, and hint. Each evaluation type corresponds with a background color (see Figure 1).
- Number of Students
- The number of students that attempted this step. Expressed in the form: count (percentage of students that attempted this step).
- Answer
- The answer that the student provided on his or her first attempt for this step
- Feedback
- The feedback message provided by the tutor to the student.
- Classification
- Used to further describe the student-problem interaction (e.g., some tutors employ multiple hint and error levels).
Viewing by knowledge component
When viewing by knowledge component, you can select multiple knowledge components, but each will be shown in its own table row.
Values shown are aggregates by knowledge component. From these you can tell, for instance, the more difficult knowledge components by examining the percentages on corrects, errors, and hints.
Knowledge Component
- Knowledge Component (KC) Name
- Displayed in bold type.
- Number of Observations
- The total number of times a student took an opportunity to demonstrate the knowledge component.
- Problem(s)
- The problem(s) that contain steps with this knowledge component.
- Sample
- The name of the sample(s) from which the shown data comes. The sample name here will one of the selected samples shown in the upper-left corner of the sidebar.
Aggregate Values
- Evaluation
- Student attempts are classified into three evaluation types: correct, incorrect, and hint. Each evaluation type corresponds with a background color (see Figure 1).
- Number of Observations
- The number of times a student took an opportunity to demonstrate the knowledge component, broken down by evaluation. Expressed in the form: count (percentage of observations for this KC).
