22-11-2012, 06:01 PM
Assessing Learners Behavior by Monitoring Online Tests through Data Visualization
Assessing Learners Behavior.pdf (Size: 329.59 KB / Downloads: 76)
Abstract
This paper presents an approach and a system to let tutors monitor several important aspects related to
online tests, such as learner behavior and test quality. The approach includes the logging of important data
related to learner interaction with the system during the execution of online tests and exploits data visualization
to highlight information useful to let tutors review and improve the whole assessment process. This paper has
focused on the discovery of behavioral patterns of learners and conceptual relationships among test items. For
this Characterization and summarization has been used. The Characterization and summarization is
implemented efficiently using Attribute Oriented Induction algorithm which discovers patterns for accessing
learners behavior. By analyzing the data visualization charts, we have detected several previously unknown test
strategies used by the learners. Last, we have detected several correlations among questions, which gave us useful
feedbacks on the test quality.
INTRODUCTION
In today’s academic environments the tutors are playing vital role that they not only plays the role of teacher but
also should play the role of guide and mentor. The tutors in corporate training and academic environments are
assessing learner’s ability and skills and accordingly he provides grading for their skills. Now based on these
grading he suggested improvements to increase the learners learning capability, thinking ability and knowledge
base. For this the tutor continuously conducts various types of tests. E-TESTING systems are being widely adopted
in academic environments, as well as in combination with other assessment means, providing tutors with powerful
tools to submit different types of tests in order to assess learners’ knowledge. Among these, multiple-choice tests are
extremely popular, since they can be automatically corrected. However, many learners do not welcome this type of
test, because often, it does not let them properly express their capacity, due to the characteristics of multiple-choice
questions of being “closed-ended.” Even many examiners doubt about the real effectiveness of structured tests in
assessing learners’ knowledge, and they wonder whether learners are more conditioned by the question type than by
its actual difficulty. In order to teach learners how to improve their performances on structured tests, in the past,
several experiments have been carried out to track learners’ behavior during tests by using the think-out-loud
method: learners were informed of the experiment and had to speak during the test to explain what they were
thinking, while an operator was storing their words using a tape recorder. This technique might be quite invasive,
since it requires learners to modify their behavior in order to record the information to analyze [1], [2], [3], [4], [5],
which might vanish the experiment goals, since it adds considerable noise in the tracked data. Thus knowledge
discovery (KDD) process is the main theme of the project.
Data Visualization
This is used to present the analysis data in different forms such as curves , charts , lines , pie charts ,bar
charts , circle , comparison lines so on. The above may be 2D or 3D. In above presentations learner’s data may be
skills, abilities and behavior. Data visualization [7] provides a graphical representation of data, documents, and
structures, which turns out to be useful for various purposes. Data visualization provides an overview of complex
and large data sets, shows a summary of the data, and helps human in the identification of possible patterns and
structures in the data[2][3][4][7]. Thus the goal of data visualization is to simplify the representation of a given data
set, minimizing the loss of information.
Visualization methods [7] can be either geometric or symbolic. In a geometric visualization, data are represented by
using lines, surfaces, or volumes and are usually obtained from physical model or as a result of a simulation or a
generic computation. Symbolic visualization represents non-numeric data using pixels, icons, arrays, or graphs.
Detailed Problem Description
This project aims to present a solution enabling the recording of learners habits during online tests without
informing them of the underlying experiment without asking them to modify their behavior which potentially yields
more realistic results. The project deals monitoring several important aspects related online tests, such as learner
behavior and test quality. The approach includes the logging of important data related to learner interaction with the
system during the execution of online tests and exploits data visualization to highlight information useful to let
tutors review and improve the whole assessment process. This focused on the discovery of behavioral patterns of
learners of learners and conceptual relationships among test items. In Particular, by analyzing the Data visualization
charts, detecting several previously unknown test strategies used by the learners. Last, this detects several
correlations among Questions, which gave us useful feedbacks on the test quality.
PROPOSED APPROACH
In this section, we describe the approach to discover knowledge related to learner activities during online tests,
which can be used by tutors to produce new test strategies. In particular, we have devised a new symbolic data
visualization strategy with help of Attribute Oriented Induction algorithm, which is used within a KDD process to
graphically highlight behavioral patterns and other previously unknown aspects related to the learners’ activity in
online tests.
In this section, we describe the system implementing the proposed approach. The system is composed of a Logging
Framework and a Log Analyzer application. The former, based on the AJAX technology, captures and logs all of the
learners’ interactions with the e-testing system interface (running in the Web browser). It can be instantiated in any
e-testing system and is further composed of a client-side and a server-side module. The latter is a stand-alone
application that analyzes the logs in order to extract information from them and to graphically represent it.
Attribute Oriented Induction Algorithm
An attribute-oriented induction method for data-driven discovery of quantitative rules in relational database uses
domain knowledge to generate descriptions for predefined subsets of a relational database. This attribute-oriented
approach uses the concept hierarchy to direct the learning process.
A set of basic principles for Attribute oriented induction in relational databases is summarized as follows.
1. Generalization should be performed only on the set of data which is relevant to the learning task.
2. Generalization should be performed on the smallest decomposable components of data relation.
3. Attribute removal: If an attribute has too many distinctive values and there is no higher level concept
provided for further generalization, it should be removed from the relation.
EXPERIMENTAL RESULTS
In order to demonstrate the effectiveness of the approach and of the proposed system, we have used them in the
context of the Web Development Technologies course in our faculty: the eWorkbook system, equipped with the new
module for tracking the learners’ interactions, has been used to administer an online test to learners. They have not
been informed of the experiment; they just knew that the grade obtained on the tests concurred to determine the final
grade of the course exam. The test, containing a set of 25 items to be completed in a maximum of 20 minutes, was
administered to 71 learners, who took it concurrently in the same laboratory. The assessment strategy did not
prescribe penalties for incorrect responses, and the learners were aware of that. The logger was enabled, and an
approximately 4-Mbyte-sized XML log file was produced. The logging activity produced no visible system
performance degradation. Then, the Log Analyzer has been used for analyzing the logs in order to extract
information from them and to graphically represent it in order to trigger a visual data mining process where the tutor
plays a central role. In the case of the mentioned experiments, the visual analysis of charts enabled a tutor to infer
interesting conclusions about both the strategies the learners used to complete tests and the correlation between
questions. In the former case, the objective was not only to understand the learners’ strategies but also to detect the
most successful of them under the circumstances explained above and, possibly, to provide learners with advice on
how to perform better next time. On the other hand, question correlation analysis aims to improve the final quality of
the test by avoiding the composition of tests with related questions.
CONCLUSION
It has been presented an approach and a system to let tutors monitor learners’ strategies during online tests. The
approach exploits data visualization to draw the data characterizing the learner’s test strategy, in order to trigger the
tutor’s attention and to let him/her discover previously unknown behavioral patterns of the learners and conceptual
relationships among test items. The tutor is provided with a powerful tool that lets him/her review the whole
assessment process and evaluate possible improvements. We have extensively used the implemented system
experimentally to evaluate online test strategies in the courses of our faculty, in order to assess the whole approach.
This lets us discover several relevant patterns regarding the test quality, the characteristics of used strategies, and
the impact on the final score. The cheating behavior of the learner can also be visualized by tracking the mouse
movements of the learner. There is co-relation among eye ball movement and mouse movement by capturing the eye
ball movement we can find the cheating behavior.