26-07-2012, 09:57 AM
Testing Programming Aptitude
programming aptitude.pdf (Size: 82.9 KB / Downloads: 33)
Abstract.
An initial cognitive study of early learning of programming aimed to extract experimental
test data to establish novices’ understanding process has been carried out
by us [1]. This empirical study was inspired by the notion that different people bring
different patterns of knowledge in any new learning process, and demonstrated that
how each student tackles the problem in a different way based on their mental model.
The initial study suggests that success in the first stage of an introductory programming
course is predictable, by noting consistency in use of the mental models which
students apply to a basic programming problem even before they have had any contact
with programming notation, but the consistency/inconsistency measurement was
somewhat subjective. In this paper I present an objective marking method which hope
will lead us to more precise and more finely-graduated predictions. This method is being
trialed in at least one experiment, and we hope that by the time of the conference I
will be able to describe the results.
Introduction
An initial cognitive study of early learning of programming aimed to extract experimental
test data to establish novices’ understanding process is described in [1].
We believe that different people bring different patterns of knowledge to any new
learning process, and learning programming is not exempted from this common fact.
Each student tackles the problem in a different way based on their mental model.
The study started with the hypothesis that “we are able to identify small number of
groups to represent novice programmers by looking at their problem solving methods
and techniques.” We were looking for any sub-populations which are likely to achieve
success. Our intention was to observe the mental models that students used when
thinking about assignment instructions and short sequences of assignments and we
hoped to be able to find out what those models are. We administered a test at the very
beginning of their course before the students had begun to be taught about assignment
and sequence, and then a second time to the same subjects after the topic had been
taught. We correlated the results of these two administrations with each other and we
found three groups: consistent using a single mental model (44%).
Related Work
An initial cognitive study of early learning of programming aimed to extract experimental
test data to establish novices’ understanding process is described in [1].
We believe that different people bring different patterns of knowledge to any new
learning process, and learning programming is not exempted from this common fact.
Each student tackles the problem in a different way based on their mental model.
The study started with the hypothesis that “we are able to identify small number of
groups to represent novice programmers by looking at their problem solving methods
and techniques.” We were looking for any sub-populations which are likely to achieve
success. Our intention was to observe the mental models that students used when
thinking about assignment instructions and short sequences of assignments and we
hoped to be able to find out what those models are. We adminis tered a test at the very
beginning of their course before the students had begun to be taught about assignment
and sequence, and then a second time to the same subjects after the topic had been
taught. We correlated the results of these two administrations with each other and we
found three groups: consistent using a single mental model (44%), inconsistent using
several mental models (39%) and blank not answering (8%), and an apparent correlation
between the consistent group and students who successfully passed the test.
Study on Learners
An initial cognitive study of early learning of programming aimed to extract experimental
test data to establish novices’ understanding process is described in [1].
We believe that different people bring different patterns of knowledge to any new
learning process, and learning programming is not exempted from this common fact.
Each student tackles the problem in a different way based on their mental model.
The study started with the hypothesis that “we are able to identify small number of
groups to represent novice programmers by looking at their problem solving methods
and techniques.” We were looking for any sub-populations which are likely to achieve
success. Our intention was to observe the mental models that students used when
thinking about assignment instructions and short sequences of assignments and we
hoped to be able to find out what those models are. We administered a test at the very
beginning of their course before the students had begun to be taught about assignment
and sequence, and then a second time to the same subjects after the topic had been
taught. We correlated the results of these two administrations with each other and we
found three groups: consistent using a single mental model (44%), inconsistent using
several mental models (39%) and blank not answering (8%), and an apparent correlation
between the consistent group and students who successfully passed the test.
Conclusion
The result of our preliminary study demonstrated that our categorisation method is
more likely to be used as a reasonable predictor of success in introductory programming.
There were few issues of subjectivity in interpretation of our result that I have
identified and attempted to clarify. I can say that the test instruments now are highly
objectified and prepared for the next experimental phase of this study. This study also
suggests more exploration of other possible tools and instruments to capture the same
categorisation, in order to examine success prediction. I emphasis on clear understanding
of novice’s mental models that is a key element to build empirical tools to
measure programming ability.