User Constructs in e-Learning

The reason I was in Beijing in the first place was to present a paper at the Computers and Advanced Technology in Education Conference, CATE 2007. The paper was titled “The Impact of User Constructs on e-Learning Effectiveness Using the Repertory Grid”, reporting on an experiment that one of my students did. It was to study the frame of mind of users of e-Learning systems, i.e., how do users perceive the different functions provided by such systems? Why do they like or dislike certain functions?

We showed the users various items in combinations of 3 at a time, and ask them which 2 among the 3 are most alike and which one is most different from the other two. Using this method we tried to understand the criteria/dimension (construct) that the user employs in the evaluation of the items. For example, a user may find car and train to be alike because they are machines, while horse is an animal. Another user may find horse and car to be alike in terms of carrying capacity, while train carries much more. By showing the functions provided by e-Learning systems in triples to the users, and studying their responses, we hoped to understand the “constructs” that the users use to interpret/evaluate the functions provided. Each construct represents a possible “dimension” in which a function can be evaluated.

In our experiment we found 11 such constructs, and we further consolidated such constructs into 2 major dimensions. One of which is “more focused and consolidating knowledge” vs “not stimulated, motivated, and boring, useless”. Another is “help to explain problems and through interacting” vs “have to take initiative to solve problems, not helping as there is no interaction/stimulation”.

Finally the common functions provided by e-Learning systems such as WebCT are evaluated by placing them in a Cartesian coordinate system formed by these two dimensions. According to our study, the best function is the bulletin board/discussion forum. The scale of this project is too small to draw much conclusions from. But it does demonstrate one way to study systematically the “usability” of e-Learning systems, and in fact, information systems in general.