Tracking Eye Movements of Students Solving Kinematic Problems with a Visual Component
By Nicholas C. Oderio
Supervisor: Dr. Sanjay Rebello
Kansas State University Physics Department REU Program
This program is funded by the National Science Foundation through grant number PHY-1157044. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Welcome to my webpage. This
page summarizes my experience doing research where it is trying to be
determined whether information presented using multimedia will help
introductory physics students learn how to solve kinematics problems that have
a visual component such as a graph. It has been found that upper-level students
across the country at various institutions often have not truly grasped the
fundamental laws of physics as they should. One way to prevent this is by
teaching the core ideas of physics more effectively at the introductory level.
The research group I was a part of conducted an experiment on
introductory-level physics students at K-State to see if example problems
presented with multimedia were more effective in teaching students to solve
kinematics problems than traditional static examples commonly found in
textbooks. There are a few potential practical applications of the research and
are not difficult to foresee. High school teachers, professors, TAs, tutors,
etc. could all benefit from this research. In certain instances it was found to
be much more beneficial to use a multimedia example to show how to solve a
problem rather than something static like in a textbook. This seems like
something the occupations previously mentioned would want to incorporate into
lectures and review sessions. Furthermore, textbooks with solutions manuals
could offer a CD or DVD that has solutions using multimedia. Going one step further,
educational games that can be played on electronic devices could be developed.
For instance, an iPhone application or a computer game could be created so that
this new format of showing an example is not restricted to the classroom.
Below, I give an overview of my project and links to various parts of my research, including the final design, the problems used in the experiment, data collected, and what still needs to be done with this project. I will have a link to my final presentation PowerPoint file as well. I have also included a brief description of the lectures given during the REU program and break them down by professor. At the bottom of the page, I list some fun activities that were offered by the REU program that I participated in so as to give a more complete description of the REU program.
Acknowledgments: I would to thank Dr. Sanjay Rebello for advising me throughout my time at Kansas State University. I would like to also thank Dr. Neelam Khan and Elise Agra for graciously accepting me into their research group and helping to provide valuable research experience, as well as Amy Rouinfar for helping our research group whenever necessary. Lastly, I want to thank Dr. Larry Weaver and Dr. Kristan Corwin for the opportunity to do research at K-State and providing a multitude of activities I may otherwise have never experienced.
Research
Project Overview: The project I am a part of was done within the PER (Physics Education Research) department under the supervision of Dr. Sanjay Rebello, as previously mentioned. I worked in collaboration with graduate students Elise Agra and Amy Rouinfar as well as visiting professor Dr. Neelam Khan. The purpose of our experiment was to present introductory level physics students, that is, students enrolled in General Physics I or II or Engineering Physics I as a Summer course at K-State, with problems that include a visual component such as a graph. In the case of our experiment, students were asked to solve problems that dealt with graphs representing the motion of an object. Each problem had two graphs and information from both graphs needed to be utilized in order to solve the problem. The exact sequence of problems can be found in the link under Research Description. At the end of this section, I discuss some potential applications of this research as I see it.
In performing the experiment, students were sat in a room and given an explanation as to what the experiment would consist of (ten problems with a visual component where some were worked out examples and some needed to be solved) and why we were conducting the experiment. The participants were audio and video recorded for the duration of the experiment after signing an informed consent release form. For the problems that needed to be solved, participants were asked to look at the problem and think about how they would solve it, and then explain their answer to us when they were ready. Two types of data were collected during the experiment: correctness data and eye movement data. Correctness data was collected by the proctor (Elise, Neelam, or myself) by taking notes on a data sheet as well as providing the participant with a printed handout of the problem and asking them to list all steps and relevant equations in their solution. The eye movement data was collected using an eye tracker and recording the eye movement of each participant for every problem. It was noted if a participant wore glasses or had contacts in, as some eye movement data was skewed or lost due to such factors. The eye tracker was recalibrated after every problem. The raw eye movement data was analyzed using a C++ program written by graduate student Amy Rouinfar but was not fully analyzed and incorporated into the final results by the time I left Kansas State University.
Research Description: To see the final research design, click here.
To see the PowerPoint file containing the actual
problems used in the study, click here.
Note: the slides in the file with the actual problems that have only an “X” on them were either for data collection purposes to distinguish when the participant was formulating their solution and when they were verbally explaining their solution or to pause and recalibrate the eye tracker.
Research Progress and Results: Here are some of the data collected. Correctness data was analyzed using a chi squared test. It was found that for the static problems, there was only a significant difference in correctness when answering the far transfer problem and not the near transfer problem. Within the animated set, there was a significant difference between the initial problem and both of the transfer problems. When comparing problems by their type, such as initial-to-initial, the only significant difference was in comparing the static and animated near transfer problems. What may be inferred from this data is that animated examples of problems yield an improvement of performance that is significant for both types of transfer problems and in the case of the near transfer problem, or a problem very similar to the example, is much more effective than a static example.
Upon analyzing the eye movement data, it was found that when compared to the very first problem or initial static problem, students always looked at areas of interest more than before being given any training problems. However, several types of tests were still being run in order to analyze eye movement data in a way that incorporates it with the correctness data (see the ‘Future Work’ section).
Below are links to see a sample of what output data from raw eye movement tracking data looks like, as well as data representing the correctness of the solutions provided by the students.
Click here to see output data from raw eye movements
Click here to see the correctness data (pdf format)
Future Work: At the time of my departure from KSU, eye movement data was still being analyzed and interpreted to be integrated with correctness data. There are a few ways in which our group was able to analyze eye movement data and in order to determine the best option it was decided that we run a few of the analyses. Eye movement analysis may also be done a second time using start and stop times for the verbal explanation of the answer participants provided. For the data presented here, the times used were for the thinking period in which participants formulated their response.
As with any research, most people want to know how the results apply to them in everyday life. Here, the most obvious application is teaching tools. High school teachers, professors, TAs, tutors, etc. could all benefit from this research. In certain instances it is much more beneficial to use a multimedia example to show how to solve a problem rather than something static like in a textbook. This seems like something the occupations previously mentioned would want to incorporate into lectures and review sessions. Furthermore, textbooks with solutions manuals could offer a CD or DVD that has solutions using multimedia. Going one step further, educational games that can be played on electronic devices could be developed. For instance, an iPhone application or a computer game could be created so that this new format of showing an example is not restricted to the classroom. So even though the information gathered from this study seems most directly employable in classrooms, there is room for other disciplines to make use of the results.
Final Presentation: Click here to download my presentation in PowerPoint format
Lectures
Throughout the summer, the students in the Physics REU were provided with
a series of physics lectures pertaining to various projects being worked on and
topics of choice. We also attended a weekly ethics lecture which focused on ethical
issues in science. Below is a brief description of some of the topics lectured
on and the professor(s) who gave them.
Physics
Dr. Larry Weaver: Dr. Weaver gave a majority of the lectures, and spoke on various topics within physics including but not limited to energy scales, light scattering, quantum mechanics, and nuclear physics. However, my personal favorite lecture was when Dr. Weaver related special relativity with quantum mechanics to show why antimatter exists.
Dr. Brian Washburn: Dr. Washburn provided three 2-hour long lectures on electronics and allowed us to build circuits using components discussed. The three main topics covered were voltage dividers, RC filters, and operational amplifiers. My favorite electronics lecture was learning about voltage dividers because not only are they very useful, but we were able to solder our circuits together, which I had not done before.
Dr. Bharat Ratra: Dr. Ratra works in the cosmology department and was a guest lecturer. He lectured on the development of the universe and why dark energy must exist.
In total, there were nine ethics lectures, which were split up between
Dr. Bruce Glymour and Dr. Amy Lara. Below is a
description of what each professor covered in their lectures.
Ethics lectures also included weekly assignments.
Dr. Bruce Glymour: Dr. Glymour lectured the first half of the
summer. The topics he covered include ethical theory, co-authorship, fraudulent
practices and data manipulation, communicating science to the public, and dangerous
science. Most of Dr. Glymour’s assignments were
reading for the following week’s lecture, although some prompted us to
formulate responses to potential situations we may face as scientists.
Dr. Amy Lara: Dr. Lara
lectured the second half of the summer. The topics she covered include
dangerous science, research misconduct, diversity in science, student-mentor
relationships in graduate school, and case studies. Dr. Lara’s assignments
involved reading for following lectures, creating case studies, and taking an
IAT test.
In addition to doing research and
attending lectures, we were given ample opportunity to enjoy summer. The REU
allowed us to take part in many fun events, such as cook outs and picnics with
university faculty, a trip to Hutchinson, KS, where we toured a salt mine and
the Cosmosphere, and a canoe trip led by a member of
the physics faculty.
About Me: My name is Nick Oderio and I grew up in Romeoville, IL, which is a suburb of Chicago. I am a student at Adrian College which is located in Adrian, MI. My major is physics and I have strong secondary interests in mathematics and philosophy. In my spare time I tutor other students, play drums, volunteer at local Humane Societies, play sports (mostly volleyball), and read.
Useful Links: