Measuring student learning

JOHNSON CITY (April 20, 2018) – An East Tennessee State University faculty member is working to develop a new method that could change the way educators measure, track and predict student success.

Dr. Matthew McBee, an associate professor in the ETSU Department of Psychology, is working on a project, “Developing and Validating a Mathematical Theoretical Model of Academic Achievement.”  His work is supported by grant funding from ETSU’s Research Development Committee.

McBee explains that most existing methods of looking at student learning are qualitative and rely on verbal descriptions.

“Most of the research in education that has looked at how students acquire knowledge or skills over time, even though it’s based on complicated statistical models, is fundamentally descriptive,” he said.  “Essentially, we look at how students’ scores increase over time in a subject like reading or math, and then describe patterns of change.  We can measure and find different patterns that help us understand which students are likely to thrive in which ways.”

McBee cites, for example, the “zone of proximal development” concept, which he says is commonly attributed to Lev Vygotsky but was actually proposed by Dorothy McCarthy.  According to this theory, students do not learn best when being taught what they already know, because it is too easy, or when they are taught at a level at which they cannot achieve even with help, because it is too hard; they learn best when taught at a level just above what they can do without assistance.

However, McBee says such descriptive models only go so far in helping educators understand how the learning process actually happens.  “At some point,” he said, “you need to construct a model that says, ‘Here’s how we think it happens,’ and then you need to see if that model can actually generate behavior that matches the reality.”

That is exactly what McBee hopes to accomplish with his project, which he is conducting with the aid of graduate assistant Natasha Godkin, a Clarksville native and Austin Peay State University graduate who is currently pursuing a Ph.D. in psychology at ETSU with a concentration in experimental psychology.

“What this project is about is taking that theory and transforming it into a mathematical model, and then using computer software that we wrote to essentially create simulated educational growth,” McBee said.  “This is kind of how climate models work.  You have a climate model based on a bunch of math and interactive elements, and you get a simulated weather pattern.  Then you can look and see if it matches reality as a way of validating the model.

“That’s essentially what we’re doing here so we can generate synthetic achievement growth from this theoretical model and see how closely it matches actual educational data.  That’s the project in a nutshell – to try to move beyond basic description of educational achievement and get a formal mathematical model for it that makes specific predictions and can be evaluated the way scientific theories are evaluated.  The model will make specific predictions about different patterns that should be observed in the real data, if the model’s correct.”

McBee and Godkin are now checking to see if those patterns are found in the data, and he says that everything they’ve found so far is promising.

McBee says his model is based on four underlying variables: the child’s basic learning rate, the educational quality of the child’s home environment, how quickly the child forgets things learned, and the types of educational experiences to which the child is exposed in school.

“I think if this works, there are huge implications for the field of education,” he said.  “What we’re finding right now is that we can actually create very realistic growth trajectories that are just coming out of these four variables.  That’s crazy, because right now we think education’s massively complicated, and we have to account for hundreds and hundreds of factors.  It’s almost hopelessly complex.  What we’re seeing in this simulation, at least, is you can get these dynamics out of really simple systems that only have a few variables in them.”

If successful, this project could help researchers make more accurate predictions regarding educational interventions at the individual student level, McBee says, “instead of just twiddling knobs and seeing how things change in retrospect.”

“That’s really what science is supposed to be about – improving our ability to make predictions, and so that’s what this project is all about,” he said.  “It’s really hard to make predictions from verbal or qualitative theories, because what you really need to make a prediction is to put some numbers in and get some numbers out.  How else are you going to know how accurate your predications were?”