Statistics, Analytics, Program Evaluation, and the Great Sell-Out

A few years ago, people were signalling the death of the University because new statistics proved the jig was up. Of course, when you hear things like “Thirty-six percent of the students saw no statistically significant gains in their CLA (Collegiate Learning Assessment) scores between their freshman and senior years”, it does sound pretty bad. The CLA is a “widely-used essay test that measures reasoning and writing skills.” So, in other words, college students are not learning to read or write.

But what if we are looking at the numbers in the wrong way?

Take this real life scenario into consideration: a certain college near where I work increased the student body population from 10,000 students to 25,000 students in less than two decades. A 250% increase. Now consider this: has the quality of the average high school graduate really increased by 250% over the past two decades? Has the quality of the average high school graduate really increased at all? Or have colleges lowered their entrance standards? You will find similar statistics in many of your “growing” colleges and universities today. If you do the math, 60% of those students may have not been ready at all for college when they entered.

I’m not trying to say that they shouldn’t be there – I want to see people educated. But in light of the consideration that maybe up to (at least) 60% of the students in any given college may not have even been prepared to be there, a statistic that says that 36% of them didn’t show statistically significant gains might be a small miracle.

Or maybe not – but if we aren’t looking at bigger picture factors in all this data we are gathering… how do we really know what we are looking at?

I know this is old news to people that are really into research and data… but some of this still seems to be shocking people out there. Today we read that “Students Might Not Be ‘Academically Adrift’ After All, Study Finds.” One of the many interesting points in this article is that the authors’ of Academically Adrift might have been incorrectly “translated by some people in politics to say, ‘College doesn’t matter.'”

Some people? Try a lot of people in key positions. People that are now rushing into certain crazes without having a bit of evidence that they work. Which is not a bad idea at an experimental level, but when entire degrees and millions of dollars are thrown at untested ideas just because people didn’t take a more nuanced look at the numbers? That is very dangerous territory for a field that is already teetering on the edge of obscurity, unable to afford another big blow to credibility.

The time of “chilling out” and just “being happy that there is attention to new ideas” is long past. The whole idea of “don’t look behind the curtain, don’t think critically, just ignore the negative because we all need to be shiny happy people that hold hands and sing kumbaya” never really worked for, well, anyone. It sounds cool to say “don’t bash the movement, just move it in the right direction using positive energy” before the sell-out happens… but it is impossible to accomplish that (and it sounds down-right Great Wizard of Oz-ish to say it) once the sell-out has already happened. And I just don’t see how anyone can look at education now and not see that the Great Sell-Out has already started to happen en masse.

I get that people don’t like that the haters are now tossing out the baby with the bathwater.. but is the only other option to keep the bathwater because we like the baby?

Matt Crosslin
Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

Leave a Reply

Your email address will not be published. Required fields are marked *