Big (Scary) Education (Retention) Data (Surveillance)

Big data in education might be the savior of our failing learning system or the cement shoes that drags the system to the bottom of the ocean depending on who you talk to. No matter what your view of big data is, it is here and we need to pay attention to it regardless of our views.

My view? It is a mixture of extreme concern for the glaring problems mixed with hope that we can correct course on those problems and do something useful for the learners with the data.

Yesterday at LINK Lab we had a peak behind the scenes at a data collection tool that UTA is implementing. The people that run the software at UTA are good people with good intentions. I also hope they are aware of the problems already hard coded in the tool (and I suspect they are).

Big Data can definitely look scary for a lot of reasons. What we observed was mostly focused on retention (or “persistence” was the more friendly term the software uses I believe). All of the data collected basically turns students into a collection of numbers on hundreds of continuums, and then averages those numbers out to rank them on how likely they are to drop out. To some, this is scary prospect.

Another scary prospect is that there is the real danger of using that data to see which students to ignore (because they are going to stick around anyways) and which students to focus time and energy on (in order to make the university more money). This would be data as surveillance more than educational tool.

While looking at the factors in this data tool that learners are ranked by led to no surprises – we have known from research for a long time what students that “persist” do and what those that don’t “persist” do (or don’t do). The lists of “at risk” students that these factors produce will probably not be much different from the older “at risk” lists that have been around for decades. The main change will be that we will offload the process of producing those lists to the machines, and wash our hands of any bias that has always existed in producing those lists in the first place.

And I don’t want to skip over the irony of spending millions or dollars on big data to find out that “financial difficulties” are the reason that a large number of learners don’t “persist.”

The biggest concern that I see is the amount of bias being programmed into the algorithms. Even the word “persistence” implies certain sociocultural values that are not the same for all learners. Even in our short time looking around in the data collection program, I saw dozens of examples of positivist white male bias hard coded in the design.

For example, when ranking learners based on grades, one measure ranked learners in relation to the class average. Those that fell too far below the class average were seen as having one risk factor for not “persisting.” This is different than looking at just grades as a whole. If the class average is a low B but a learner has a high B, they would be above the class average and in the “okay” zone for “persistence.”

But that is not how all cultures view grades. My wife is half Indian and half Australian. We have been to India and talked to many people that were under intense stress to get the highest grades possible. It is a huge pressure for many in certain parts of that culture. But even a low A might not register as a troubling signal if the class average is much lower. But to someone that is facing intense pressure to get the best grades or else come home and work in Dad’s business… they need help.

(I am not a fan of grades myself, but this is one area that stuck out to me while poking around in the back end of the data program)

This is an important issue since UTA is designated as a Hispanic Serving Institute. We have to be careful not get into the same traps that education has fallen into for centuries related to inequalities. But as our LINK director Lisa Berry pointed out, this is also why UTA needs to dive into Big Data. If we don’t get in there with our diverse population and start breaking the algorithms to expose where they are biased, who else will?  Hopefully there are others, but the point is that we need to get in there and critically ask the hard questions, or else we run the risk of perpetuating educational inequalities (by offloading them to the machines).

For now, a good place to start is by asking the hard questions about privacy and ownership in our big data plan:

Are the students made aware that this kind of data is being collected?

If not, they need to be made aware. Everywhere that data is collected, there should be a notification.

Beyond that, are they given details on what specific data points are being collected?

If not, they need to know that as well. I would suggest a centralized ADA-compliant web page that explains every data point collected in easy to understand detail (with as many translations to other languages as possible).

Can students opt-out of data collection? What about granular control over the data that they do allow to be collected?

Students should be able to opt out of data collection. Each class or point of collection should have permissions. Beyond that, I would say they should be able to say yes or no to specific data points if they want to. Or even beyond that, what about making data collection opt-in?

Who owns the students’ data (since it is technically their actions that create the data)?

This may seem radical to some, but shouldn’t the student own their own data? If you say “no,” then they should at least have the right to access it and see what is being collected on them specifically.

Think of it this way: How will the very substantial Muslim population at UTA feel about a public school, tied to the government, collecting all of this data on them? How will our students of color feel about UTA collecting data on them while they are voicing support for Black Lives Matter? How would the child of illegal immigrants feel about each class at UTA collecting data about them that could incriminate their parents?

edugeek-journal-avatarThese issues are some of the hard things we have to wrestle with in the world of Big Data in Education. If we point it towards openness, transparency, student ownership, and helping all learners with their unique sociocultural situations, then it has potential. If not, then we run the risk of turning Big Education Data into Scary Retention Surveillance.

Matt Crosslin
Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

Pokemon Go and the Gimmickification of Education

I almost dread looking at my social media feed today. Pokemon Go (GO? G.O.? (wake me up before you) Go-Go?) received a large bit of media attention this weekend, apparently even already spawning posts about how it will revolutionize education and tweets about how we need what it produces in education:

All I could think about is: how did we get to this point? Every single tech trend turns into a gimmick to sell education mumbo jumbo kitsch tied to every cool, hip trend that pops up on the social media radar. I guess I shouldn’t been that surprised once Block-chain became educational, or Second Life was used to deliver classes, or Twitter replaced LMSs, or MySpace became the University of the future, or DVDs saved public schools, and so on and so forth. I bet at some point thousands of years ago there was a dude in white toga standing up in an agora somewhere telling Plato how chariots would revolutionize how he taught his students.

I’m all for examining new trends through an educational lens, but every time I just want to say “too far, Ed-Tech®, too far!”

We all know education needs to change. It always has been changing, it always will, and will always need to have a critical lens applied to how and why it is changing. But with every new technology trend that gets re-purposed into the next savior of education, I can’t stop this gnawing feeling that our field is becoming a big gimmick to those outside of it.

A gimmick is basically just a trick intended to attract attention. One or two seem harmless enough. Well, not that harmful? But once everything that comes down the pipe starts become this trick to get people to look at education, the gimmick gets old. People are still asking what happened to Second Life, to Google Wave, to you name the trend. After a while, they stop buying into the notion that any of us know what we are talking about. Just think of the long-term effect on the larger discourse of so many people declaring so many things to be the savior of education, only to abandon each one after a year or two.

edugeek-journal-avatarThe problem with the hype cycle of Ed-Tech is that is buries the real conversations that have been happening for a long time on whatever the hype-de-jour is. Do you want the Pokemon Go for education, where students are engaged, active, social, etc? We already have a thousand projects that have done that to some degree. Those projects just can’t get attention because everyone is saying “Pokemon Go will revolutionize education!” (well, at least those that say that un-ironically – sarcastic commentary that apparently went over many people’s head not included).

(see also “Pokemon GO is the xMOOC of Augmented Reality“)

Matt Crosslin
Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

What If The Problem Isn’t With MOOCs But Something Else?

Is this another post about how MOOCs are misunderstood ideas that the critics all get wrong? Not quite. There are problems with MOOCs, but I’m still looking at the conversation about MOOCs in general (continuing from my last post kind of). The general conversation about MOOCs (and for that matter other ed tech innovations such as flipped learning, gamification, etc) tends to be all over the place: insightful, missing the forest for the trees, really odd, kind of just there, etc. All of that is great and makes for interesting discussion. One of the concepts that seems to be getting more traction the past few weeks is “motivation.”

The article about “Why Technology Will Never Fix Education” has already been the subject of many insightful observations. I want to zoom in on one part:

The real obstacle in education remains student motivation. Especially in an age of informational abundance, getting access to knowledge isn’t the bottleneck, mustering the will to master it is. And there, for good or ill, the main carrot of a college education is the certified degree and transcript, and the main stick is social pressure.

I don’t think we can just pass over that last statement with just a simple “for good or ill.” There is a lot of “ill” with that carrot that needs to be unpacked. In an article that very correctly examines the problems of inequality in education, a huge systemic problem is skipped over.

Of course, this article is not the only one. Many other articles have pointed at “student motivation” as being a huge problems with MOOCs. MOOCs are like any other education idea: subject to good and bad instructional design. So you shouldn’t blame the overall idea when learners are just getting bored with bad instructional design. But even beyond that, the above quote speaks to how our system in the U.S. relies on motivational techniques that are predominantly extrinsic in nature. We spend decades indoctrinating learners with this context, and then when an idea comes along that relies mostly on intrinsic motivation, we blame the idea itself rather than our system.

What if MOOCs are just a mirror that shows us the sociocultural problems we don’t want to deal with in our system?

What if the problem is not with the learners, but the way they have been programmed through the years? Grades, credits, failure, tuition, fees, gold stars, extra recess for good grades, monetary rewards, etc are all programmed into learners from a young age.

You can say MOOCs are failing because they lack sufficient “student motivation,” but what if it was actually the case that society has been failing for decades and MOOCs are just exposing this?

Of course, we all recognize many ways that society is failing in education. But what if there are other ways? What if relying on too much extrinsic motivation is a failure? What if we are failing to embrace all of the current and historical research in motivation? What if we know a lot about motivation, but fail to real utilize any of that knowledge? On Twitter yesterday, Rolin Moe pointed out that he never reads discussion of Herman Witkin, cognitive styles, field dependence/independence, etc in relation to motivation. In my circles, I have heard Witkin brought up, but to be honest – I can’t recall anyone trying or applying his ideas (kind of in the same way people in education rattle off Skinner or Bandura and then just don’t really use any of their ideas). These are all ways that our educational system was failing just in the area of motivation for decades before MOOCs (or many other Ed Tech ideas) even came along.

Yet what happens is that the ideas like MOOC are blamed for the historical failure of the system, and those that feel more comfortable within that system recommend pulling the wild ideas back in to make them look more like the existing system. Just think about it: what are the recommendations for fixing “student motivation” in MOOCs? Find a way to add back extrinsic motivation!

I would say: no. We need to find a different path. In fictional entertainment, one of the foundational constructs is to reach for is “suspension of disbelief.” You have to help the readers come to a place of either gaining interest in your story or believability in the fiction elements so that they suspend skepticism and engage the story. Traditional education has typically sought for a “suspension of laziness” – looking for ways to get learners to get off their rears and learn (because we always assume that when they don’t want to learn it is their motivation instead of our design). Newer ideas like MOOCs are going past that, to what I guess could be called “suspension of extrinsic motivation” (for lack of better words). What does learning design look like when you remove all of these carrot sticks (or actual paddling sticks) and leave learners to just pure learning? Well… maybe purer learning than what we had.

edugeek-journal-avatarThere are many, many more angles to explore here (not to mention problems with extrinsic/intrinsic motivation constructs), but I am already getting long-winded. The important idea to consider is that instead of pulling emerging technology and design back towards the tradition of what we already know (which is actually a power struggle by those in power), we need to push forward towards the direction that we already know we need to go.

(image credit: Manu Mohan, obtained from freeimages.com)

Matt Crosslin
Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

More #et4online Reflections: The Major Values of Teacher Tank

So in Laura Pasqunin’s most excellent reflections on the OLC Emerging Technologies conference, she pointed out how some people had cast doubt on my favorite part of the conference: Teacher Tank. I expressed my disagreement with that, and she asked for input. My comment turned into a book, so I decided to turn it into a blog post.

First of all let me say – I realize that Teacher Tank will most likely never happen again, because it will probably chase sponsors away. Which is disappointing, but also part of the problem. The vendors have too often controlled a one-way conversation at conferences. They position themselves as the main financial support for the conferences and then people don’t want to criticize them for fear of killing the conference (I could write a series of long posts about what I was told I could and couldn’t say at conferences by vendors, how they cut-off my microphone so I couldn’t ask questions in online sessions, how they have said to rooms full of people that feedback was not allowed, etc). So it was a gutsy move to have the tank in the first place, but I get why it will probably never happen again. Doing it in another format would probably just not produce the same effect.

I will also say that I know several good vendors and start-ups that are not like the others. Unfortunately, there is noticeable “standard” type of vendor that we see dominating conferences, and those are the ones that I have concerns about.

Personally, I think that commentary and entertainment are two massive values for a conference that most are usually missing for the most part. So many conferences would be more interesting to me if they provided more commentary and even a little entertainment. Don’t get me started on the “cutesy motivational speakers as keynotes” or the “light snacks for dinner and a bunch of tables” forms of “entertainment.” We need actual entertainment sessions to prove a mental break from the sessions from time to time. We also need actual commentary from experts (in addition to thought-provoking keynotes and informative sessions) as well as mote avenues for public commentary by conference attendees.

From the audience perspective, here is the big value that I saw happening over and over again at Teacher Tank:

  • Vendor says some unfounded educational urban legend (“students love to learn with video”)
  • Half of audience nods in agreement
  • Judge pushes back against legend (“like hell they like to learn with video”)
  • Same half of audience looks shocked
  • Judge explains why the legend was wrong (“they like entertainment, but they learn little, etc”)
  • Same half of audience has an obvious light bulb moment and starts nodding again
  • Educational Urban Legend busted for many people.

That happened a lot of times for a lot of people in the audience at Teacher Tank. Huge value if you ask me.

The thing I am most disappointed in is the reaction from some of the vendors to Teacher Tank. Most of the feedback was constructive, a lot of it was positive, and some of it was negative. But it was a MASSIVE amount of feedback they got – more so than they will probably ever get from any one event. All for free, and very little of it from people that are just telling them what they want to hear. A ton of honest feedback. But now some are upset about the tank, saying things like it perpetrated “myths” about start-ups, and so on. Very disappointing.

Look, if you are a start-ups and you want to win me back, then learn to listen, research, and improve based on the feedback… rather than claiming the image I have of start-ups is mythical (while responding in a way that proves they aren’t myths :) ). Prove that you are not thin-skinned and come back to conferences that have criticized you. Maybe even turn your one-way hype session presentation into a Teacher Tank format. Why should conference organizers have to be the ones facilitating valuable honest feedback for you? I have heard about the focus groups and customer research you do – interesting results, but no where near as critical as you would get in the tank.

People always ask me to get involved in conferences more behind the scenes, and I am usually very hesitant. So many conferences today are somewhat dominated by vendors that are controlling one-way conversations (next time you are at a booth and they ask you “what do you think?”, try being honest and see how well that goes 905 of the time). To push back against that means losing sponsors and killing the conference – so I understand why conferences are like that. I don’t blame them. The Teacher Tank and the #et4snark tag were a breath of fresh air in the conference space. Most of my ideas for conferences are along those lines, and usually don’t fit in. I mean, I brought a “buzzword buzzer” to my own session and let people get on my case if I used an EdTech Urban Legend Buzzword. Judging by the Twitter feedback, people loved my session. But I doubt those ideas would go over well with vendors or even most presenters.

I get it that there is a lot of bad feedback out there. Look at some of the MOOC criticism out there (“cMOOCs probably aren’t MOOCs because they are neither massive or open” I was told at a conference recently). But many of the MOOC criticisms are legitimate points, as much as they may annoy myself or others (“most MOOCs are designed using simple and ineffective pedagogy” for example). But don’t mistake those criticisms, even the negative ones, as meaning people hate you or even want you to go away. I love WordPress, for example, but I can also give you a long list or where it needs to be improved.

edugeek-journal-avatarWhat should be more concerning is when people stop criticizing you – when they give up and just write you off as someone that doesn’t listen. I see this happening way too much in the EdTech world in regards to Tech companies. I have lost count of the number of people that have told me they have given up talking to all of the companies that just don’t listen. That should be more concerning to vendors than being a position to get a few minutes of uninterrupted feedback in the Teacher Tank.

(image credit: leenapics, obtained from freeimages.com)

Matt Crosslin
Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

The Disruption That Never Will Be in Education

Don’t get me wrong – change is coming to education, and disruption will be part of it. But all of the comparisons to the music industry are off base, because much of the “disruption caused by mp3s” narrative is a smokescreen from the music industry intended to distract from other questionable activities they are participating in. And also to quote Jim Groom: “Why are we so hell bent on disrupting everything right now?

But let’s start with a historical look at the music industry. If you are old enough, you probably remember seeing this sticker quite often:

Home_taping_is_killing_music

When the cassette tape came out, it quickly became a cheap means for creating your own tapes at home. While people like to act like the mp3 created the “unbundling/rebundling” phenomenon, the truth is that it was the mixtape that started it. Many people like to act like all they did was make a personal favorites list from their own collection, but the truth is that most of us used the mixtape to get a bunch of songs we liked from friends so that we wouldn’t have to buy a whole album for one song. Some of us even coordinated music buying with friends and family so that we could get all the songs we wanted for the least amount of money. This led to the rise of the home taping movement along with the music industry creating several PSAs about how this movement was killing their business.

Which, of course, it obviously did not.

So the ability to unbundle and rebundle music is nothing new. Neither is the ability to get free music. The same holds true in other forms of entertainment: people that didn’t want to buy newspapers knew what coffee shops to hit at what time to get a free copy. People set up elaborate systems for trading VHS and Betamax tapes. Or they learned how to tape movies off of broadcast TV once you were allowed to pause recording during commercials. The digital revolution sped this process up and anonymized it considerably, but there were actually other factors that contributed more to the disruption that occurred in the music business. Of course, you rarely hear about these because it exposes a more questionable side to the music business. Not to mention that “home burning” is probably bigger than online piracy:

“It seems the ripping of CDs borrowed from friends and family accounts for almost as much music piracy as online file sharing anyway, which is an interesting discovery. This is something that has been rife since before online piracy music became a mainstream activity.”

Remember what happened when the music industry introduced new physical formats (vinyl to 8-track to cassettes to CD)? Everyone had to spend a ton of money upgrading to the new format, because the new format was in no way compatible to the old one. Most of us had to sit around figuring out which albums we liked the most because we could only upgrade a few. Even after the CD, the industry tried to introduce new formats like Super Audio CDs and MiniDiscs, but none of those caught on. People were still trying to upgrade to CDs and just didn’t bite. But also many people noticed that the early CDs sounded horrible when compared to the new albums recorded for CDs. Remember those first Led Zeppelin CDs? It was obvious they were just dumping old music on the new format without trying to upgrade the sound quality. They weren’t expecting this CD thing to last.

Additionally, think about how flimsy all of those physical formats were. They could break, warp, scratch, crack, stretch, and wear out easily. In addition to the massive amounts of money they made off of making consumers upgrade every few years, they also made a lot of money off of people replacing broken or worn media (even CDs wear out if you play them too much).

Mp3s and cloud storage changed this. Once you get your music digitally apart from the physical media, it can always be compatible with newer formats. Look at how many formats iTunes plays. Some new format comes out? Download the update and keep going. Mp3 player breaks? Just re-download the songs.

There was one area that the digital revolution did obviously disrupt. The one thing that home taping couldn’t deal with was the need to still buy an entire album to just get the songs you wanted. Sure, there were 45s and cassingles and even CD singles, but those just had the one hit song (and a throw away song if that). Usually three of those would equal the cost of a full album, and most hit bands would have at least three hit singles. So most of us just got the album and skipped the process of waiting for singles. MP3s did change that radically, in that you could just buy the songs you like at $1 a pop and skip the rest of the filler. Because, let’s face it, most hit albums are a few good songs that are obvious singles and a bunch of boring filler. But no record company is going to point out how little effort they put into the whole album. So yes, the mp3 did disrupt the business of tricking people into buying a full album of filler in order to get the 2-4 songs that the record company spent actual time and money on developing into hit songs.

This all points to the real disruption in the music business that the industry will never mention. Some of their more lucrative side-effect revenue streams were cut off over night (upgrading old media, replacing damaged media, and buying the full media to just enjoy a small part). These disruptions will not transfer to the education sector until someone invents a way to improve the human brain. Once we “download” education, its not permanent. We will need refreshers. We will need updates. For now at least, we can’t download the new information directly to our brains once the old goes out of date. We will need to constantly learn new information and enforce existing information, so education is still needed in some form and free online content will not change that.

So, in addition to the real music-industry disruption being something that most aren’t focusing on, we also have the issue that those at the top (record companies) are still doing well despite what they are saying. The music industry still made $16.5 billion dollars in 2013. That may be half of what they made 10 years ago, but a lot of that loss can be accounted for through the loss of the “lucrative side-effect revenue streams” I mentioned. And o you really think they laid off any corporate head honchos because of those losses? Doubtful. We do know there are less artists getting signed, less music being produced by older artists, and less newer artists clogging up the airwaves. The people at the top are still making money by squeezing more out of the people at the bottom. Look at all of the hit songs that are “featuring” guest appearances from other artists. How do you increase the sales of a hit song? Get another famous person to guest on that song and all of their fans will also buy the song. Instance 2-for-1 sales bump! Sound familiar?

Of course, this is not isolated to a few colleges. Faculty around the world are reporting being required to do more with less resources and support while upper level administration seems to continue to increase.

Something else to think about. Recent research is showing that people that download the most free content illegally are also the ones that buy the most legal content. Those that already have the service being offered are the main ones that are consuming the free version of it. Sound familiar? Like how most people that take MOOCs already have a college degree?

What this points to is that any disruption that the education industry would go through in common with the music industry has already happened.

So we have a few reality check factors to consider:

  • Unbundling and rebundling is nothing new and existed well before the digital revolution
  • Access to free content also existed well before the digital revolution
  • A lot of the “disruption” that occurred in the music industry is a smoke screen from the music industry itself designed to garner support for current questionable actions as well as hide questionable practices in the past.
  • Much of the actual disruption that happened due to mp3s and digital content can’t really transfer to the education industry due to the education sector being much more complex.
  • The disruptions that can transfer from the music/entertainment industry to the education industry have already happened.

All of this to say that music metaphors need to stop. Changes and disruptions are going to happen (and have been happening for a long time), but it seems we seldom see the people that have a more realistic grasp on the changes that are coming speaking at big educational conferences. This post was originally meant to be a two or three paragraph intro to a blog post called “Ask Not What Disruption Will Do To You, But What You Can Do For the Coming Disruption” – but that will have to wait until next time. We need to stop this focus on disrupting everything now based on a busted music industry model and instead ask how we can guide the changes that are coming to be beneficial for learners and faculty and not the big dogs at the top.

Matt Crosslin
Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

Research Says: Online or Face to Face Is Better?

You know what they say about getting into an argument with an instructional designer over learning design? Oh… they don’t? Well, they should. Anyway… if they did say anything about it, they would say not to do it because instructional designers pretty much shoot holes in everything.

People argue all the time over whether online learning is better or worse than face-to-face. But you ask an instructional designer which is better? Well, neither, both, and… it kinda depends.

Confusing? Yeah, well blame the research. Research is important. Research tells us a lot. Research raises a lot of good questions. But it seems like we as the educational community are misusing and over simplifying the results of the research.

A lot of research is based on numbers. And those numbers might tell us that, say, there is a statistically significant difference between the number of learners that passed the test in the face-to-face version of a course and the number of those that passed in the online version. Or substitute “test” with whatever metric you are using to determine which is better. And so face-to-face is declared the winner and online is the loser that has to slink off and die because it *lost*!

The problem is – online learning obviously worked great for those students that passed – even if there were statistically significantly fewer of them (did I just butcher the English there?). Research is not a contest to show which option is the one right one. We are not in a giant game of Highlander: Education. There can be more than one right way. It can be online and blended and face-to-face. We are not waiting to see which one beheads all the others to become the clear champion of the universe.

So when the Department of Education came out and declared blended learning the best, that did not mean that online and face-to-face were horrible or ineffective. They just found a higher number for blended. That’s all. That doesn’t invalidate the other two. They are  a national entity that has to look at what works for millions of students.

One way that we know that online learning is working is by learner testimonials. There are thousands and thousands of learners all over the nets saying how online learning worked for them. And guess what – some of them actually failed their courses! Wait – am I telling you that scores don’t matter? Well, of course they matter if you want to earn a piece of paper. But many learners don’t look at a passed test or course as a sign of “working.” Earning an “F” in a course could mean they don’t take tests well, or they had a death in the family during the semester, or they went off on a tangent and forgot to take the final because they were too busy learning informally.

Then there is the other end of the spectrum, where students get annoyed at classes and give them bad satisfaction ratings because they were required to do actual work and they thought they should get an A just for paying for the class.

So ultimately, if a student says an online course worked for them because it challenged them to think and learn, that is good evidence that it worked. Test scores and completion rates and satisfaction surveys might also tell us something, but typically those are ranking systems and not a “winner takes all” cage matches.

But another huge problem – one that instructional designers would point out to you – is that even the best research studies cannot really tell you if online or face-to-face is better. They can compare how the learners in one type of online learning design for a specific time period performed against another group of learners in one type of face-to-face learning design for that same period. There are so many different ways to design for learning online, and there are so many different ways to design for face-to-face, and so many ways that different instructors can affect their classes, and so many ways the learner population can affect the mood of the class, and so on. Research gives us a snap shot of what is going on in specific set of classes at a specific time – but the goal should be to ponder what this means for our personal situation and adapt and experiment ourselves. Not “this works! This doesn’t” and move on.

So the instructional designer will tell you that, yes we know a whole lot about what “works” in the macro sense of education, but in a lot of ways we also know very little of what “works” also. We can tell you want generally works in online or face-to-face and what doesn’t… but it ends up being a long vague list that you still have to take a stab at to see what does and doesn’t work for you specifically.

And the kicker is – despite all the research and facts I knew when I started as an instructional designer… I didn’t really get all of this until I started teaching online. Once you start teaching yourself, and trying to actually do what the research says… you begin to realize that it’s not so black and white. There are no champions of the universe, no best practices, no learning styles, no easy categories for everything to fit in. Oh, sure – you “know” that before you start teaching, but it’s kind of like you “know” parenting is tough until you have a kid and see how tough (and wonderful) it can be for yourself. First-hand knowledge changes your perspective radically. And simplistic answers from research goes out the window. The research itself (or at least the good research) doesn’t really ever give easy answers – people just misread it and think it does. Once you start teaching yourself, you begin to realize that you will use research to inform your practice instead of dictate it.

Some day soon I hope we move beyond this pointless rhetoric about online or face-to-face or blended learning being better or a good way to learn or whatever. All education is distributed over a distance anyways. Learners have declared that all work for them. Its better to start looking at what worked or didn’t work for the learners and go from there. That might call for some – gasp – qualitative research!

“So okay, Matt, stop with the whole ‘there is no spoon BS’ and tell me straight – does online learning work or not” you might say. Online learning works – for certain students. What all of the research is really telling us is that what doesn’t work is forcing all students into one-size fits all learning designs. Therefore, that leads me back to why I like working with the dual layer MOOC group – how can we offer students options to determine for themselves what works best? How can we create multiple paths that are truly multiple paths and not just “five different version of the same silo”? How can we create learning designs that emphasize diversity, experience, and autonomy in learning? Especially when so many students are used to instructivist learning?

Matt Crosslin
Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

When Hype Eats the Real Concept

A co-worker emailed me the other day and asked if I had heard of “Online-Only Flipped Classrooms.” After discussing it with him, it seems like this is a real  thing. But it also seems that every reference I can find to “Online-Only Flipped Classrooms” really just describes what we used to just call “online learning”  less than 10 years ago.

Great.

You know that the hype around Flipped Classrooms and MOOCs has gained a life of its own when people start writing books about “new directions” for those concepts and don’t even realize they are just describing basic online learning. Think about it: you watch a video or read some text and then come together to discuss or work on a group project. That has been basic online learning for centuries. But now it is being called “Online-Only Flipped Classrooms.” Oh, and not to mention it is labeled as “student-centered learning.”

I guess that is another post – at what point did “making your students work together for the majority of the time” become synonymous with “student-centered learning”?

Matt Crosslin
Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

But The Algorithm Said I Passed!

Many people have noticed a growing focus on automation in learning, especially around the idea of computer-graded assignments. Not just telling students that they picked the right answer on a multiple choice test, but the actual grading of term papers based on complex algorithms. EdX among others are working on systems that will grade thousands of student submissions based on what it thinks the instructor would have given the students.

Some love this idea, some are creeped out.

Students seem to love the idea of removing instructor bias from the grading equation. Or do they just love the idea that they can learn to game the algorithms? We will see in the future, of course.

But at some point, how do we know that learners have actually mastered anything if there is no intelligence in the process that really understands what the student is trying to communicate. After all, if there is anything to all of this social constructivism or connectivism stuff… what happens when one part of the equation is not really intelligent or alive and therefore not social?

Well, you might say, some day the program will get complex enough that computers will have artificial intelligence. The problem with that is, in order to have true intelligence, you have to have a bias of some kind. If someone puts your life on the line versus another person that you don’t know, you will probably fight to live. That is a bias. Or maybe you will take the high road and put the other person’s life ahead of yours. That is another bias. If a machine can not make a choice between preserving itself or thinking of others first, it is following what it was programmed to do and is not truly intelligent. And even worse, it means that it had a certain bias programmed into it by its creator.

All of this might not phase a pure empiricist/behaviorist at all. But to those of us that subscribe to anything from pragmatism to constructivism to connectivism, there are huge problems even if programmers can in some way figure out the perfect algorithm. If one side of the equation is not really intelligent – how can learning really be occurring? Even if you are a cognitivist at heart… how do you know that the computer program with the grading algorithm is compatible with the human computer we call a brain? Or how do you know that the organic brain didn’t just find a way to game the digital one? Will we really be able to create programs that see past the elaborate smoke screen that many humans are known to create?

Matt Crosslin
Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

Opinion Polls and the Continuing Mission of Ed Tech

So the news today is that opinion polls found that the general population still finds traditional classes to be better than online ones in a few key areas. I find polls like these fascinating, but not because I think they spell trouble for online learning. These are opinion polls and therefore don’t necessarily spell out fact. What they do reflect is what message is most often heard. And what is currently dominating the message? xMOOCs. So you are probably dealing with some misunderstood terms here.

Anyone in online learning knows that online courses can be just as good or bad at things like “instruction tailored to each individual,” providing “high-quality instruction from well-qualified instructors,” or offering “rigorous testing and grading that can be trusted.” The reality is that those are all factors that depend more on design and instructors than whether something is online or not. Personally I have never been in a face-to-face class that allowed for “instruction tailored to each individual.” But I know many online programs that do. So I am making a case that individual perception is biased – so what?

The so what part is that we still need the evangelists out there, spread the good news of online learning. Not as the only method of learning for the future, but as one that is every bit as equal to face-to-face learning. I think many of us just sit back and think the case has been made and the battle has been won, when polls like this one still show some ground to be gained.

And did anyone else notice that this poll found a much higher satisfaction rate with college in general than the “death of the university” crowd has been preaching? I may to hold off my early retirement plans if this whole university thing survives….

Matt Crosslin
Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

Statistics, Analytics, Program Evaluation, and the Great Sell-Out

A few years ago, people were signalling the death of the University because new statistics proved the jig was up. Of course, when you hear things like “Thirty-six percent of the students saw no statistically significant gains in their CLA (Collegiate Learning Assessment) scores between their freshman and senior years”, it does sound pretty bad. The CLA is a “widely-used essay test that measures reasoning and writing skills.” So, in other words, college students are not learning to read or write.

But what if we are looking at the numbers in the wrong way?

Take this real life scenario into consideration: a certain college near where I work increased the student body population from 10,000 students to 25,000 students in less than two decades. A 250% increase. Now consider this: has the quality of the average high school graduate really increased by 250% over the past two decades? Has the quality of the average high school graduate really increased at all? Or have colleges lowered their entrance standards? You will find similar statistics in many of your “growing” colleges and universities today. If you do the math, 60% of those students may have not been ready at all for college when they entered.

I’m not trying to say that they shouldn’t be there – I want to see people educated. But in light of the consideration that maybe up to (at least) 60% of the students in any given college may not have even been prepared to be there, a statistic that says that 36% of them didn’t show statistically significant gains might be a small miracle.

Or maybe not – but if we aren’t looking at bigger picture factors in all this data we are gathering… how do we really know what we are looking at?

I know this is old news to people that are really into research and data… but some of this still seems to be shocking people out there. Today we read that “Students Might Not Be ‘Academically Adrift’ After All, Study Finds.” One of the many interesting points in this article is that the authors’ of Academically Adrift might have been incorrectly “translated by some people in politics to say, ‘College doesn’t matter.'”

Some people? Try a lot of people in key positions. People that are now rushing into certain crazes without having a bit of evidence that they work. Which is not a bad idea at an experimental level, but when entire degrees and millions of dollars are thrown at untested ideas just because people didn’t take a more nuanced look at the numbers? That is very dangerous territory for a field that is already teetering on the edge of obscurity, unable to afford another big blow to credibility.

The time of “chilling out” and just “being happy that there is attention to new ideas” is long past. The whole idea of “don’t look behind the curtain, don’t think critically, just ignore the negative because we all need to be shiny happy people that hold hands and sing kumbaya” never really worked for, well, anyone. It sounds cool to say “don’t bash the movement, just move it in the right direction using positive energy” before the sell-out happens… but it is impossible to accomplish that (and it sounds down-right Great Wizard of Oz-ish to say it) once the sell-out has already happened. And I just don’t see how anyone can look at education now and not see that the Great Sell-Out has already started to happen en masse.

I get that people don’t like that the haters are now tossing out the baby with the bathwater.. but is the only other option to keep the bathwater because we like the baby?

Matt Crosslin
Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.