To Grade or Not to Grade: Does the Learner Get a Say?

Grading has been a contentious topic in some circles of education for decades now. Many outside of those circles seem to accept grading as either a good thing at best, or a necessary evil at worst, all the while never questioning if it is possible to not grade in the first place.

For me, I have found grades problematic for a long time. I used to be firmly against them, but years of teaching has changed my status from “against always” to “still not a fan, but it’s complicated.” What exactly has been complicating my views? Talking to students about grades.

I used to teach various undergrad instructional design courses at the University of Texas at Brownsville (before it merged to become UT Rio Grande Valley). We would have many online discussions about all kinds of education topics in these courses. I often found out how much I had framed my progressive / connectivist / critical lens of instructional design through things that looked good to me as a white person (as well as a male) without considering how those stances looked to people of color.

My anti-grading stance was one of these lens.

So I want to try and capture the overarching conversations and issues that were brought to my attention through these conversations with students and many others through the years.

When educators say “grades are bad,” we often offer a “conversational” approach to grading as an alternative. This typically means that instructors should have a dialogue with learners to give them feedback instead of grades, or at least let students come up with their own grade. Of course, this sounds great when you are white (or male) and you will be having this conversation with someone that probably looks a lot like you. But how does this “conversation” sound to an immigrant to this country? What about a first generation college student that is new to the whole system? What about a woman in a field that is typically dominated by men? What about a person of color at a predominately white university? Would they feel comfortable with having their success in the course rely on a conversation with someone that might not realize their own prejudices?

This is an area where we need to put our own anecdotes aside. Sure, students might be comfortable conversing with you… but what about all instructors in your department? All professors at your university? All teachers anywhere?

Sure, my white privilege makes having a conversation in place of a grade sound great… because my privilege will help me get the upper hand in that conversation more often than not.

And then, even in a situation where you have a white male student and a white male teacher, the teacher is still in the position of authority… and not everyone feels comfortable with trusting all positions of authority. Often for good reason.

We keep talking about how grading is bad because we need to learn to trust students, but how dangerous is it to replace grading with something else that requires all students to trust all teachers at all times? What does it mean to force them to trust all of us all the time?

So does any of this make grading “good” by comparison? No, it still doesn’t. As many of my students pointed out, they still recognize that tests, papers, assignments, etc have problems, bias, and inequalities. But with a multiple choice question, you know the right answer is there. With a complex rubric, you know exactly what you need to do to get a good grade. Those questions and rubrics are usually problematic, but at least the answer is there – and you just have to learn the game to get the grade.

With a “conversation” about a grade – you don’t know the rules going in. You don’t know which teachers are harboring unacknowledged racism or sexism that will skew the conversation and hold you down. At least with a graded test, if they give you a lower grade because of racism and sexism, you can point easily to that corruption and say “I chose the right answer and you still marked it wrong!” or “the rubric line says 1000 words and I had 1112 and still got points taken off!”

As many students explained to me in many different ways: even though the white dudes will have advantages and privileges with many tests and assessments, those assessments are so cut and dried that there is a clear path for everyone to achieve equality (even if it comes at a greater price for people of color than for the white students). For some, a clear – but more difficult – path to equality can be more attractive than an unclear, unknown path through potentially dangerous conversations. Of course, not all found it to be more attractive, but it is certainly attractive to some. Those students are important voices to listen to.

Then, of course, there are the students that say they are used to grades, so they prefer to have them. They will even go as far to say they don’t see the harm in grades. Not a “good” reason to me… but who am I to say that is the wrong? Research backs me up? Or does it?

Let’s face it – it is really hard to research the impact of grades themselves apart from student opinion. How do we know that we are researching the grades themselves and not the assignment that produces the grades? How do we know that we are looking at grades and not bad assessment design, or even poor teaching of the topic being assessed? Or motivation, or privilege, or tardiness, or…. any number of issues that a “grade” could reflect? The biggest problem with grades is that so many factors go into them that we have a hard time really telling if the grades are the problem or if one of many factors that produced the grade is the real problem.

So that leaves us with looking at things like “student satisfaction” and “self-reported motivation” to tell if grades are a problem or not – which is not a bad thing. But once you go down that path, you have to be careful not to look at the results the wrong way. Too many times, educators see a study and assume the results mean that we know the one solution for all learners at all times: “this study found that group work is good, so let’s make all learners do group work!” Well, the study probably found something like “test scores went up 31% when group work was utilized” or “45% of students indicated greater whatever on such and such survey when group work was utilized.” Studies show averages and statistics – not something that is true for all learners at all times. Research is helpful, but rarely answers one solution for all learners. Educators have to rely on their judgement when making decisions on grading.

Therefore, whether we as teachers choose to grade or not to grade, we are usually centering the decision to be graded on our bias for or against grading, regardless of whether individual learners want it or not.

(What I say is that we need systems that allow for learners to choose how they get graded and by whom…. something we are working on with self-mapped learning pathways (aka dual-layer, customizable modalities, and all the other terms I can’t decide on… :) ), but that is post for another day.)

To be clear, I still find grades incredibly problematic. When I read a post like “Why I Don’t Grade” by Jesse Stommel, I completely agree. I also recognize that there are complications and other sides to many of these points as well. Just as a quick example of what I mean, I want to run through the graphic from this post that has probably been shared the most – not to disagree, but to extend the conversation (probably while repeating some of what was said in this post and others out there – apologies for that):

For example, its true that grades are not good feedback. This is probably because I can’t find much evidence they were ever meant to be “feedback” per se, at least initially. It seems that grades and feedback were meant to be separate but complimentary ideas at one time. The further back in time you go, the more likely you are to see grade-books and graded papers with separate feedback columns. But as society as devalued and de-funded education, you see grades replacing feedback as a means to cope with what society is doing to education. So is that really a problem with grades, or society? I would have to honestly point more at society.

Also, I completely agree that grades make a horrible incentive. They never encouraged me to study harder or lesser. But talk about a can of worms there – try discussing what is an incentive? with a diverse group of students, and you might end up with a heated argument. Everyone seems to find incentive in different things. Its a pretty relative thing in some ways (not in others, of course). But do some students really, truly feel that grades are a positive incentive for them? You bet. Do I disagree? Yes. But if I make my stance on incentive the default one for the entire class, what does that mean about my class? Who is the center then?

For the next one, again I agree that grades are not good markers of learning. But that is because they never really were meant to be markers of learning. Again, you go back in time, there was a greater understanding that grades were a representation of a learner’s ability to apply what they learned to an assessment or paper or what have you. But not as an actual marker of actual learning. There was a greater understanding (at least in older books) that learning was difficult to measure, and a grade was only evaluating the application of learning to an assignment or test. Through the years, many have lost sight of this. Many who champion grades have lost sight of this. Many administrators and policymakers and news-makers and so on have lost sight of this. Again, not the fault of grades but society.

This reasoning also applies to how grades are not reflective of the idiosyncratic, subjective, and emotional character of learning. I completely agree, but again, that is because grades are not supposed to reflect that. They just reflect how learners can apply the idiosyncratic, subjective, and emotional aspects of learning to specific tasks, assignments, assessment, and so on. Again, much of society has just lost sight of that, there by devaluing those aspects of learning to the point of barely acknowledging them in so many corners.

And then, I agree that we see a lot of competitiveness over grades – just look at the news. This is not a good thing. But where does this competitiveness come from? Most grades are based on a scale of 0-100% for a reason: they are supposed to reflect how close an individual learner got to perfecting the graded task (a problematic statement of it’s own, of course). Grades only become competitive when we put one learner’s grades next to another and compare them. That is another societal thing, and it is one of many major issues threatening education on all fronts. I know some people actually like that kind of competition, and I try to be understanding of that mindset. But I think that we can see so many quantitative and qualitative effects of that competition currently consuming education, that we can at least say that we should at least massively dial it back. But again, that is really a societal thing more than a grade thing.

On to the last one. This point is one that gets at the common argument in support of grades: objectivity. This is another one that is a huge can of worms once you get discussing it with students, as there are so many versions of “what counts as fair.” Some see grades as a way to make assessment fair, others see the problems with grading as making that fairness impossible. Really, when it comes down to it, our bias and other factors influence most arguments for or against various definitions of fair. So, unfortunately, the best we can usually come up with is “I think this is fair because I think it is so, and I think that is unfair because I think it is so as well.” We can lean on societal definitions and social contract, but history has proven that is not always the morally best option. Few or our positions are very defensible either way if we were to argue our case before an impartial observer.

However, I think within the “fairness” argument is a way to frame grades that objectively encapsulates one argument against grades that can’t be influenced by bias or context or the whims of society. One of the reasons grades are not fair is that a grade, by itself, does not tell you anything about how the learner earned that grade. You see an 80 – does that mean it was a perfect score that had 20 points taken off for being late, or a slightly above effort turned in on time by the learner? There is no way of knowing either way just by looking at a grade by itself. Educators mix so many things into grades – punctuality, following directions, context, format, etc – that by the end, any single grade is a reflection of all kinds of things in addition to how well the learner could apply their learning to an assignment. Then, if one adds up all the assignments in one class into an “A, B, C, D, E, F” grade… and then, add up all the class grades into a GPA… you end up with a number that really tells you nothing about the exact way that grade was determined. Whether you label that as fair or not, it is still a huge issue.

Of course, one could say that this issue is easily solved if we just add the ability for every instructor to input qualitative feedback to each grade. Have the instructor tell why that grade happened, and problem solved, right? Very true, but you would then have just eliminated the need for grades for anything other than competition – and no one cares about that competition after you have graduated. They only care about GPAs as a proxy for that qualitative feedback. But if they have it….

In other words, giving the explanation for why someone got a specific grade ultimately negates the need to have the grade listed in the first place.

When one looks at all of my reasons and others’ reasons for not liking grades, so many of them can be written off by the “pro-grade side” (if that really is such a thing) as bias, taking grades out of context, historical misunderstanding, applying relativistic standards, or blaming grades for societal problems. And I still stick with my views on grades even though all of this is true, because you can’t easily separate all of that out even if attempting to “reform” or “fix” or “reclaim” grading. But at the end of all that, I still come down to the fact that a grade by itself doesn’t explain what it really means… and adding that meaning removes the need for the grade in the first place in the bigger picture of what really matters in education (because even if you like competition, you have to recognize it is not what really matters in the grand scheme of education). The main thing that we can do to best fix grades would make them obsolete, and that should say a lot.

Matt Crosslin

Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

Can the Student Innovate? An #OLCInnovate Reflection

The 2nd OLC Innovate conference is now over. I am sure there will be many reflections out there on various aspects of the conference. I hope to get to reflect on my presentation on learning pathways and some of the ideas that attendees shared. But I wanted to first dig into one of the more problematic aspects of the conferee: the place and role of students.

The biggest problem related to students at the conference was how they were framed as cheaters at every turn. Chris Gilliard wrote a blog post that explores this aspect in depth. I was able to finally meet and hang out with Chris and many others at Innovate. Those of us that got to hang out with Chris got to hear him pondering these issues, and his blog post makes a great summary of those ponderings.

The other student issue I wanted to reflect was also part of what Chris pondered at the conference as well:

Of course, as soon as I tweeted that, we found there were a few sessions that had students there. But for the most part, the student voice was missing at OLC Innovate (like most conferences).

At some levels, I know how difficult it is to get students at conferences. Even giving them a discounted or free registration doesn’t help them with expensive hotel or travel costs. Sponsoring those costs doesn’t help them get a week off from class or work or both to attend. Its a daunting thing to coordinate. But considering the thousands of attendees at OLC Innovate representing tens or hundreds of thousands of learners out there, surely some effort to find the money would have brought in a good number if the effort had been there.

But beyond that, it seemed that in many places the whole idea of students even being able to “innovate” was left out of some definitions of innovation. Not all, of course. Rolin Moe brought his Innovation Installation back to OLC Innovate, which served as a welcome space to explore and ponder the difficulties in defining “innovation” (those pesky-post modernists always wanting us to “deconstruct” everything….) Rolin did an excellent job of looking at situating the definition of innovation as an open dialogue – a model I wish more would follow:

The definitions of innovation became problematic in the sessions and keynotes. The one that really became the most problematic was this quote from one keynote:

https://twitter.com/mrkampmann/statuses/850107391387611136

(I am also not a fan of the term “wicked problems”)

The context for this definition was the idea that innovation is a capability that is developed, and really only happens after a certain level of ability is obtained (illustrated by a pianist that has to develop complex technical skill before they can make meaningful innovative music). The idea that some creativity/innovation isn’t “good” was highlighted throughout the same keynote:

For context, here is the list of “Innovation Capabilities” that were shared:

There was also various other forms of context, all of which I thought were good angles to look at, but still very top-down:

This was capped off by the idea that there are “good kinds” of innovation and “bad kinds” of innovation, and we should avoid the bad innovations:

Of course, the master of all meme media Tom Evans made a tool to help us make these decisions:

What one person sees as a “bad” innovation might be a “good” innovation to another. Not sure how to make the determination in such an absolute sense.

There was also an interesting terms of “innovation activist” that was thrown in there that many questioned:

I get that many want a concrete definition of innovation. But I think there are nuances that get left out when we push too strongly in any one direction for our definitions. For example, I agree that innovation is a capability that can be trained and expanded in individuals. But it is also something that just happens when a new voice looks at a problem and comes up with a random “out of the blue” idea. My 6 year old can look at some situation for the first time and blurt out innovative ideas that I had never heard of. Of course, he will also blurt out many ideas that are innovative to him, but that I am already aware of. And there lies the difficulty of defining “innovation”….

Whatever innovation is, there is a relative element to it where certain ideas are innovative to some but not to others. Then there is the relative element that recognizes that innovation is a capability that can be cultivated, but cultivation of that capability is not necessarily a prerequisite to doing something “innovative.”

In other words, any definition of innovation needs to include the space for students to participate, even if they are new to the field that is “being innovated.” The list of Educational Capabilities pictured above is very instructor/administrator/leader centric. Some of those items could be student-centered, but the vocabulary on the slide seems to indicate otherwise. But ultimately I guess it goes back to whether one sees innovation as absolute or relative to begin with. If Innovation (with a capital “I”) is absolute, then there are definitely some things that are innovative at all times in all contexts and some things that aren’t, and therefore Innovation is a capability that has to be developed and studied in order to be understood before participating. But if innovation (with a lower case “i”) is relative, then anyone that is willing to can participate. Including students. But you rarely (at any conference) see the student voice represented in the vendor hall. And as with any conference, how goes the vendor hall, so goes the conference….

Matt Crosslin

Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

Big (Scary) Education (Retention) Data (Surveillance)

Big data in education might be the savior of our failing learning system or the cement shoes that drags the system to the bottom of the ocean depending on who you talk to. No matter what your view of big data is, it is here and we need to pay attention to it regardless of our views.

My view? It is a mixture of extreme concern for the glaring problems mixed with hope that we can correct course on those problems and do something useful for the learners with the data.

Yesterday at LINK Lab we had a peak behind the scenes at a data collection tool that UTA is implementing. The people that run the software at UTA are good people with good intentions. I also hope they are aware of the problems already hard coded in the tool (and I suspect they are).

Big Data can definitely look scary for a lot of reasons. What we observed was mostly focused on retention (or “persistence” was the more friendly term the software uses I believe). All of the data collected basically turns students into a collection of numbers on hundreds of continuums, and then averages those numbers out to rank them on how likely they are to drop out. To some, this is scary prospect.

Another scary prospect is that there is the real danger of using that data to see which students to ignore (because they are going to stick around anyways) and which students to focus time and energy on (in order to make the university more money). This would be data as surveillance more than educational tool.

While looking at the factors in this data tool that learners are ranked by led to no surprises – we have known from research for a long time what students that “persist” do and what those that don’t “persist” do (or don’t do). The lists of “at risk” students that these factors produce will probably not be much different from the older “at risk” lists that have been around for decades. The main change will be that we will offload the process of producing those lists to the machines, and wash our hands of any bias that has always existed in producing those lists in the first place.

And I don’t want to skip over the irony of spending millions or dollars on big data to find out that “financial difficulties” are the reason that a large number of learners don’t “persist.”

The biggest concern that I see is the amount of bias being programmed into the algorithms. Even the word “persistence” implies certain sociocultural values that are not the same for all learners. Even in our short time looking around in the data collection program, I saw dozens of examples of positivist white male bias hard coded in the design.

For example, when ranking learners based on grades, one measure ranked learners in relation to the class average. Those that fell too far below the class average were seen as having one risk factor for not “persisting.” This is different than looking at just grades as a whole. If the class average is a low B but a learner has a high B, they would be above the class average and in the “okay” zone for “persistence.”

But that is not how all cultures view grades. My wife is half Indian and half Australian. We have been to India and talked to many people that were under intense stress to get the highest grades possible. It is a huge pressure for many in certain parts of that culture. But even a low A might not register as a troubling signal if the class average is much lower. But to someone that is facing intense pressure to get the best grades or else come home and work in Dad’s business… they need help.

(I am not a fan of grades myself, but this is one area that stuck out to me while poking around in the back end of the data program)

This is an important issue since UTA is designated as a Hispanic Serving Institute. We have to be careful not get into the same traps that education has fallen into for centuries related to inequalities. But as our LINK director Lisa Berry pointed out, this is also why UTA needs to dive into Big Data. If we don’t get in there with our diverse population and start breaking the algorithms to expose where they are biased, who else will?  Hopefully there are others, but the point is that we need to get in there and critically ask the hard questions, or else we run the risk of perpetuating educational inequalities (by offloading them to the machines).

For now, a good place to start is by asking the hard questions about privacy and ownership in our big data plan:

Are the students made aware that this kind of data is being collected?

If not, they need to be made aware. Everywhere that data is collected, there should be a notification.

Beyond that, are they given details on what specific data points are being collected?

If not, they need to know that as well. I would suggest a centralized ADA-compliant web page that explains every data point collected in easy to understand detail (with as many translations to other languages as possible).

Can students opt-out of data collection? What about granular control over the data that they do allow to be collected?

Students should be able to opt out of data collection. Each class or point of collection should have permissions. Beyond that, I would say they should be able to say yes or no to specific data points if they want to. Or even beyond that, what about making data collection opt-in?

Who owns the students’ data (since it is technically their actions that create the data)?

This may seem radical to some, but shouldn’t the student own their own data? If you say “no,” then they should at least have the right to access it and see what is being collected on them specifically.

Think of it this way: How will the very substantial Muslim population at UTA feel about a public school, tied to the government, collecting all of this data on them? How will our students of color feel about UTA collecting data on them while they are voicing support for Black Lives Matter? How would the child of illegal immigrants feel about each class at UTA collecting data about them that could incriminate their parents?

edugeek-journal-avatarThese issues are some of the hard things we have to wrestle with in the world of Big Data in Education. If we point it towards openness, transparency, student ownership, and helping all learners with their unique sociocultural situations, then it has potential. If not, then we run the risk of turning Big Education Data into Scary Retention Surveillance.

Matt Crosslin

Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

Pokemon Go and the Gimmickification of Education

I almost dread looking at my social media feed today. Pokemon Go (GO? G.O.? (wake me up before you) Go-Go?) received a large bit of media attention this weekend, apparently even already spawning posts about how it will revolutionize education and tweets about how we need what it produces in education:

All I could think about is: how did we get to this point? Every single tech trend turns into a gimmick to sell education mumbo jumbo kitsch tied to every cool, hip trend that pops up on the social media radar. I guess I shouldn’t been that surprised once Block-chain became educational, or Second Life was used to deliver classes, or Twitter replaced LMSs, or MySpace became the University of the future, or DVDs saved public schools, and so on and so forth. I bet at some point thousands of years ago there was a dude in white toga standing up in an agora somewhere telling Plato how chariots would revolutionize how he taught his students.

I’m all for examining new trends through an educational lens, but every time I just want to say “too far, Ed-Tech®, too far!”

We all know education needs to change. It always has been changing, it always will, and will always need to have a critical lens applied to how and why it is changing. But with every new technology trend that gets re-purposed into the next savior of education, I can’t stop this gnawing feeling that our field is becoming a big gimmick to those outside of it.

A gimmick is basically just a trick intended to attract attention. One or two seem harmless enough. Well, not that harmful? But once everything that comes down the pipe starts become this trick to get people to look at education, the gimmick gets old. People are still asking what happened to Second Life, to Google Wave, to you name the trend. After a while, they stop buying into the notion that any of us know what we are talking about. Just think of the long-term effect on the larger discourse of so many people declaring so many things to be the savior of education, only to abandon each one after a year or two.

edugeek-journal-avatarThe problem with the hype cycle of Ed-Tech is that is buries the real conversations that have been happening for a long time on whatever the hype-de-jour is. Do you want the Pokemon Go for education, where students are engaged, active, social, etc? We already have a thousand projects that have done that to some degree. Those projects just can’t get attention because everyone is saying “Pokemon Go will revolutionize education!” (well, at least those that say that un-ironically – sarcastic commentary that apparently went over many people’s head not included).

(see also “Pokemon GO is the xMOOC of Augmented Reality“)

Matt Crosslin

Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

What If The Problem Isn’t With MOOCs But Something Else?

Is this another post about how MOOCs are misunderstood ideas that the critics all get wrong? Not quite. There are problems with MOOCs, but I’m still looking at the conversation about MOOCs in general (continuing from my last post kind of). The general conversation about MOOCs (and for that matter other ed tech innovations such as flipped learning, gamification, etc) tends to be all over the place: insightful, missing the forest for the trees, really odd, kind of just there, etc. All of that is great and makes for interesting discussion. One of the concepts that seems to be getting more traction the past few weeks is “motivation.”

The article about “Why Technology Will Never Fix Education” has already been the subject of many insightful observations. I want to zoom in on one part:

The real obstacle in education remains student motivation. Especially in an age of informational abundance, getting access to knowledge isn’t the bottleneck, mustering the will to master it is. And there, for good or ill, the main carrot of a college education is the certified degree and transcript, and the main stick is social pressure.

I don’t think we can just pass over that last statement with just a simple “for good or ill.” There is a lot of “ill” with that carrot that needs to be unpacked. In an article that very correctly examines the problems of inequality in education, a huge systemic problem is skipped over.

Of course, this article is not the only one. Many other articles have pointed at “student motivation” as being a huge problems with MOOCs. MOOCs are like any other education idea: subject to good and bad instructional design. So you shouldn’t blame the overall idea when learners are just getting bored with bad instructional design. But even beyond that, the above quote speaks to how our system in the U.S. relies on motivational techniques that are predominantly extrinsic in nature. We spend decades indoctrinating learners with this context, and then when an idea comes along that relies mostly on intrinsic motivation, we blame the idea itself rather than our system.

What if MOOCs are just a mirror that shows us the sociocultural problems we don’t want to deal with in our system?

What if the problem is not with the learners, but the way they have been programmed through the years? Grades, credits, failure, tuition, fees, gold stars, extra recess for good grades, monetary rewards, etc are all programmed into learners from a young age.

You can say MOOCs are failing because they lack sufficient “student motivation,” but what if it was actually the case that society has been failing for decades and MOOCs are just exposing this?

Of course, we all recognize many ways that society is failing in education. But what if there are other ways? What if relying on too much extrinsic motivation is a failure? What if we are failing to embrace all of the current and historical research in motivation? What if we know a lot about motivation, but fail to real utilize any of that knowledge? On Twitter yesterday, Rolin Moe pointed out that he never reads discussion of Herman Witkin, cognitive styles, field dependence/independence, etc in relation to motivation. In my circles, I have heard Witkin brought up, but to be honest – I can’t recall anyone trying or applying his ideas (kind of in the same way people in education rattle off Skinner or Bandura and then just don’t really use any of their ideas). These are all ways that our educational system was failing just in the area of motivation for decades before MOOCs (or many other Ed Tech ideas) even came along.

Yet what happens is that the ideas like MOOC are blamed for the historical failure of the system, and those that feel more comfortable within that system recommend pulling the wild ideas back in to make them look more like the existing system. Just think about it: what are the recommendations for fixing “student motivation” in MOOCs? Find a way to add back extrinsic motivation!

I would say: no. We need to find a different path. In fictional entertainment, one of the foundational constructs is to reach for is “suspension of disbelief.” You have to help the readers come to a place of either gaining interest in your story or believability in the fiction elements so that they suspend skepticism and engage the story. Traditional education has typically sought for a “suspension of laziness” – looking for ways to get learners to get off their rears and learn (because we always assume that when they don’t want to learn it is their motivation instead of our design). Newer ideas like MOOCs are going past that, to what I guess could be called “suspension of extrinsic motivation” (for lack of better words). What does learning design look like when you remove all of these carrot sticks (or actual paddling sticks) and leave learners to just pure learning? Well… maybe purer learning than what we had.

edugeek-journal-avatarThere are many, many more angles to explore here (not to mention problems with extrinsic/intrinsic motivation constructs), but I am already getting long-winded. The important idea to consider is that instead of pulling emerging technology and design back towards the tradition of what we already know (which is actually a power struggle by those in power), we need to push forward towards the direction that we already know we need to go.

(image credit: Manu Mohan, obtained from freeimages.com)

Matt Crosslin

Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

More #et4online Reflections: The Major Values of Teacher Tank

So in Laura Pasqunin’s most excellent reflections on the OLC Emerging Technologies conference, she pointed out how some people had cast doubt on my favorite part of the conference: Teacher Tank. I expressed my disagreement with that, and she asked for input. My comment turned into a book, so I decided to turn it into a blog post.

First of all let me say – I realize that Teacher Tank will most likely never happen again, because it will probably chase sponsors away. Which is disappointing, but also part of the problem. The vendors have too often controlled a one-way conversation at conferences. They position themselves as the main financial support for the conferences and then people don’t want to criticize them for fear of killing the conference (I could write a series of long posts about what I was told I could and couldn’t say at conferences by vendors, how they cut-off my microphone so I couldn’t ask questions in online sessions, how they have said to rooms full of people that feedback was not allowed, etc). So it was a gutsy move to have the tank in the first place, but I get why it will probably never happen again. Doing it in another format would probably just not produce the same effect.

I will also say that I know several good vendors and start-ups that are not like the others. Unfortunately, there is noticeable “standard” type of vendor that we see dominating conferences, and those are the ones that I have concerns about.

Personally, I think that commentary and entertainment are two massive values for a conference that most are usually missing for the most part. So many conferences would be more interesting to me if they provided more commentary and even a little entertainment. Don’t get me started on the “cutesy motivational speakers as keynotes” or the “light snacks for dinner and a bunch of tables” forms of “entertainment.” We need actual entertainment sessions to prove a mental break from the sessions from time to time. We also need actual commentary from experts (in addition to thought-provoking keynotes and informative sessions) as well as mote avenues for public commentary by conference attendees.

From the audience perspective, here is the big value that I saw happening over and over again at Teacher Tank:

  • Vendor says some unfounded educational urban legend (“students love to learn with video”)
  • Half of audience nods in agreement
  • Judge pushes back against legend (“like hell they like to learn with video”)
  • Same half of audience looks shocked
  • Judge explains why the legend was wrong (“they like entertainment, but they learn little, etc”)
  • Same half of audience has an obvious light bulb moment and starts nodding again
  • Educational Urban Legend busted for many people.

That happened a lot of times for a lot of people in the audience at Teacher Tank. Huge value if you ask me.

The thing I am most disappointed in is the reaction from some of the vendors to Teacher Tank. Most of the feedback was constructive, a lot of it was positive, and some of it was negative. But it was a MASSIVE amount of feedback they got – more so than they will probably ever get from any one event. All for free, and very little of it from people that are just telling them what they want to hear. A ton of honest feedback. But now some are upset about the tank, saying things like it perpetrated “myths” about start-ups, and so on. Very disappointing.

Look, if you are a start-ups and you want to win me back, then learn to listen, research, and improve based on the feedback… rather than claiming the image I have of start-ups is mythical (while responding in a way that proves they aren’t myths :) ). Prove that you are not thin-skinned and come back to conferences that have criticized you. Maybe even turn your one-way hype session presentation into a Teacher Tank format. Why should conference organizers have to be the ones facilitating valuable honest feedback for you? I have heard about the focus groups and customer research you do – interesting results, but no where near as critical as you would get in the tank.

People always ask me to get involved in conferences more behind the scenes, and I am usually very hesitant. So many conferences today are somewhat dominated by vendors that are controlling one-way conversations (next time you are at a booth and they ask you “what do you think?”, try being honest and see how well that goes 905 of the time). To push back against that means losing sponsors and killing the conference – so I understand why conferences are like that. I don’t blame them. The Teacher Tank and the #et4snark tag were a breath of fresh air in the conference space. Most of my ideas for conferences are along those lines, and usually don’t fit in. I mean, I brought a “buzzword buzzer” to my own session and let people get on my case if I used an EdTech Urban Legend Buzzword. Judging by the Twitter feedback, people loved my session. But I doubt those ideas would go over well with vendors or even most presenters.

I get it that there is a lot of bad feedback out there. Look at some of the MOOC criticism out there (“cMOOCs probably aren’t MOOCs because they are neither massive or open” I was told at a conference recently). But many of the MOOC criticisms are legitimate points, as much as they may annoy myself or others (“most MOOCs are designed using simple and ineffective pedagogy” for example). But don’t mistake those criticisms, even the negative ones, as meaning people hate you or even want you to go away. I love WordPress, for example, but I can also give you a long list or where it needs to be improved.

edugeek-journal-avatarWhat should be more concerning is when people stop criticizing you – when they give up and just write you off as someone that doesn’t listen. I see this happening way too much in the EdTech world in regards to Tech companies. I have lost count of the number of people that have told me they have given up talking to all of the companies that just don’t listen. That should be more concerning to vendors than being a position to get a few minutes of uninterrupted feedback in the Teacher Tank.

(image credit: leenapics, obtained from freeimages.com)

Matt Crosslin

Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

The Disruption That Never Will Be in Education

Don’t get me wrong – change is coming to education, and disruption will be part of it. But all of the comparisons to the music industry are off base, because much of the “disruption caused by mp3s” narrative is a smokescreen from the music industry intended to distract from other questionable activities they are participating in. And also to quote Jim Groom: “Why are we so hell bent on disrupting everything right now?

But let’s start with a historical look at the music industry. If you are old enough, you probably remember seeing this sticker quite often:

Home_taping_is_killing_music

When the cassette tape came out, it quickly became a cheap means for creating your own tapes at home. While people like to act like the mp3 created the “unbundling/rebundling” phenomenon, the truth is that it was the mixtape that started it. Many people like to act like all they did was make a personal favorites list from their own collection, but the truth is that most of us used the mixtape to get a bunch of songs we liked from friends so that we wouldn’t have to buy a whole album for one song. Some of us even coordinated music buying with friends and family so that we could get all the songs we wanted for the least amount of money. This led to the rise of the home taping movement along with the music industry creating several PSAs about how this movement was killing their business.

Which, of course, it obviously did not.

So the ability to unbundle and rebundle music is nothing new. Neither is the ability to get free music. The same holds true in other forms of entertainment: people that didn’t want to buy newspapers knew what coffee shops to hit at what time to get a free copy. People set up elaborate systems for trading VHS and Betamax tapes. Or they learned how to tape movies off of broadcast TV once you were allowed to pause recording during commercials. The digital revolution sped this process up and anonymized it considerably, but there were actually other factors that contributed more to the disruption that occurred in the music business. Of course, you rarely hear about these because it exposes a more questionable side to the music business. Not to mention that “home burning” is probably bigger than online piracy:

“It seems the ripping of CDs borrowed from friends and family accounts for almost as much music piracy as online file sharing anyway, which is an interesting discovery. This is something that has been rife since before online piracy music became a mainstream activity.”

Remember what happened when the music industry introduced new physical formats (vinyl to 8-track to cassettes to CD)? Everyone had to spend a ton of money upgrading to the new format, because the new format was in no way compatible to the old one. Most of us had to sit around figuring out which albums we liked the most because we could only upgrade a few. Even after the CD, the industry tried to introduce new formats like Super Audio CDs and MiniDiscs, but none of those caught on. People were still trying to upgrade to CDs and just didn’t bite. But also many people noticed that the early CDs sounded horrible when compared to the new albums recorded for CDs. Remember those first Led Zeppelin CDs? It was obvious they were just dumping old music on the new format without trying to upgrade the sound quality. They weren’t expecting this CD thing to last.

Additionally, think about how flimsy all of those physical formats were. They could break, warp, scratch, crack, stretch, and wear out easily. In addition to the massive amounts of money they made off of making consumers upgrade every few years, they also made a lot of money off of people replacing broken or worn media (even CDs wear out if you play them too much).

Mp3s and cloud storage changed this. Once you get your music digitally apart from the physical media, it can always be compatible with newer formats. Look at how many formats iTunes plays. Some new format comes out? Download the update and keep going. Mp3 player breaks? Just re-download the songs.

There was one area that the digital revolution did obviously disrupt. The one thing that home taping couldn’t deal with was the need to still buy an entire album to just get the songs you wanted. Sure, there were 45s and cassingles and even CD singles, but those just had the one hit song (and a throw away song if that). Usually three of those would equal the cost of a full album, and most hit bands would have at least three hit singles. So most of us just got the album and skipped the process of waiting for singles. MP3s did change that radically, in that you could just buy the songs you like at $1 a pop and skip the rest of the filler. Because, let’s face it, most hit albums are a few good songs that are obvious singles and a bunch of boring filler. But no record company is going to point out how little effort they put into the whole album. So yes, the mp3 did disrupt the business of tricking people into buying a full album of filler in order to get the 2-4 songs that the record company spent actual time and money on developing into hit songs.

This all points to the real disruption in the music business that the industry will never mention. Some of their more lucrative side-effect revenue streams were cut off over night (upgrading old media, replacing damaged media, and buying the full media to just enjoy a small part). These disruptions will not transfer to the education sector until someone invents a way to improve the human brain. Once we “download” education, its not permanent. We will need refreshers. We will need updates. For now at least, we can’t download the new information directly to our brains once the old goes out of date. We will need to constantly learn new information and enforce existing information, so education is still needed in some form and free online content will not change that.

So, in addition to the real music-industry disruption being something that most aren’t focusing on, we also have the issue that those at the top (record companies) are still doing well despite what they are saying. The music industry still made $16.5 billion dollars in 2013. That may be half of what they made 10 years ago, but a lot of that loss can be accounted for through the loss of the “lucrative side-effect revenue streams” I mentioned. And o you really think they laid off any corporate head honchos because of those losses? Doubtful. We do know there are less artists getting signed, less music being produced by older artists, and less newer artists clogging up the airwaves. The people at the top are still making money by squeezing more out of the people at the bottom. Look at all of the hit songs that are “featuring” guest appearances from other artists. How do you increase the sales of a hit song? Get another famous person to guest on that song and all of their fans will also buy the song. Instance 2-for-1 sales bump! Sound familiar?

Of course, this is not isolated to a few colleges. Faculty around the world are reporting being required to do more with less resources and support while upper level administration seems to continue to increase.

Something else to think about. Recent research is showing that people that download the most free content illegally are also the ones that buy the most legal content. Those that already have the service being offered are the main ones that are consuming the free version of it. Sound familiar? Like how most people that take MOOCs already have a college degree?

What this points to is that any disruption that the education industry would go through in common with the music industry has already happened.

So we have a few reality check factors to consider:

  • Unbundling and rebundling is nothing new and existed well before the digital revolution
  • Access to free content also existed well before the digital revolution
  • A lot of the “disruption” that occurred in the music industry is a smoke screen from the music industry itself designed to garner support for current questionable actions as well as hide questionable practices in the past.
  • Much of the actual disruption that happened due to mp3s and digital content can’t really transfer to the education industry due to the education sector being much more complex.
  • The disruptions that can transfer from the music/entertainment industry to the education industry have already happened.

All of this to say that music metaphors need to stop. Changes and disruptions are going to happen (and have been happening for a long time), but it seems we seldom see the people that have a more realistic grasp on the changes that are coming speaking at big educational conferences. This post was originally meant to be a two or three paragraph intro to a blog post called “Ask Not What Disruption Will Do To You, But What You Can Do For the Coming Disruption” – but that will have to wait until next time. We need to stop this focus on disrupting everything now based on a busted music industry model and instead ask how we can guide the changes that are coming to be beneficial for learners and faculty and not the big dogs at the top.

Matt Crosslin

Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

Research Says: Online or Face to Face Is Better?

You know what they say about getting into an argument with an instructional designer over learning design? Oh… they don’t? Well, they should. Anyway… if they did say anything about it, they would say not to do it because instructional designers pretty much shoot holes in everything.

People argue all the time over whether online learning is better or worse than face-to-face. But you ask an instructional designer which is better? Well, neither, both, and… it kinda depends.

Confusing? Yeah, well blame the research. Research is important. Research tells us a lot. Research raises a lot of good questions. But it seems like we as the educational community are misusing and over simplifying the results of the research.

A lot of research is based on numbers. And those numbers might tell us that, say, there is a statistically significant difference between the number of learners that passed the test in the face-to-face version of a course and the number of those that passed in the online version. Or substitute “test” with whatever metric you are using to determine which is better. And so face-to-face is declared the winner and online is the loser that has to slink off and die because it *lost*!

The problem is – online learning obviously worked great for those students that passed – even if there were statistically significantly fewer of them (did I just butcher the English there?). Research is not a contest to show which option is the one right one. We are not in a giant game of Highlander: Education. There can be more than one right way. It can be online and blended and face-to-face. We are not waiting to see which one beheads all the others to become the clear champion of the universe.

So when the Department of Education came out and declared blended learning the best, that did not mean that online and face-to-face were horrible or ineffective. They just found a higher number for blended. That’s all. That doesn’t invalidate the other two. They are  a national entity that has to look at what works for millions of students.

One way that we know that online learning is working is by learner testimonials. There are thousands and thousands of learners all over the nets saying how online learning worked for them. And guess what – some of them actually failed their courses! Wait – am I telling you that scores don’t matter? Well, of course they matter if you want to earn a piece of paper. But many learners don’t look at a passed test or course as a sign of “working.” Earning an “F” in a course could mean they don’t take tests well, or they had a death in the family during the semester, or they went off on a tangent and forgot to take the final because they were too busy learning informally.

Then there is the other end of the spectrum, where students get annoyed at classes and give them bad satisfaction ratings because they were required to do actual work and they thought they should get an A just for paying for the class.

So ultimately, if a student says an online course worked for them because it challenged them to think and learn, that is good evidence that it worked. Test scores and completion rates and satisfaction surveys might also tell us something, but typically those are ranking systems and not a “winner takes all” cage matches.

But another huge problem – one that instructional designers would point out to you – is that even the best research studies cannot really tell you if online or face-to-face is better. They can compare how the learners in one type of online learning design for a specific time period performed against another group of learners in one type of face-to-face learning design for that same period. There are so many different ways to design for learning online, and there are so many different ways to design for face-to-face, and so many ways that different instructors can affect their classes, and so many ways the learner population can affect the mood of the class, and so on. Research gives us a snap shot of what is going on in specific set of classes at a specific time – but the goal should be to ponder what this means for our personal situation and adapt and experiment ourselves. Not “this works! This doesn’t” and move on.

So the instructional designer will tell you that, yes we know a whole lot about what “works” in the macro sense of education, but in a lot of ways we also know very little of what “works” also. We can tell you want generally works in online or face-to-face and what doesn’t… but it ends up being a long vague list that you still have to take a stab at to see what does and doesn’t work for you specifically.

And the kicker is – despite all the research and facts I knew when I started as an instructional designer… I didn’t really get all of this until I started teaching online. Once you start teaching yourself, and trying to actually do what the research says… you begin to realize that it’s not so black and white. There are no champions of the universe, no best practices, no learning styles, no easy categories for everything to fit in. Oh, sure – you “know” that before you start teaching, but it’s kind of like you “know” parenting is tough until you have a kid and see how tough (and wonderful) it can be for yourself. First-hand knowledge changes your perspective radically. And simplistic answers from research goes out the window. The research itself (or at least the good research) doesn’t really ever give easy answers – people just misread it and think it does. Once you start teaching yourself, you begin to realize that you will use research to inform your practice instead of dictate it.

Some day soon I hope we move beyond this pointless rhetoric about online or face-to-face or blended learning being better or a good way to learn or whatever. All education is distributed over a distance anyways. Learners have declared that all work for them. Its better to start looking at what worked or didn’t work for the learners and go from there. That might call for some – gasp – qualitative research!

“So okay, Matt, stop with the whole ‘there is no spoon BS’ and tell me straight – does online learning work or not” you might say. Online learning works – for certain students. What all of the research is really telling us is that what doesn’t work is forcing all students into one-size fits all learning designs. Therefore, that leads me back to why I like working with the dual layer MOOC group – how can we offer students options to determine for themselves what works best? How can we create multiple paths that are truly multiple paths and not just “five different version of the same silo”? How can we create learning designs that emphasize diversity, experience, and autonomy in learning? Especially when so many students are used to instructivist learning?

Matt Crosslin

Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

When Hype Eats the Real Concept

A co-worker emailed me the other day and asked if I had heard of “Online-Only Flipped Classrooms.” After discussing it with him, it seems like this is a real  thing. But it also seems that every reference I can find to “Online-Only Flipped Classrooms” really just describes what we used to just call “online learning”  less than 10 years ago.

Great.

You know that the hype around Flipped Classrooms and MOOCs has gained a life of its own when people start writing books about “new directions” for those concepts and don’t even realize they are just describing basic online learning. Think about it: you watch a video or read some text and then come together to discuss or work on a group project. That has been basic online learning for centuries. But now it is being called “Online-Only Flipped Classrooms.” Oh, and not to mention it is labeled as “student-centered learning.”

I guess that is another post – at what point did “making your students work together for the majority of the time” become synonymous with “student-centered learning”?

Matt Crosslin

Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.

But The Algorithm Said I Passed!

Many people have noticed a growing focus on automation in learning, especially around the idea of computer-graded assignments. Not just telling students that they picked the right answer on a multiple choice test, but the actual grading of term papers based on complex algorithms. EdX among others are working on systems that will grade thousands of student submissions based on what it thinks the instructor would have given the students.

Some love this idea, some are creeped out.

Students seem to love the idea of removing instructor bias from the grading equation. Or do they just love the idea that they can learn to game the algorithms? We will see in the future, of course.

But at some point, how do we know that learners have actually mastered anything if there is no intelligence in the process that really understands what the student is trying to communicate. After all, if there is anything to all of this social constructivism or connectivism stuff… what happens when one part of the equation is not really intelligent or alive and therefore not social?

Well, you might say, some day the program will get complex enough that computers will have artificial intelligence. The problem with that is, in order to have true intelligence, you have to have a bias of some kind. If someone puts your life on the line versus another person that you don’t know, you will probably fight to live. That is a bias. Or maybe you will take the high road and put the other person’s life ahead of yours. That is another bias. If a machine can not make a choice between preserving itself or thinking of others first, it is following what it was programmed to do and is not truly intelligent. And even worse, it means that it had a certain bias programmed into it by its creator.

All of this might not phase a pure empiricist/behaviorist at all. But to those of us that subscribe to anything from pragmatism to constructivism to connectivism, there are huge problems even if programmers can in some way figure out the perfect algorithm. If one side of the equation is not really intelligent – how can learning really be occurring? Even if you are a cognitivist at heart… how do you know that the computer program with the grading algorithm is compatible with the human computer we call a brain? Or how do you know that the organic brain didn’t just find a way to game the digital one? Will we really be able to create programs that see past the elaborate smoke screen that many humans are known to create?

Matt Crosslin

Matt is currently the Learning Innovation Coordinator with the UT Arlington LINK Research Lab. His research focuses on Learning Theory, Innovation, and learner empowerment. Matt holds a Ph.D. in Learning Technologies from the University of North Texas, a Master of Education in Educational Technology from UT Brownsville, and a Bachelors of Science in Education from Baylor University. His research interests include instructional design, learning pathways, sociocultural theory, heutagogy, virtual reality, and open networked learning. He has a background in instructional design and teaching at both the secondary and university levels and has been an active blogger and conference presenter. He also enjoys networking and collaborative efforts involving faculty, students, administration, and anyone involved in the education process.