The Problem of Learning Analytics and AI: Empowering or Resistance in the Age of “AI”

So where to begin with this series I started on Learning Analytics and AI? The first post started with a basic and over-simplified view of the very basics. I guess the most logical place to jump to is… the leading edge of the AI hype? Well, not really… but there is an event in that area happening this week, so I need to go there anyways.

I was a bit surprised that the first post got some attention – thank you to those that read it. Since getting booted out of academia, I have been unsure of my place in the world of education. I haven’t really said much publicly or privately, but it has been a real struggle to break free from the toxic elements of academia and figure out who I am outside of that context. I was obviously surrounded by people that weren’t toxic, and I still adjunct at a university that I feel supports its faculty… but there were still other systemic elements that affect all of us that are hard to process once you are gone.

So, anyway, I just wasn’t sure if I could still write anything that made a decent point, and I wasn’t too confident I did that great of a job writing about such a complex topic in a (relatively) short blog post last time. Maybe I didn’t, but even a (potentially weak) post on the subject seems to resonate with some. Like I said in the last post, I am not the first to bring any of this up. In fact, if you know of any article or post that makes a better point than I do, please feel free to add it in the comments.

So, to the topic at hand: this week’s Empowering Learners in the Age of AI conference in Australia. My concern with this conference is not with who is there – it seems to be a great group of very knowledgeable people. I don’t know some of them, but many are big names in the field that know their stuff. What sticks out to me is who is not there, as well as how AI is being framed in the brief descriptions we get. But neither of those points is specific to this conference. In fact, I am not really looking at the conference as much as some parts of the field of AI, with the conference just serving as proof that the things I am looking at are out there.

So first of all, to address the name of the conference. I know that “empowering learners” is a common thing to say not just in AI, but education in general. But it is also a very controversial and problematic concept as well. This is one concern that I hang on all of education and even myself as I like the term “empower” as well. No matter what my intentions (or anyone else’s), the term still places the institution and the faculty as the center of the power in the learning process – there to decide whether the learners get to be empowered or not. One of the best posts on this topic is by Maha Bali: The Other Side of Student Empowerment in a Digital World. At the end of the post, she gets to some questions that I want to ask of the AI field, including these key ones:

“In what ways might it reproduce inequality? How participatory has the process been? How much have actual teachers and learners, especially minorities, on the ground been involved in or consulted on the design, implementation, and assessment of these tools and pedagogies?”

I’ll circle back to those throughout the post.

Additionally, I think we should all question the “Age of AI” and “AI Society” part. It is kind of complicated to get into what AI is and isn’t, but the most likely form of AI we will see emerge first is what is commonly called “Artificial General Intelligence” (AGI), which a is deceptive way of saying “pretending to act like humans but not really be intelligent like we are.” AGI is really a focus on creating something that “does” the same tasks humans can, which is not what most people would attribute to an “Age of AI” or “AI Society.” This article on Forbes looks at what this means, and how experts are predicting that we are 10-40 years away from AGI.

Just as an FYI, I remember reading in the 1990s that we were 20-40 years away from AGI then as well.

So we aren’t near an Age of AI, probably not in many of our lifetimes, and even the expert options may not end up being true. The Forbes articles fails to mention that there were many problems with the work that claimed to be able to determine sexuality from images. In fact, there is a lot to be said about differentiating AI from BS that rarely gets brought up by the AI researchers themselves. Tristan Greene best sums it up in his article about “How to tell the difference between AI and BS“:

“Where we find AI that isn’t BS, almost always, is when it’s performing a task that is so boring that, despite there being value in that task, it would be a waste of time for a human to do it.”

I think it would have been more accurate to say you are “bracing learners for the age of algorithms” than empowering for an age of AI (that is at least decades off but may never actually happen according to some). But that is me, and I know there are those that disagree. So I can’t blame people for being hopeful that something will happen in their own field sooner than it might in reality..

Still, the most concerning thing about the field of AI is who is not there in the conversations, and the Empowering Learners conference follows the field – at least from what I can see on their website. First of all, where are the learners? Is it really empowering for learners when you can’t really find them on the schedule or in the list of speakers and panelists? Why is their voice not up front and center?

Even bigger than that is the problem that has been highlighted this week – but one that has been there all along:

The specific groups she is referring to are BIPOC, LGBTQA, and Disabilities. We know that AI has discrimination coded into it. Any conference that wants to examine “empowerment” will have to make justice front and center because of long existing inequalities in the larger field. Of course, we know that different people have different views of justice, but “empowerment” would also mean each person that faces discrimination gets to determine what that means. Its really not fair to hold a single conference accountable for issues that long existed before the conference did, but by using the term “empowerment” you are setting yourself up to a pretty big standard.

And yes, “empowerment” is in quotes because it is a problematic concept here, but it is the term the field of AI and really a lot of the world of education uses. The conference web page does ask “who needs empowering, why, and to do what?” But do they mean inequality? And if so, why not say it? There are hardly any more mentions of this question after it is brought up, much less anything connecting the question to inequality, in most of the rest of the program. Maybe it will be covered in conference – it is just not very prominent at all as the schedule stands. I will give them the benefit of the doubt until after the conference happens, but if they do ask the harder questions, then they should have highlighted that more on the website.

So in light of the lack of direct reference to equity and justice, the concept of “empowerment” feels like it is taking on the role of “equality” in those diagrams that compare “equality” with “equity” and “justice”:

Equality vs equity vs justice diagram
(This adaption of the original Interaction Institute for Social Change image by Angus Macguire was found on the Agents of Good website. Thank you Alan Levine for helping me find the attribution.)

If you aren’t going to ask who is facing inequalities (and I say this looking at the fields of AI, Learning Analytics, Instructional Design, Education, all of us), then you are just handing out empowerment the same to all. Just asking “who needs empowering, why, and to do what?” doesn’t get to critically examining inequality.

In fact, the assumption is being made by so many people in education that you have no choice but to utilize AI. One of the best responses to the “Equality vs Equity vs Justice” diagrams has come from Bali and others: what if the kids don’t want to play soccer (or eat an apple or catch a fish or whatever else is on the other side of the fence in various versions)?

Resistance is a necessary aspect of equity and justice. To me, you are not “empowering learners” unless you are teaching them how to resist AI itself first and foremost. But resistance should be taught to all learners – even those that “feel they are safe” from AI. This is because 1) they need to stand in solidarity with those that are the most vulnerable, to make sure the message is received, and 2) they aren’t as safe as they think.

There are many risks in AI, but are we really taking the discrimination seriously? In the linked article, Princeton computer science professor Olga Russakovsky said

“A.I. researchers are primarily people who are male, who come from certain racial demographics, who grew up in high socioeconomic areas, primarily people without disabilities. We’re a fairly homogeneous population, so it’s a challenge to think broadly about world issues.”

Additionally, (now former) Google researcher Timnit Gebru said that scientists like herself are

“some of the most dangerous people in the world, because we have this illusion of objectivity.”

Looking through the Empowering Learner event, I don’t see that many Black and Aboriginal voices represented. There are some People of Color, but not near enough considering they would be the ones most affected by discrimination that would impede any true “empowerment.” And where are the experts on harm caused by these tools, like Safiya Noble, Chris Gilliard, and many others? The event seems weighted towards those voices that would mostly praise AI, and it is a very heavily white set of voices as well. This is the way many conferences are, including those looking at education in general.

Also, considering that this is in Australia, where are the Aboriginal voices? Its hard to tell on the schedule itself. I did see on Twitter that the conference will start with an Aboriginal perspective. But when is that? In the 15 minute introductory session? That is no where near enough time for that. Maybe they are elsewhere on the schedule and just not noted well enough to tell. But why not make that a prominent part of the event rather than part of a 15 minute intro (if that is what it is)?

There are some other things I want to comment on about the future of AI in general:

  • The field of AI is constantly making references to how AI is affecting and improving areas such as medicine. I would refer you back to the “How to tell the difference between AI and BS” article for much  of that. But something that worries me about the entire AI field talking this way is that the are attributing “artificial intelligence” to things that boil down to advanced pattern recognition mainly using human intelligence. Let’s take, for example, recognizing tumors in scans. Humans program the AI to recognize patterns in images that look like tumors. Everything that the AI knows to look for comes directly from human intelligence. Just because you can then get the algorithm to repeat what the humans programmed it to thousands of times per hour, that doesn’t make it intelligence. It is human intelligence pattern recognition that has been digitized, automated, and repeated rapidly. This is generally what is happening with AI in education, defense, healthcare, etc.
  • Many leaders in education in general like to say that “institutions are ill-prepared for AI” – but how about how ill-prepared AI is for the equity and reality?
  • There is also often talk in the AI community about building trust between humans and machines that we see examples of at the conference as well: “can AI truly become a teammate in group learning or a co-author of a ground-breaking scientific discovery?” I don’t know what the speaker plans to say, but the answer is no. No we shouldn’t build trust and no we shouldn’t anthropomorphize AI. We should always be questioning it. But we also need to be clear, again, that AI is not the one that is writing (or creating music or paintings). This is the weirdest area of AI – they feed a bunch of artistic or music or literary patterns into AI, tell it how to assemble the patterns, and when something comes out it is attributed to AI rather than the human intelligence that put it all together. Again, the machine being able to repeat and even refine what the human put there in the first place is not the machine creating it. Take, for example, these different AI generated music websites. People always send these to me and say “look how well the machine put together ambient or grindcore music or whatever.” Then I  listen… and it is a mess. They take grindcore music and chop it up in to bits and then run those bits through pattern recognition and spit out this random mix – that generally doesn’t sound like very good grindcore. Ambient music works the best to uninitiated ears, but to fans of the music it still doesn’t work that great.
  • I should also point out about the conference that there is a session on the second day that asks “Who are these built for? Who benefits? Who has the control?” and then mentions “data responsibility, privacy, duty of care for learners” – which is a good starting point. Hopefully the session will address equity, justice, and resistance specifically. The session, like much of the field of AI, rests on the assumption that AI is coming and there is nothing you can do to resist it. Yes the algorithms are here, and it is hard to resist – but you still can. Besides, experts are still saying 10-40 years for the really boring stuff to emerge as I examined above.
  • I also hope the conference will discuss the meltdown that is happening in AI-driven proctoring surveillance software.
  • I haven’t gotten much into surveillance yet, but yes all of this relies on surveillance to work. See the first post. Watch the Against Surveillance Teach-In Recording.
  • I was about to hit publish on this when I saw an article about a Deepfake AI Santa that you can make say whatever you want. The article says “It’s not nearly as disturbing as you might think”… but yes, it is. Again, people saying something made by AI is good and realistic when it is not. The Santa moves and talks like a robot with zero emotion. Here again, they used footage of a human actor and human voice samples and the “AI” is an algorithm that chops it up into the parts that makes your custom message. How could this possibly be misused?
  • One of the areas of AI that many in the field like to hype are “conversational agents” aka chatbots. I want to address that as well since that is an area that I have (tried) to research. The problem with researching agents/bots is that learners just don’t seem to be impressed with them – it’s just another thing to them. But I really want to question how these count as AI after having created some myself. The process for making a chatbot is that you first organize a body of information into chunks of answers or statements that you want to send as responses. You then start “training” the AI to connect what users type into the agent (aka “bot”) with specific statements or chunks of information. The AI makes a connection and sends the statement or information or next question or video or whatever it may be back to the user. But the problem is, the “training” is you guessing dozens of ways that the person might ask a question or make a statement (including typos or misunderstandings) that matches with the chunk of information you want to send back. You literally do a lot of the work for the AI by telling it all the ways someone might type something into the agent that matches each chunk of content. They want at least 20 or more. What this means is that most of the time, when you are using a chatbot, it gives you the right answer because you typed in one of the most likely questions that a human guessed and added to the “training” session. In the rare cases where some types something a human didn’t guess, then the Natural Language Processing kicks in to try and guess the best match. But even then it could be a percentage of similar words more than “intelligence.” So, again, it is human intelligence that is automated and re-used thousands of times a minute – not something artificial that has a form of intelligence. Now, this might be useful in a scenario when you have a large body of information (like an FAQ bot for the course syllabus) that could use something better than a search function. Or maybe a branching scenarios lesson. But it takes time to create a good chatbot. There is still a lot of work and skill to creating the questions and responses well. But to use chatbots for a class of 30, 50, 100? You probably will spend so much time making it that it would be easier to just talk to your students.
  • Finally, please know that I realize that what I am talking about still requires a lot of work and intelligence to create. I’m not doubting the abilities of the engineers and researchers and others that put their time into developing AI. I’m trying to get at the pervasive idea that we are in an Age of AI that can’t be avoided. Its a pervasive idea that was even made in a documentary web series a year ago. I also question whether “artificial intelligence” is the right term for all of this, rather than something more accurate like “automation algorithms.”

Again, everything I touch on here is not as much about this conference, as it is about the field of AI since this conference is really just a lot of what is in the AI field concentrated into two days and one website. The speakers and organizers might have already planned to address everything I brought up here a long time ago, and they just didn’t get it all on the website. We will see – there are some sessions with no description and just a bio. But still, at the core of my point, I think that educators need to take a different approach to AI than we have so far (maybe by not calling it that when it rarely is anything near intelligent) by taking justice issues seriously. If the machine is harming some learners more than others, the first step is to teach resistance, and to be successful in that all learners and educators need to join in the resistance.

The Problem of Learning Analytics and AI

For some time now, I have been wanting to write about some of the problems I observed during my time in the Learning Analytics world (which also crosses over into Artificial Intelligence, Personalization, Sentiment Analysis, and many other areas as well). I’m hesitant to do so because I know the pitchforks will come out, so I guess I should point out that all fields have problems. Even my main field of instructional design is far from perfect. Examining issues with in a field (should be) a healthy part of the growth of a field. So this will probably be a series of blog posts as I look at publications, conferences, videos, and other aspects of the LA/PA/ML/AI etc world that are in need of a critical examination. I am not the first or only person to do this, but I have noticed a resistance by some in the field to consider these viewpoints, so hopefully adding more voices to the critical side will bring more attention to these issues.

But first I want to step back and start with the basics. At the core of all analytics, machine learning, AI, etc are two things: surveillance and algorithms. Most people wouldn’t put it this way, but let’s face it: that is how it works. Programs collect artifacts of human behavior by looking for them, and then process those through algorithms. Therefore, the core of all of this is surveillance and algorithms.

At the most basic level, the surveillance part is a process of downloading a copy of data from a database that was intentionally recording data. That data is often a combination of click-stream data, assignment and test submissions, discussion forum comments, and demographic data. All of this is surveillance, and in many cases this is as far as it goes. A LOT of the learning analytics world is based on click stream data, especially with an extreme focus on predictive analytics. But in a growing number of examples, there are also more invasive forms of surveillance added that rely on video recordings, eye and motion detection, bio-metric scans, and health monitoring devices. The surveillance is getting more invasive.

I would also point out that none of this is accidental. People in the LA and AI fields like to say that digital things “generate” data, as if it is some unintentional by-product of being digital: “We turned on this computer, and to our surprise, all this data magically appeared!”

Data has to be intentionally created, extracted, and stored to exist in the first place. In fact, there usually is no data in any program until programmers decide they need it. They will then create a variable to store that data for use within the program. And at this moment is where bias is introduced. The reason why certain data – like names, for example – are collected and others aren’t has to do with a bias towards controlling who has access and who doesn’t. Then that variable is given a name – it could be “XD4503” for all the program cares. But to make it easier for programmers to work together, they create variables names that can be understood by everyone on the team: “firstName,” “lastName,” etc.

Of course, this designation process introduces more bias. What about cultures that have one name, or four names? What about those that have two-part names, like the “al” that is common in the Arabic names, but isn’t really used for alphabetizing purposes? What about cultures that use their surname as their first name? What about random outliers? When I taught eighth grade, I had two students that were twins, and their parents gave them both nearly identical sets of five names. The only difference between the two was that the third name was “Jevon” for one and “Devon” for the other. So much of the data that is created – as well as how it is named, categorized, stored, and sorted – is biased towards certain cultures over others.

Also note here that there is usually nothing that causes this data to leave the program utilizing it. In order for some outside process or person to see this data, programmers have to create a method for displaying and / or storing that data in database. Additionally, any click stream, video, or bio-metric data that is stored has to be specifically and intentionally captured in ways that can be stored. For example, a click in itself is really just an action that makes a website execute some function. It disappears after that function happens – unless someone creates a mechanism for recording what was clicked on, when it was clicked, what user was logged in to do the click, and so on.

All of this to say that none of this is coincidental, accidental, or unplanned. There is a specific plan and purpose for every piece of data that is created and collected outside of the program utilizing them. None of the data had to be collected just because it was magically “there” when the digitials were turned on. The choice was made to create the data through surveillance, and then store it in a way that it could be used – perpetually if needed.

Therefore, different choices could be made to not create and collect data if the people in control wanted it that way. It is not inevitable that data has to be generated and collected.

Of course, most of the few people that will read this blog already know all of this. The reason I state this all here is for anybody that might still be thinking that the problems with analytics and AI is created during the design of the end user products. For example, some believe that the problems that AI proctoring has with prejudice and discrimination started when the proctoring software was created… but really this part is only the continuation of problems that started when the data that these AI systems utilized was intentionally created and stored.

I think that the basic fundamental lens or mindset or whatever you want to call it for publishing research or presenting at conferences about anything from Learning Analytics to AI has to be a critical one rooted in justice. We know that surveillance and algorithms can be racist, sexist, ablest, transphobic, and the list of prejudices goes on. Where people are asking the hard questions about these issues, that is great. Where the hard questions seem to be missing, or people are not digging deep enough to see the underlying biases as well, I want to blog about it. I have also noted that the implementation of LA/ML/AI tools in education too often lacks input from the instructional design / learning sciences / etc fields – so that will probably be in the posts as well.

While this series of posts is not connected to the Teach-In Against Surveillance, I was inspired to get started on this project based on reflecting on why I am against surveillance. Hopefully you will join the Teach-In tomorrow, and hopefully I will get the next post on the Empowering Learners for the Age of AI conference written in this lifetime. :)

People Don’t Like Online Proctoring. Are Institutional Admins Getting Why?

You might have noticed a recent increase in the complaints and issues being leveled against online proctoring companies. From making students feeling uncomfortable and/or violated, to data breaches and CEOs possibly sharing private conversations online, to a growing number of student and faculty/staff petitions against the tools, to lawsuits being leveled against dissenters for no good reason, the news has not been kind to the world of Big Surveillance. I hear the world’s tiniest violin playing somewhere.

It seems that the leadership at Washington State University decided to listen to concerns… uhhh… double down and defend their position to use proctoring technology during the pandemic. While there are great threads detailing different problems with the letter, I do want to focus in on a few statements specifically. Not to specifically pick on this one school, but because WSU’s response is typical of what you hear from too many Higher Ed administrations. For example, when they say…

violations of academic integrity call into question the meaningfulness of course grades

That is actually a true statement… but not in the way it was intended. The intention was to say that cheating hurts academic integrity because it messes up the grade structures, but it could also be taken to say that cheating calls into highlights the problem with the meaningfulness of grades because cheating really doesn’t affect anyone else.

Think about it: someone else cheats, and it casts doubt on the meaning of my grade if I don’t cheat? How does that work exactly? Of course, this is a nonsense statement that reals highlights how cheating doesn’t change the meaning of grades for anyone else. Its like the leaders at this institution are right there, but don’t see the forest for the trees: what exactly does a grade mean if the cheaters that get away with it don’t end up hurting anyone but themselves? Or does cheating only cause problems for non-cheaters when the cheaters get caught? How does that one work?

But let’s focus here: grades are the core problem. Yes, many people feel they are arbitrary and even meaningless. Still others say they are unfair, while some look at them as abusive. At the very least, you really should realize grades are problematic. Students can guess and get a higher grade than what they really actually know. Tests can be gamed. Questions have bias and discrimination built in too many times. And so on. Online proctoring is just an attempted fix for a problem that existed long before “online” was even an option.

But let’s see if the writers of the letter explain exactly how one person cheating harms someone else… because maybe I am missing something:

when some students violate academic integrity, it’s unfair for the rest. Not only will honest students’ hard work not be properly reflected…. Proctoring levels the playing field so that students who follow the rules are not penalized in the long run by those who don’t.

As someone that didn’t cheat in school, I am confused as to how this exactly works. I really never spent a single minute caring about other students’ cheating. You knew it happened, but it didn’t affect you, so it was their loss and not yours. In fact, you never lost anything in the short or long run from other student’s cheating. I have no clue how my hard work was not “properly reflected” by other students’ cheating.

(I would also note that this “level the playing field” means that they assume proctoring services catch all “cheaters” online, just like have instructors in the classroom on campus meant that all of the “cheaters” in those classes. But we all know that is not the case.)

I have never heard a good answer for how does supposed “penalization” works. Most of the penalization I know of from classes are systemic issues against BIPoC students that happens in ways that proctoring never deals with. You sometimes wish institutions would put as much money into fighting that as they would spying through student cameras….

But what about the specific concerns with how these services operate?

Per WSU’s contract, the recorded session is managed by an artificial intelligence “bot” and no human is on the other end at ProctorU watching the student. Only the WSU instructor can review the recorded session.

A huge portion of the concern about proctoring has been about the AI bots – which are here presented as an “it’s all okay because” solution…? Much of the real concern many have expressed is with the algorithms themselves and how they are usually found to be based on racist, sexist, and ableist norms. Additionally, the other main concern is what the instructor might see when they do review a recording of a student’s private room. No part of the letter in question addresses any of the real concerns with the bigger picture.

(It is probably also confusing to people whether or not someone is watching on the other side of the camera when there are so many complaints online from students that have had issues with human proctors, especially ones that were “insulting me by calling my skin too dark” as one complaint states.)

The response then goes on to talk about getting computers that will work with proctoring service to students that need them, or having students come in to campus for in-person proctoring if they just refuse to use the online tool. None of this addresses the concerns of AI bias, home privacy, or safety during a pandemic.

The moral of the point I am making here is this: if you are going to respond to concerns that your faculty and staff have, make sure you are responding to the actual concerns and not some imaginary set of concerns that few have expressed. There is a bigger picture as to why people are objecting to these services, which – yes – may start with feeling like they are being spied on by people and/or machines. But just saying “look – no people! (kind of)” is not really addressing the core concerns.

QM and the Politics of the “Unbiased” Bias

So it started off innocent enough, with a Tweet about concerns with the popular QM rubric for course review:

Different people have voiced concerns with the rubric through the years, usually not saying it is all bad or anything, just noting that it presents itself as a “one rubric for all classes” but seems to show a bias for instructor-centered courses with pre-determined content and activities. Which describes many good classes, don’t get me wrong. But there are other design methodologies and learning theories that produce excellent courses as well.

The responses from the QM defenders to the tweet above were about many things that no one was questioning: how it is driven by research (we know, we have done the research as well), lots of people have worked for decades on it (obviously, but we have worked in the field for decades as well; plus many really bad surveillance tools can say the same, so be careful there), QM is not meant to be used this way (even though we are going by what the rubric says), it is the institution’s fault (we know, but I will address this), people who criticize QM don’t know that much about it (I earned my Applying the QM Rubric (APPQMR) certificate on October 28, 2019 – so if don’t understand, then whose fault is that? :) ), and so on.

Now technically I wasn’t “criticizing” QM as much as discussing it’s limitations. Rubrics are technology tools, and in educational technology are told not to look at tools as the one savior of education. We are supposed to acknowledge their uses, limitations, and problems. But every time I want to discuss the limitations of QM, I get met with a wall of responses bent on proving there are no limitations to QM.

The most common response is that QM does not suggest how instructors teach, what they teach, or what materials they use to teach. It is only about online course design. True enough, but in online learning, there isn’t such a clear line between design and teaching. What you design has an effect on what is taught. In fact, many online instructors don’t even call it “teaching” in a traditional sense, but prefer to use words like “delivery” or “facilitate” in place of “teaching.” Others will say things like “your instructional design is your teaching.” All of those statements are problematic to some degree, but the point is that your design and teaching are intricately linked in online education.

But isn’t the whole selling point of QM the fact that it improves your course design? How do you improve the design without determining what materials work well or not so well? How do you improve a course without improving assignments, discussions, and other aspect of “what” you teach? How do you improve a course without changing structural options like alignment and objectives – the things that make up “how” you teach?

The truth is, General Standards 3 (Assessment and Measurement), 4 (Instructional Materials), and 5 (Learning Activities and Learner Interaction) of the QM rubric do affect what you teach and what materials you use. They might not tell you to “choose this specific textbook,” but they do grade any textbook, content, activity, or assessment based on certain criteria (which is often a good thing when bad materials are being used). But those three General Standards  – along with General Standard 2 (Learning Objectives (Competencies)) – also affect how you teach. Which, again, can be a good thing when bad ideas are utilized (although the lack of critical pedagogy and abolitionist education in QM still falls short of what I would consider quality for all). But we do have to recognize that QM does affect the “what” and “how” of online course design, which is the guide for the “what” and “how” of online teaching. That is the whole selling point, and it would be useless as a rubric if it didn’t help improve the course to do this.

So, yes, specific QM review standards require certain specific course structures that do dictate how the course is taught. The QM rubric is biased towards certain structures and design methodologies. If you are wanting to teach a course that works within that structure (and there are many, many courses that do), QM will be a great tool to help you with that structure. However, if you start getting into other structures of ungrading, heutagogy / self-determined learning, aboriginal pedagogy, etc, you start losing points fast.

This is has kind of been my point all along. Much of the push back against that point dives into other important (but not related to the point) issues such as accreditation, burnout, and alignment. Sometimes people got so insulting insulting that I had to leave the conversation and temporarily block and shut it out of my timeline.

QM evangelists are incredibly enthusiastic about their rubric. As an instructional designer, I am taught to question everything – even the things I like. Definitely not a good combination for conversation it seems.

But I want to go back and elaborate on the two points that I tried to stick to all along.

The first point was a response to how some implied that QM is without bias… designed for all courses, and because of this, if some institutions end up using it as a template to force compliance, that is their bias and not QM’s fault. And I get it – when you create something and people misuse it (which no one is denying happens), it can be frustrating to feel like you are being blamed for other’s misuse. But I think if we take a close look at how QM is not unbiased, and how there are politics and bias built into every choice they made, we can see how that has the effect of influencing how it is misused at institutions.

QM is a system based on standardization that was created by specific choices that were chosen through bias, in a method that biases it towards instructor-centered standardized implementation by institutional systems that are known to prefer standardization.

I know that sounds like a criticism, but there are a couple of things to first point out:

  • Bias is not automatically good or bad. Some of the bias in QM I agree with on different levels. Bias is a personal opinion or organizational position, therefore choosing one option over another always brings in bias. There is no such thing as bias-free tech design.
  • The rubric in QM is Ed-Tech. All rubrics are Ed-Tech. That makes QM an organization that sells Ed-Tech, or an Ed-Tech organization.  This is not saying that Ed-Tech is their “focus” or anything like that.

So the thing with QM is that most people understand how it was designed to be flexible. But even those QM design choices had bias in them. All design choices have bias, politics, and context. And when the choices of an entity such as QM are packaged up and sold to an institution, they are not being sold to a blank slate with no context. The interaction of the two aspects causes very predictable results, regardless of what the intent was.

For example, the QM rubric adds up to 100 points. That was a bias point right there. Why 100? Well, we are all used to it, so it makes it easy to understand. But it also connects to a standardized system that we were mostly all a part of growing up, one that didn’t have a lot of flexibility. If we wanted to score higher, we had to conform. When people see a rubric that adds up to 100, that is what many think of first. Choosing a point total that connects with pre-existing systems that also utilize that highest score is a choice that brings in all of the bias, politics, and assumptions that are typically associated with that number elsewhere.

Also, the ideal minimum score is 85. Again, that is biased choice. Why not 82, or 91, or 77? Because 85 is the beginning of “just above average” of the usual “just above average score” (a “B”) that many are used to. Again, this connects to a standardized system we are used to, and reminds people of the scores they got in grade school and college.

In fact, even using points in general, instead of check marks or something else, was another choice that QM made that is also a biased choice. People see points and they think of how those need to add up to the highest number. This mindset affects people even when they get a good number: think of how many students get an 88 and try to find ways to bump it up to a 90. This is another systemic issue that many people equate to “follow the rules, get the most points.”

Then, when you look at how long some of the explanations of the QM standards are, again that was a choice and it had bias. But when combined with an institutional system that keeps its faculty and staff very busy, it creates a desire to move through the long, complicated system as fast as possible to just get it done. This creates people that game the system, and one of the best ways to hack a complex process that repeats itself each course is to create a template and fill it out.

While templates can be a helpful starting place for many (not all), institutional tendency is to do what institutions do: turn templates into standards for all.

This is all predictable human behavior that QM really should consider when creating its rubric. I see it in my students all the time – even though I tell them that there is flexibility to be creative and do things their way, most of them still come back to mimicking the example and giving me a standardized project.

So you can see it all up and down the QM rubric – each point on the rubric is a biased choice. Which is not to say that they are all bad, it’s just that they are not neutral (or even free from political choices). Just some specific examples:

  • General Standard 3.1 is based on “measure the achievement” – which is great in many classes, but there are many forms of ungrading and heutagogy and other concepts that don’t measure achievement. Some forms of grading don’t measure achievement, either.
  • General Standard 3.2 refers to a grading policy that doesn’t work in all design methodologies. In theory, you could make your grading policy about ungrading, but in reality I have heard that this approach rarely passes this standard.
  • General Standard 3.3 is based on “criteria,” which is a popular paradigm for grading, but not compatible with all of them.
  • General Standard 3.4 is hard to grade at all in self-determined learning when the students themselves have to come up with their own assessments (and yes, I did have a class like that when I was a sophomore in college – at a community college, actually). Well, I say hard – you really can’t depending on how far you dive into self-determined learning.
  • General Standard 3.5 seems like it would fit a self-determined heutagogical framework nicely… in theory. In reality, its hard to get any points here because of the reasons covered in 3.4

Again, the point being that it is harder for some approaches like heutagogy, ungrading, and connectivism to pass. If I had time and space, I would probably need to go into what all of those concepts really mean. But please keep in mind that these methods are not “design as you go” or “failing to plan.” These are all well-researched concepts that don’t always have content, assessment, activities, and objectives in a traditional sense.

Many of the problems still come back to the combination of a graded rubric being utilized by a large institutional system. A heutagogical course might pass the QM rubric with, say, an 87 – but the institution is going to look at it as a worse course than a traditional instructivist course that scores a 98. And we all know that institutional interest in the form of support, budgets, praise, awards, etc will go towards the courses that they deem as “better” – this is all a predictable outcome from choosing to use points on a 100 point scale.

There are many other aspects to consider, and this post is getting too long. A couple more rabbit trails to consider:

So, yes, I do realize that QM has helped in many places where resources for training and instructional designers are low. But QM is a rubric, and rubrics always fall apart the more you try to make them become a one-size-fits-all solution. Instead of trying to expand the one Higher Ed QM rubric to fit all kinds of design methodologies, I think it might be better to look at what kind of classes it doesn’t work with and move to create a part of the QM process that identifies those courses and moves them on to another system.

Emergency Educational Measures in a Time of Pandemics, Anti-Racism Protests, and Political Chaos: Is This All Going… Anywhere?

A few months ago, a former co-worker of mine Erika Beljaars-Harris asked me to come talk to the University she now works at – RMIT Australia. They were interested in hearing about the current problems U.S. education is facing and where it is going. While I am not a futurist by training, well, neither are half the people that put the term in their bio. But I think there are at least three obvious trends that are having  – and will continue to have – immediate impacts on education. I also think it is pretty obvious where they are going if you put aside what you would like to see happening and be honest about how difficult it is going to continue to get.

So I came up with the title for the presentation (that I re-used for this post) and a quick 10-minute intro to frame the discussion. I thought I would write this post to capture the basic thoughts I shared in that intro. I covered the three trends I mentioned in the title and where I think we (in the U.S., but many other countries are facing similar issues as well) are going with those, both good and bad.

Pandemic Responses

When I first wrote up the description, it was a month ago and few were really talking Fall seriously. Now a lot of places have started talking, and even releasing some plans. These plans, for the most part, say nothing concrete other than “back to campus! (we hope but won’t commit to fully)” I made a joke about how solid those plans feel:

Campuses need students, faculty, and staff back on campus to generate revenue. So they are going to wait as long as they can to make any decisions other than going back. But coronavirus numbers are growing. Politicians and leaders are not doing enough to reverse that. A small handful of schools have put out some good plans for the Fall (thank you to those that have), but most have released nothing, or even worse, confusing complicated plans that try to hard to get as many students on campus as possible.

So where is this going? Classes are going to be online at some point in the Fall – or even before the Fall. It will probably depend on how much death and sickness we can accept before making the decision. But make no mistake: we will be back online sooner rather than later. And I realize that is not necessarily a good thing for all – but it will happen regardless. As much as you will be told to get ready for on campus or hybrid options, I would put more effort towards getting your classes and yourself ready for online. Make plans to keep yourself and your students as safe as possible. Don’t wait for guidance from leaders, and if it does come, realize you will probably have to go above and beyond anything they plan for.

Anti-Racism and Education

I am not totally sure I used correct English in the title – but the main idea is that we have witnessed sustained protests against systemic racism in several ways following the murder of George Floyd, and many are noting that they seem to be having some effects. I suddenly got a bunch of emails on June 18th telling me about Juneteenth celebrations or events happening the next day – as if a bunch of companies suddenly realized they better do something. Sure, many people will probably lose interest in social justice and equality once the news cycle moves on, but I think we have finally crossed a line to where you need to decide to be anti-racist or not. The lines have been drawn

So where is this going? Don’t look at “diversity” as a thing that you do for a special one day thing in class, or leave to the People of Color in your life. Don’t look at it as something our Black colleague can fill you in on when ever you remember to ask. Don’t ask your Indigenous student to speak for all Indigenous Peoples during a class discussion. You will need to start doing the work to bring marginalized voices into your content from the beginning of class to the end. It will no longer be acceptable to lean on “traditional experts” (aka white males) to make up the main focus of your content. You will need to do the work of searching and learning for yourself who needs to be added to your lessons. You will also need to do the work yourself to examine your own biases and prejudices and how they affect how you teach and interact with students, colleagues, and co-workers. Critical educational practices / pedagogy will need to be practiced by all, not just those that study it as an academic field.

Update: an important aspect of this work is also listening:

Political Chaos

So even beyond academic leaders, it seems that political leaders at all levels and political parties are also falling apart in their response to COVID. Some have done well, but most have not. From reacting too late, to re-opening too soon, to realizing they opened too soon but took too long to reverse course, it has been a failure of partisan politics up and down the board. I am not getting into both side-ism here – one party clearly wanted to follow Science and the other did not (even though even those that wanted to follow Science could have done much better in most cases). Wearing masks, social distancing, supporting the economy after you ordered businesses to shut down, and so on should NOT have ever been political issues to disagree over. And I know that this problem comes from the top and affects the ability of all levels underneath to make decisions. But more leaders at state and local levels should have decided to do the right things regardless of what was happening at the national level. Many did and that helped for the people living in those areas. But many did not. And instead of getting over the worst of it by now as we should be, we are gearing up for the worst of it to still come.

So where is this going? I think we have to face the hard truth that we are on our own for now. You will have to find what ever group you can that will do the right thing and band together to do what you can. Whether a smaller group in the city, at the department level, a subsection of your department, an ongoing Twitter DM group, a Slack channel, a group Facebook Message, whatever it might be – find some people to connect with and band together to help. Whether it is emotional help online or in-person help in your community (socially distanced, of course), find those that are willing to do the right thing and form networks. Try to reach out to as big a level as you can, but don’t count on leaders to get things done. Some leaders may get some things done and that is great – connect with them. If not, find the people that will and band together with them to make it through.

 

Crash Course in Online Teaching 1: Starting With Theory (Wait, Wait – Give It a Chance! Really!)

With several universities now coming to grips with the fact that they will still be online in the Summer (and most likely the Fall), several are turning to how to quickly train their entire faculty in online teaching in a hurry. There really isn’t one ideal way to do this, but I want to offer up the way I would do it if given the chance to design a Crash Course in Online Learning (insert your Budgie/Metallica song parody here).

I would personally start off with a very basic intro to… stay with me here… learning theory. Wait – don’t click away just yet. Many would balk at the idea of starting with theory and not practical tool/building skills, while learning theorists would cry foul at thinking that a basic intro to learning theory is even possible. But I have found that a few basic concepts taught at a very intro level can help faculty not only understand how they have been taught in the past, but also how they can try new ideas and designs they may have never thought of.

So give it chance if you are tired of academia and theory, or give me some wide wiggle room for scaffolding if you are deep into learning theory. I’m going to combine and give examples that will make theory easy to digest, but also blur the lines of complexity that exist. Please keep that in mind as we go forward.

First of all, I will point out that I have published and taught this before. I will give the basics below, but if you have 30 minutes to an hour to dig in, there are slightly more explanatory resources out there. The first is the paper that I published called “From Instructivism to Connectivism: Theoretical Underpinnings of MOOCs.” Yes, it is about MOOCs, but the ideas can be applied to any course. Plus, it comes with a worksheet that you can use to plan your course. The second is a video archive of a training session I led from last year on the paper. If you prefer video more than blog posts or papers, this might work for you:

So, for those that are wanting the summary version, here we go….

Overall Power Dynamics

Out of all the different ways to approach learning theory, I like focus on power dynamics first when it comes to designing a course. So think about the overall power dynamic you want to see happening in your course. This can change from week to week, but in general most courses stick to one for the most part. The question is: who determines what learners will learn in your course, and who directs how it is learned? There are many ways to look at this, but I like to focus on three different -isms:

  • Instructivism: The course is controlled by an expert instructor, determined and directed by that expert.
  • Constructivism: Constructed self-discovery, sometimes determined by the expert, but usually directed by the learner.
  • Connectivism: Learner-determined and directed by the learner, enhanced by networking (connecting) with others (including other learners, the instructor, online resources, etc).

Yes, these can mix, and you can move between different ones. The main thing to think about is who is determining the overall knowledge and/or skills to be learned, and then who is directing how the learners will learn those knowledge and skills and then how they will prove they learned them.

Again, there are many different ways of looking at these terms, many other -isms you can use, and a lot of ambiguity that I am glossing over here. These three -isms (and how I described them) are just a good place to start for those that are new. Generally you pick one, but also think how elements of others might also be utilized in your course.

Methodology of Course Design

Course design methodology often overlaps with power structures. However, within various power structures, there still is room for different design methodologies. For example, even in connectivism there it is still possible to design a course that focuses on transmission of knowledge from experts, even if those experts are not always the instructor.

In this stage, you are thinking about where knowledge and/or skills training comes from, not just who controls the overall power dynamics. Again, there are many, many different ways to look at this. I want to start with three popular -agogies:

  • Pedagogy: many people use this as a catch-all phrase for all teaching design, but in a traditional sense for several centuries it has meant focusing on knowledge transfer from an expert (Update: please note there are many different ways of looking at the term that have gained traction over the past 80-100 years that  hope to cover in upcoming parts – see viewing pedagogy as a philosophy rather than a theory in fields like critical pedagogy).
  • Andragogy: learners draw upon their experience to connect what they already know with new content / knowledge / skills / etc (some have advocated to use the term “Anthropagogy” in place of Andragogy to be more inclusive, but I use the more common term here).
  • Heutagogy: learners focus on learning how to learn about a particular topic rather than just what to learn. Heutagogy is often seen as a critical response to the limitations of other -agogies.

Typically, you see pedagogy matching with instructivism, andragogy matching with constructivism, and heutagogy matching with connectivism. But it is possible for other combinations, such as constructivist heutagogy or connectivivist pedagogy. There is a chart of page 94 of the article above that explains the nine different combination and gives examples.

The main idea is that you choose which combination of the two you want for your class most often (even if if changes from time to time). I tend to advocate for a connectivist heutagogical approach the most often, as that is what more and more people need in the world today. Rather than memorizing expert facts as determined by a the instructor, we need more learners that know how to grow and learn about a topic by connecting with the people and resources that can teach them what they need to know.

At this point, you will start considering what activities and assignments you will be using in your course. It is also good to have some well-written and aligned goals, objectives, competencies, or other standards. I will cover that as a separate post next, but more than likely, you are transitioning a course that already exists on campus into an online course. So I will continue with the theory first, but keep in mind that even if you already have goals and objectives, it would be a good idea to review them after you work through learning theory.

Types of Interaction

Of course, your class will have all types of interaction. However, I have found that once people jump into creating activities and assignments and content first, they leave out interaction until after the bulk of the course activities have been created. At that point, interaction becomes an afterthought or an add-on addition to what has already been created. Which is not ideal.

Thinking through the types of communication that can happen in a course is a good way to proactively plan out different ways to foster interaction as you create content and activities. Most of us think of different types of communication like student-to-student, teacher-to-student, student-to-content, etc. There are already 12 types that have been identified in the literature, but there could be up to 20 emerging. I gave my run-down of communication types that currently exist and how they might change in the future here:

Updating Types of Interactions in Online Education to Reflect AI and Systemic Influence

You will probably want to pick several types of interaction for different parts or times in your course. Again, if you don’t plan for it from the beginning, it may never make it into your goals, objectives, or lesson plans. However, please keep in mind that you can come back and change/add/remove types throughout the design process.

Also, make sure to match the different types of interaction with the methodology and power structure you selected earlier, at least as you initially see them working out in your course. If you don’t have a good match for your previous choices, then you probably need to consider adding some appropriate interaction types.

Communicative Actions

The final theoretical part to think about will probably be something that you consider now, but come back to once you have an idea of what activities you want in your course. But I will cover it here since it is also in the article above, and it helps to think about it from the beginning as well. Once you know the power structure, methodology, and types of interaction you want, you will need to think through the form that various communication acts will take in your course.

There are many different theories of communication – one that I have found works well for instructors is Learning and Teaching as Communicative Actions (LTCA) theory (based on the work of Jurgen Habermas, but created mainly by one of doctoral committee members Dr. Scott Warren for full disclosure). Current LTCA theory proposes four types of communicative actions:

  • Normative communicative actions: communication of knowledge that
    is based on past experiences (for example, class instructions that
    explain student learning expectations).
  • Strategic communicative actions: communication through textbooks,
    lectures, and other methods via transmission to the learner (probably
    the most utilized educational communicative actions).
  • Constative communicative actions: communication through
    discourses, debates, and arguments intended to allow learners to make
    claims and counterclaims (utilizing social constructivism and /or
    connectivism).
  • Dramaturgical communicative actions: communication for purposes
    of expression (reflecting or creating artifacts individually or as a group
    to demonstrate knowledge or skills gained).

As you can see, you will most likely need to mix these during the class – even for each lesson. The goal on this part is not to pick one or two, but to think through how you communicate what is happening in your course. Think through the activities you will have in your course, and then match those with at least one communicative action and power dynamic/methodological combination.

Pulling It All Together

So that is really it for a quick run through some of the basics of theory that can help you begin to design an online course. Like I have said, there are many other theories than those covered here, and deeper/more complex ways of looking at the ones that were covered. This is meant to be a quick guide to just get started, whether you are designing a new online course from scratch or converting an existing on-campus course to an online version. If you looked at the article, you saw that there is a one-page worksheet at the end to help you work through all of these theories in a fairly quick manner. I have also created a Word Doc version, a Google Doc version, and a PDF form version that you can use to fill out and use as you like.

In parts 2 and 3, I want to go back to some topics I have covered before – but for now, here are links to past posts that cover those basics if you can’t wait:

Goals, Objectives, and Competencies

An Emergency Guide (of sorts) to Getting This Week’s Class Online in About an Hour (or so)

Update: I wasn’t clear enough that this is a basic beginner’s way to look at the terms and ideas that are used in learning theory. This will continue to go deeper as I look at other areas of theory in future posts. Some people are not happy that I avoided using the term “Critical Pedagogy” anywhere in the article. I apologize for that. My main thought was that Critical Pedagogy is often classified as an educational philosophy because it puts theory into action, and therefore it would be better to cover that in practical areas like formative evaluation, writing objectives, creating content, etc. Examining power dynamics, who controls communication, and what forms communication take are one of the foundations of being critical about education, so it is still the foundation of everything in this post.

A Template, a Course, and an OER for an Emergency Switch to Online

So the last few weeks have been… something. Many of us found ourselves in the rush to get entire institutions online, often with incredibly limited resources to do so. I’ve been in the thick of this as well. Recently I shared some thoughts about institutions going online, as well as an emergency guide to taking a week of a class online quickly. I would like to add some more resources to the list that we have been developing since those posts.

First of all, I would like to repeat what many have said (and what I tried to emphasize in that first post): take care of your self, your family, and those around you first. Don’t expect perfection from yourself. Practice self-care as much as possible (I know that easier said that done). Then make sure to take care of your students as well. Communicate with them as much as possible, be flexible, remember that many aspects of their lives have been suddenly upended, and above all, make sure to be a voice of care in these times.

I also know that at some point, you will be expected to put your course online and teach something, whether you think it is a good idea or not. So for those that are at that stage, here are some more resources to help.

First of all, I am working with some other educators to put together a free course called Pivoting to Online Teaching: Research and Practitioner Perspectives (I didn’t really like the word “pivot,” but I was overruled). It is a course that you can take for free from edX, but for those that don’t want to register, we have been placing all of the content on an alternative website that requires no sign-up. Lessons are being created in H5P (remixable) and traditional html format. Archives of past events are also being stored here as well. We are halfway through Week 1, so plenty of time to join us.

As part of that course, I created a module template for an emergency switch to online. This is basically a series of pages that work together as a module that you can copy and modify to quickly create course content. It tends to follow many of the concepts we are promoting in the class (Community of Inquiry, ungrading, etc), but it can also easily be modified to fit other concepts as well. I basically went through my earlier post “An Emergency Guide (of sorts) to Getting This Week’s Class Online in About an Hour (or so)” and followed it in making a Geology module. Then I add some notes in red to talk about options and things you should think about if you are new to this. You can find the Canvas or IMS Common Cartridge version in the Canvas Commons that can be imported in Canvas, or downloaded and imported to systems that support IMS. However, since there are also other systems that don’t use either of these formats, I also made a Google Docs version as well as a Microsoft Word version for download.

And finally, the OER – our book Creating Online Learning Experiences:A Brief Guide to Online Courses, from Small and Private to Massive and Open is still available through Mavs OpenPress in Pressbooks (with Hypothes.is enabled for comments as well). I want to highlight a few of the chapters:

Of course, I like the whole book, so it was hard to pick just a few chapters, but those are the ones that would probably help those getting online quickly. When you get more caught up, I would also suggest the Basic Philosophies chapter as one to help guide you think through many underlying aspects to teaching online.

An Emergency Guide (of sorts) to Getting This Week’s Class Online in About an Hour (or so)

With all of the concern the past few weeks about getting courses online, many people are collecting or creating resources for how to get courses online in case of a last minute emergency switch to online teaching. Some are debating whether to call it “emergency remote teaching” (or some variation of that) instead of actual “online teaching.” I agree with the difference, but don’t think that the academic definitions of either one really brings about much change in the practical work of getting online.

There are many problematic issues to address that many are not talking about. Accessibility, student support, and social support structures that schools provide don’t always switch online so well. Some students are even being kicked off of campus, with little mention being made of finding out where they will go if they don’t have a place to go this early, if they can afford to get where they need to go if so, and if the environment they go to can even allow for them learning from online. On top of all of that, few are talking about the difficulty and chaos that going online will create.

Of course, a lot can be said about if closing schools or going online instead of canceling is a good idea. All are good questions to ask. But a lot of people out there have already been forced to go online whether they agree or not, and many more will be forced to do so in the coming weeks. So we have to talk practical steps for those that are in this predicament.

There are many okay to good guides out there for switching online. I see most of them will tell faculty to examine their syllabus to see what can and can’t go online. This is a good first step, but it often ends up being the last step mentioned in this process. There needs to be some quick and blunt guides for what that means to examine your syllabus. So I am going to dive into that here.

Most Instructional Designers will be able to put a week’s worth of a class online in a very short amount of time IF given free reign to apply effective practices focused on the bare minimum needed and a complete set of content based on those principles. Once IDs start getting away from that – adding in time consuming online options that faculty love but that are not absolutely necessary, or waiting for faulty to get them content – the time to create a class increases quickly. However, if you are willing to focus on the bare bones of good online course design, there are many things you can learn from IDs in a pinch.

As I go through this, I will be addressing accessibility issues along the way. The main thing to keep in mind is that media (mainly video, but also images) is easy to make accessible (due to built in alt tags and captioning features), but also the most time consuming. Auto captioning usually doesn’t cut it. You will still need to manually read through and correct any mistakes by hand. The more you can get away from relying on video or video services, the less time it will take to prepare course work (in general).

The first step in going online is to talk with your students about what that would mean before you are forced to make the switch. Talk with them about what it takes to learn online. Have them go through your syllabus and brainstorm ideas for how to transition your objectives to online. Give them the freedom to suggest changes to objectives, or to even think of different activities to meet objectives. Ask them to talk to you privately if they don’t have Internet access at home, of they need other support services. Make sure they all have a way to check in with you (just so you know they are okay), and a back-up method or two in case the main communication method is not working well (or goes down).

If you have already been forced online, or there will be no class meetings between now and when the switch happens, you will need to think through this yourself. Of course, thinking through this yourself might help you guide the discussion with your students – so do it either way.

Content Creation

The first thing to ask yourself is how new information/content/etc will be communicated to students:

  1. One-Way Communication: Typical lecture method, where you share the new information that students learn. Easiest communication to make accessible, but captions could still be time consuming if you are relying on video (especially longer ones). If your goal is one way communication, you don’t need synchronous video tools, even for questions (students can contact you for that, email them, comment if you use a blog, etc). Also, note that this type of communication can be made to work on mobile devices easier.
  2. Discussion Between Instructor and Students: If you really want the ability to interact, and not just answer questions, you do need to look at tools for interaction. Discussion forums are the most accessible (but a little less human), while video conferencing tools are more problematic in regards to accessibility. For example, people with various hearing issues report that Zoom’s accessibility tools start to fall way short of ideal once four people get on a session. So if you really need this, you might want to consider using small group structures that can use a variety of tools (even a phone). Which would bleed over into another communication modality:
  3. Students Communicating With Each Other: Yes, this would include small group discussion. However, also consider how you can encourage students to use your class as a support network. Don’t just lock-down class tools to only be used for class activities. Help students get that human connection they will start to miss once social distancing sets in. This communication modality can be very effective for mobile devices and accessibility needs if you can be flexible about tools and structures.

Here is the thing that will save you the most time: If it were up to me for a class I was teaching, I would not try to schedule meetings for online lectures or even record videos of my lectures to get those online. It is possible to do that well when going online, but time consuming and problematic in regards to accessibility. Even typing out my lecture for the week can take a while. I would go straight to the Internet:

  • With so much out there, you can probably find articles, blogs, websites, etc that contain the content you need in a 15-30 minute search (or less if you already know of some sources).
  • Then use this link to see how to set up a really fast accessibility check tool in your browser. Use this tool on each source you find.
  • Be careful of video sources – make sure they have accurate captions.
  • Then take another 15-30 minutes to create a content page or blog post that lists each source and adds any core concepts you couldn’t find. This will be the most accessible form of content to make, as well as mobile friendly, as long as the services you use are accessible and mobile friendly.
  • This is a great way to get a wider array of perspectives on course topics than a textbook usually provides. But check to make sure you have diverse perspectives – if your list relies heavily on white Western heterosexual cis males, then you will need to change the parameters of your search to be more inclusive.
  • If needed, you can print articles out on paper for those without Internet access (and just hope there is a way to get it to them still functioning).

Obviously, if you have a textbook that you base your lectures on, they you already have a source of content that you can write up some notes on. This won’t necessarily be the most diverse perspective, but it will be quick. Just be sure to think through issues like students that couldn’t afford the textbook (ahem – OER?), if they can access the eBook texts at home, and so on.

Even more advanced quick method: Turn to some student-centered design methodologies to make the course more engaging:

  • Spend the 15-30 minutes creating an activity for your students to go find the content for the week (online, at a library, etc).
  • Towards the end of the week, create a page or blog post collecting those sources with your commentary.
  • Put some time into creating something more than just “go find content!”
  • Think through how to address accessibility issues, as well as how students that don’t have Internet will find content. Be flexible on that last one – not every student can just go to the local library when they want (and what if libraries close?).

Be ready to have to use the mail service for some students if you have to, and don’t worry too much about deadlines if so.

If you really need to use video, you will need to have well-edited captions, This can take a while. There are really three main options you have:

  • Option 1: Record your video, upload for auto-captioning, and then edit the captions for errors. Not all video upload services allow all of this, so check before time. This will probably sound the most natural, but you will also probably be surprised at how much time you waste going “ummm…” or “let me start over….”
  • Option 2: Type out your content and read from the page, without worrying about the way it makes you look “stiff.” You don’t usually write the way you talk, so it will just have to come out that way in crunch. But you will probably focus without too many side tracks. If you keep the video length to about 2-3 minutes, you can probably write and record it in 30 minutes, depending on how fast you write.
  • Option 3: Record your video, upload to YouTube or something else that has auto captioning, download the auto captions, edit for mistakes (not style), and re-record the video reading this script. The fact that it was based on your natural speech will make it sound more natural. Plus you had a built in practice run. A better end result (not perfect), but also more time consuming.

Activity Creation

This part of the course could possibly be the easiest part to create, or the hardest part, depending on your topic. If you have extensive lab requirements that can’t or aren’t simulated online, you will need to get together as a department and figure out how to translate that into the online space. Unfortunately, there is no quick way around that.

Also, keep in mind that you don’t have to come up with a project or test every week. Sometimes projects take more than one week. Sometimes it is good to just relax and learn. Plus, tests are a problematic concept to begin with, and proctoring solutions will be hard to implement when a lot of people start staying home in the same house. So as much as you can move away from high stakes big tests, the better the online experience will be for you and your learners.

Here is the thing that will save you the most time: One thing that is very cliche but effective is to use your LMS test tool to create a low stakes understanding check:

  • 5-10 questions that covers the core things students should have learned for the week.
  • Give students unlimited attempts so that learners can take them over as needed to get all the questions right.
  • This is not the coolest online design method, but it does give students some relief to know they are on the right track.
  • It should only take 20-30 minutes to create 5-10 questions… if your focus is on making sure students have had some contact with core concepts, and not on trick questions or “gotcha!” fake answers.
  • The goal with this kind of activity is not to catch cheaters, but to help students know what you think is important.

Even more advanced quick method: Really what I would focus on is creating authentic / experiential / etc projects that allows learners to engage more deeply with what they are learning:

  • Think of something that would allow learners to apply the course content to their real lives.
  • Think of something that would also let them apply course knowledge to a real world situation.
  • Let students think of how they will communicate what they have learned. Don’t limit them to just what you think they should produce (like a paper).
  • If you are spending more than 15-20 minutes writing out the instructions to this, you are over-thinking it.
  • If you spend less than 5 minutes writing instructions, you are probably not giving students that are possibly new to this level of agency enough guidelines on how to do the project. Remember the students that are new to all of this.
  • Provide 3-4 example ideas of how students could complete the assignment. Don’t worry too much of several students use your idea. Think of some outside the box ideas, like skits, graphics, etc.
  • Remember flexibility and accessibility. If you have to accept a hand-written project sent through the mail – or maybe even one transcribed by a sibling or a spouse – then no problem. Just be glad they are learning.
  • Its best to grade these in more of a general way. Don’t get bogged down in exact point totals for every mistake. Consider ungrading if possible in your institutional structure.
  • For larger classes, have students self-organize into groups based on interests and/or desired artifact to create. Don’t forget that students might need help self-organizing online or at a distance. Again, remember those with accessibility issues, or internet access issues.

At this point, I would have spent about an hour creating my class for the week. This would have been 30 minutes researching and writing a blog post containing the week’s content (FYI – this blog post is waaaay too long for that, so don’t follow my example :) ), and 30 minutes creating a student-centered authentic open-ended project. And this project would take students 3-4 weeks to complete. If you are new to creating classes this way, your first time or two at doing this might take longer (especially if you are new to the tools you will use). Also, if you need other things like video or labs, that will take more time that this. But there is something else to plan for as well.

Course Communications

I have been touching on communication issues through out this post, so I will try not to repeat what I have already stated here. You will need to communicate other things like class norms and online etiquette outside of content and activities. While it may seem like I am against synchronous communication methods here, that is not the case. You can use both. You will just need to consider how to use synchronous tools in ways that addresses accessibility and internet access issues, as well as make sure the tools works well on mobile devices (which is hard to do for all, since a lot of that is personal preference).

Like I mentioned in a previous post, scheduling synchronous sessions can be tricky at best in shutdown / quarantine situations. Yes, you can do the “we will record it and you can watch later if you miss” thing, but that is also problematic. Some of the reasons that cause people to miss – like overwhelmed internet service – will prevent the same people from watching the recorded video. They will also feel a bit left out.

But here is what I would do for communication in a sudden switch to online:

  • Talk to students before hand if possible. Create a plan.
  • Use synchronous tools for open communal office hours.
  • Set-up alternate options for communication – phone, email, even mail if needed.
  • Send out an email, text, mass phone call, or something weekly. This is a good way to humanize your online learning – stick with those principles as much as possible.
  • Don’t totally avoid video if you don’t want to. Sometimes, a quick 1-2 minute “welcome to the remote version of class” can help. Just get the captions right first!
  • Students might be new to learning online. If your school has a Code of Conduct for online interactions, make sure to follow that and make your students aware of it. If your school doesn’t have one, or has one that you feel is not adequate enough, consider creating your own Code of Conduct for your class.

Every time I edit this post, I add a bunch of new sentences all over the place. There is a lot more that could be said, but I will stop here. I hope that this post gives faculty the idea that they can focus their classes on what works in online learning, not just re-creating the face to face class. Also, I hope this empowers you to save some time in the process, without sacrificing effective practices in the name of an emergency. If you have access to an Instructional Designer to help, please talk to that person even if you have read this post. They can give you even more specific advice related to your unique course needs than this post can.

Let’s Talk About “Shifting an Entire University Online as Disaster Preparedness”…

With the rapid spread of Covid-19 (aka “the Coronavirus” in shorthand for now), there has been an explosion of discussions about preparing for quarantines and societal closures of various kinds. Among these discussions are moving conferences, courses, and even entire institutions online. I have been tweeting about this for a few days, so I wanted to collect and expand some of my thoughts on this topic.

Since I have a Masters in Instructional Design and a Ph.D. in Learning Technologies, we spent many assignments in courses for both degrees discussing various benefits and pitfalls of online learning, and yes, switching to online in the case of an emergency was frequently covered. It’s a complicated and problematic idea, so this will be a bit dark and complex / rambling post to make.

First of all, let me start off by stating that no matter how well you plan, switching to online will be more chaotic and hard than you can imagine, and it will cause greater damage to disadvantaged students than you will probably notice. Your first and foremost duty is to consider your disadvantaged learners first, and to work on navigating chaos rather than trying to stop it. Because you won’t avoid chaos. Remember – it is called DISASTER preparedness for a reason.

Many disaster preparedness plans I have seen, as well as many conference and institutional reactions to Covid-19, seem to only focus on able-bodied younger people that are not older, immuno-compromised, living with those that are older or immuno-compromised, already affected by food insecurity, homeless or on the edge of homelessness, affected by digital redlining, dealing with disability, held back by systemic discrimination and intersectionality, and so on. The reactions are only taking into account young able-bodied people living with other young able-bodied people, with maybe a link to an external resource that mentions everyone else (occasionally in passing). This is not going to cut it. Please keep your entire population in mind, not just those that will probably be okay no matter what you do.

Next, you need to realize that just because a conference or university can pivot to online, there are still institutional / organizational barriers to the overall idea of online. Some just won’t pivot because they are against online in general. If an institution does not allow remote work options, they probably do so for reasons that other institutions that do so have already dealt with. Its usually an institutional preference to be against remote work at this point, so that will likely carry over to online courses as well. Therefore, don’t assume some big switch will happen in those situations. Besides, how will institutions switch courses over to online if they don’t already have the procedures in place for their employees to go online?

Where I work for my day job, we already allow remote work, and already have a robust array of online courses. We have also been providing LMS shells for every course section, to be utilized even in on-campus courses. The structure is already there for the switch. Yes, I know LMSs are not hugely popular right now – I’m with you on that. But the important issue is that there is a space online there already for every class. It could be in WordPress blogs or many other tools for all that it matters.

But I guess we should talk first about how there are different types of disasters at different levels. There aren’t really any hard lines between these categories (I kind of made them up on the spot)… but in general, you see a few different kinds (and probably more than these):

  • Individualized/localized disasters: This is anything from one or more person getting sick in an atypical way (for them, at least), to something that affects only certain people in one or some locations. Tornadoes are an example of that – living in Texas, we frequently have to consider plans on how adjust courses based on the fact that a portion of our class will be affected and the other portion won’t. These plans have to often create more flexibility for students. But they can also happen out of the blue, and cause portions of your courses to be out of sync.
  • Displacement disasters: These can affect entire cities, regions, states, or even countries. There might be some warning, but the situation does come about very suddenly. Things like hurricanes, floods, and other mass displacement events. In general, your students’ first priority will not be education. They will need to get out, find shelter, food, water, etc. Usually, this will call for cancellation, postponement, etc. I teach online courses for UT Rio Grande Valley. When hurricanes were heading for the valley, we had to postpone even the online courses. People are usually fleeing something, so don’t plan to switch to online right away – or maybe even ever. Do look for ways to find out where your students are and how they are doing. Don’t just lock down the campus and say “good luck!”
  • Quarantine/lockdown/closed borders disasters: To be honest, this is the one that most of us did not think much about in the U.S. until Covid-19. On a global scale, it is probably more common than we realize. Neighborhoods, cities, states, etc could be quarantined, closed down, blocked off, etc due to disease, civil unrest, climate change displacement, even economic issues. Some of these might happen suddenly, while others might happen on a slower basis with time to prepare. I think some institutions think you can just switch your face-to-face courses to video conference tools and be done with it, but it is really much more complicated than this.

Something to remember: not all disasters bring about changes for all learners. Some are already living in disaster conditions. We tend to make disaster plans with the stereotypical “traditional” student in mind – young, flexible, and financially stable people that are so focused on education that they will skip meals or sleep to study more. These imaginary students are also probably perfectly flexible in our minds, so our first plan is that education has to be switched online right away to help them. The truth is, many of our students are already not sure where they will get their next meal or place to sleep. While you are thinking about how to adjust your class for disaster preparedness, why not consider how those changes could go ahead and happen at your institution – as a way to help those that are already in a disaster situation personally?

So… how to make the switch to online – if you are at one of places that do that? First of all, I’m focusing mostly on higher ed here, which might also work for High School classes as well as maybe some Junior High contexts. Not sure if I would recommend K-6 going online – but if someone has found a way to do that without leaving behind disadvantaged students, I am all ears.

First I want to touch a little bit on what will likely happen IF some try to make the switch. I realize that more schools have some kind of “sudden switch to online” plan than many of us think, so it is possible that some schools might go ahead and make that switch. These kinds of plans were a thing for a while, so I know they are out there. Some of those plans are inadequate (probably mainly from becoming outdated), but also probably based on some popular faux-futurist scaremongering and not true trend analysis. But that depends on a lot of things, so that may not be the case your plan. If the plan focuses on one big, easy solution  – its not going to work that well. Look for one that is realistic with the idea that “its gonna be rough, here are several possible options and ideas.”

Even those well-thought out plans can not account for everything, so a sudden or relatively slow-moving switch to online will be mass chaos whether a school has no plan, has an adequate plan, or has a detailed plan. Its just that the detailed flexible plans will make people realize there will be chaos.

The ability to navigate through the chaos will probably depend on the size of your Distance Education / Instructional Design group in relation to the number of classes. Those that have small DE/ID groups will find that even a great contingency plan falls apart without enough people in their DE/ID department. Those with a large DE/ID group will find the chaos much more manageable (even if they lack a solid contingency plan) thanks to having a group of people that know how the tools work, as well as the theory and research and history of how to use it (and how not to).

Places that have found themselves in need of a mass switch to online have also found that humans can manage chaos to some degree even without a plan, and that the switch can happen… it just won’t be something to brag about. What will likely happen – assuming the typical modest contingency plan & small DE/ID group – is that most faculty will suddenly ditch a lot of what they are doing in their on-campus courses. They will just stick with “the basics” for their switch online (whatever that means to them – more than likely email with attachments and synchronous video conference sessions). Which, of course, raises all kinds of questions about what they really do in their class if so much can be ditched last minute. Many will find out that email will work just the same as the LMS for a lot of what they want to do, which will lead many people to live the reality that the technology should not be the focus of course design, even if they don’t realize it.

(One weird thing down the road that I would predict is that instructors with the overall attitude of “I just do whatever” will probably come out looking like stars, and will probably get several keynotes/TED talks/etc out of it. The lack of structure and planning in their on-campus courses will, unfortunately, work out quite well… for once. Meanwhile, the Instructional Designers and Tech Support People that saved their butts by fixing their mistakes behind the scenes will sit in the crowd, ignored…. but showing up still because that is what we always do…. yes, there is a reason I sound like I am speaking from experience… :) )

But let’s also discuss some ways to at least try and steer things towards some better outcomes, even amidst the chaos. Keep in mind I am talking about possible options that may not work out perfectly every time. Disaster response happens in situations when a lot of things like proper design of online courses (which takes a long time to do properly) has to take place in a very short time frame. Nothing is wrong with going with what just works in the moment. These are just some ideas of what to think about and possibly try to incorporate in your plans.

So, if you really are going to go down the path of planning for a mass switch to online, the biggest issue you need to deal with is convincing professors to switch from synchronous methods to asynchronous as much as possible (I always assume everyone knows what is meant by that, but that is not always the case – here is a good summary if you are not totally sure what I mean here). I can’t stress enough how disaster could cause synchronous to break down.

When a disaster strikes, especially like a quarantine, people lose a lot of the control over their schedule. Issues such as “when food arrives,” “when family are available to check in,” or even possibly “when they are able to use the Internet” (uh-oh!) will possibly be out of their control. All parts of life suddenly have to become coordinated, scheduled, and controlled by others, who themselves are most likely doing the best they can to get supplies out with limited staff and mobility. Students will need the ability to work learning around society that is not fully functioning – not the other way around.

The problem with quarantines is that you have a massive strain suddenly on resources in a matter of hours that last longer than the usual cyclical strains. Whether the quarantine is short or long term, there will be possibly be rationing and rolling stoppages of all kinds of services to offset loads on systems (don’t forget how this could also affect cell phone service when local towers become overloaded). So what are you going to do when a quarter of your students don’t have Internet service when you schedule synchronous sessions, and then when those students have Internet, another quarter is hit by rolling blackouts, and so on? Or what about other issues, like those that suddenly have to change work hours to keep income flowing to themselves or their employer due to societal upheaval?

I know that oftentimes, the solution to this is “schedule the teaching session through Zoom or some other tool, but record it for those that miss.” From experience, I can tell you that more and more will start missing because they can, even if they are available. Then they start to not get the same learning experience as others since they can’t ask questions live, or feel as connected to other learners. I’m not saying “don’t do synchronous meetings ever,” but just consider those are not the best way to do online learning by default. There are additional consequences for those that miss that can harm them in the long run, so why not consider ways to make it equitable for all?

Even in the first disaster scenario I described – the individualized disaster – some people say that they will just get a camera and live stream their class. That might work great, but what if that student is in the hospital, and not in control of their schedule enough to be online when your class meeting happens? Are you expecting the hospital (or family caregivers if they are at home) to re-arrange their schedules around your class? Probably not a priority when someone is getting medical help. Just a thought.

Additionally, don’t forget the technical problems that can arise during mass migration to synchronous sessions (which many will be familiar with because they also happen in non-disaster situations as well).  For example:

or

…just for a few examples. There are hundreds to consider. For asynchronous as well – there are pros and cons to both.

While many know that there are major differences between asynchronous and synchronous course design considerations that have to be accounted for in the switch between those two modalities, sometimes we also forget that even switching from on-campus classrooms to synchronous online sessions is also not that simple.

If you have used Zoom or some other video conferencing tool in an online setting, you know that there are big differences between on-campus and online meetings. Just remember that not all of your students and faculty have used those tools in online learning contexts, so they will need guidance on how to do that. Where I work for my day job, they are already on top of that aspect.

There are also many other issues to consider, and this post is too long as it is. Hopefully you can start thinking through all of the unique complexities that a switch to online would run into. For example, are you prepared to let students that are locked in dorms rooms and bored work ahead in courses/programs to fill their time? Are you going to let people use campus tools to organize supply distribution, news updates, just chat if bored, etc if needed? Are you considering the mental effects of long-term quarantine, and how to address that while still in quarantine? There are so many to think about.

And to those that say “how likely is that to happen?” Well, not very to be honest. But that is kind of the point of emergency disaster preparedness. You never know until its too late, so think about it now.

A final note – I am not trying to draw hard lines between asynchronous and synchronous. Asynchronous does not mean you can’t ever have the option of synchronous class sessions. Its not always an either/or. Like I have said, there are also things you have to do because that is all you can in a pinch. These are just some ideas to consider. Keep in mind that there are many great ideas that mix both, like putting students in groups and having those groups meet synchronously. These options can work out very well (especially since smaller groups are easier to schedule common times). Or another idea – you can (and should) meet with individual students synchronously as well.

Don’t draw hard lines. It will be chaos. Plan to be flexible. Good luck, and Godspeed…

(Top photo by Kelly Sikkema on Unsplash)

Instructure Wars, Private Equity Concerns, and The Anatomy of Monetization of Data

The Annals of the Dark and Dreadful Instructure Wars of 2019

as told by

Matt, the Great FUD Warrior, Breaker of Keyboards, Smacker of Thine Own Head, Asker of Questions That Should Never Be Asked

If you were lucky, you were spared the heartache that came out of nowhere over the announcement that Instructure will possibly be sold to Thoma Bravo, a private equity firm.

Well, it should be stated that the concern from those that always have concerns over these sales announcements was expected. The quick “shush shush – nothing to worry about here” that came in response from people that usually understand the concerns over past Private Equity sales in Ed-Tech (Blackboard being the typical example) was the surprising part.

For the record, the first skirmish actually started when Jon Becker asked what possible outcomes there could be of the sale, Audrey Watters responded with her thoughts on that, and someone made a sexist attack on Audrey’s knowledge of private equity. Also for the record, they initially did not disagree with her points that prices would go up or that Instructure would be broken up and sold off. They stated this was impossible because there is nothing about Instructure that could be broken into parts. Many of us pointed out the sexists problems with the way he expressed his opinion (not his underlying opinion about PE), but he dug his heels in. We also pointed out that there were actually many things within Instructure that could be broken apart, but that was apparently grounds for fighting (even though it is true there are many parts of Instructure that could be broken off and sold, and just because Thoma Bravo has a history of buy-and-build strategy, there is nothing stopping them from still selling off parts they don’t fit their strategy. “Buy-and-build strategy” and “selling off parts” are not mutually exclusive). Within that argument, the idea that the all of the data that Instructure has been bragging about for a few years could either be sold or monetized (more on the important difference there later).

Sometime within the next day, Jesse Stommel made the tweet that really kicked off the main war (I don’t know if he was replying to comments about the value of data in the earlier arguments or it was a coincidence). This is going to be a long post, so I am trying not to embed Tweets here like I usually do. In what was an obvious reference to Instructure bragging that their data was core to their value as a company, Jesse made the comment that we can now know that this value they were bragging about has a price tag of $2 Billion dollars.

Now, can I just say here – I don’t think in any way that Jesse thought that “Instructure data actually cost $2 Billion.” I’m pretty certain he knows that personnel, assets, code, customer payments, etc all are part of that value. Its just that there was a lot of bragging about data being core to the company value, and a huge gap between the market value at the time) and $2 Billion dollars, and that his professional analysis was that data contributed to that in a big way.

Then there was some debate over the value of data in an Ed-Tech company. This was followed by some shooshing and tone policing towards anyone that thought there should be concern over the lack of transparency about data that Instructure has become known for recently, as well as concerns over what could change with new owners. This led to people retreating to their own corners to express their side without having to be interrupted with constant tangential arguments (and there is nothing wrong with this retreating).

Audrey Watters has written her account of the ordeal, which I recommend reading in its entirety first. I am tempted to quote the whole thing here, so really go read it. I’ll wait.

Okay, first I want to clarify something. In my mind, there is a difference between “selling data” and “monetizing data” even though there are obvious overlaps:

  • “Selling Data” is taking a specific set of data (like from a SQL data dump) and selling that to companies that will turn around and sell it to others (which does happen with educational data – more on that later). When someone says “what good is someone knowing that I submitted Quiz 2 back in 2016?”, they are referring to data as a set archive of database rows from a set date. It is kind of looking at data as a crop of apples that were harvested at a specific time. There was concern over this as a possibility, and we will look at that later.
  • “Monetizing Data” is any form of making money directly from creating, manipulating, transporting, etc data. This happens a lot in every day life, and not all instances of it are bad. The core business of most for-profit LMS companies is the monetization of data – nothing in an LMS works without data. Grade Books need data to work. Discussion forums are empty without data. Analytics dashboards show nothing without data. This is kind of looking at data like a the fruits of a field of apple trees that are constantly growing once picked. You could wipe an LMS database of all past data (well, assuming you could find a way to do so without shutting down functionality), but as soon as you turn it on it the code starts generating a massive set of new data. For many, the main concern with the monetization of data is who controls the data and what will they use it for in the LMS? Will I get manipulated by my own data being compared to past learner’s data without either of us knowing about it?

Now, to be fair, most responses were fairly nuanced between the two “sides” of the war. For the record, my “side” in the great Instructure War of 2019 is that “data has the potential to be used in ways that users may not want, which could include monetization, and both Instructure and their potential buyer are not saying enough about what their plans are.” I think that is close to what many others thought as well, but our position was mainly reduced to “all data bad!” While the other side was reduced to “data has no value so stop worrying!”, I do want to examine the idea of whether educational data can have value (to be sold or monetized) outside of either side.

Instructure’s View of Their Own Data

First, I think it is prudent to start with Instructure’s own view on their data. While it would be hard to reference they amount of bragging they have been doing about the value of data at conferences and sales calls, we only have to look at their own words on their own website to see how they view data.

First there is Project Dig. They start off by proclaiming that they have become “passionate about leveraging the growing Canvas data to further improve teaching and learning.” That passion “became our priority, and over the years we’ve provided greater access to more data and designed new, easy-to-understand (and act on) course analytics.” How can a priority of the company not be a huge factor in what they are worth? This is all under the banner of “we’ve been focused on delivering technology that makes teaching and learning easier for everyone.” Obviously, as an LMS, that focus is their main revenue maker as well. And now data is the priority for that focus.

FYI – the target launch date for their tools that will “identify and engage at-risk students, improve online instruction, and measure the impact of teaching with technology.” is…. 2020. Conveniently after the proposed sales date it seems. Again, how could this priority focus of the company that is improving what they offer to customers not be a huge factor in the current sales price?

But they do recognize that there are some problems with digging into data. What word gets mentioned A LOT in the FAQs about potential issues (hint: it involves a word combined with data that starts with “s” and rhymes with “felling”).

Some key highlights from the FAQs:

  • “Will your practices be consistent with your data privacy or security policies?”They say they “are not selling or sharing institutions’ data” – but only because they choose not to. It important to note that the question about selling is there because they feel they can do just that if they want. But they assure us they won’t. Of course, new owners can change that.
  • “Is this really just my data, monetized?”Basically, they say it is not an example of monetizing your data just because… they choose not to, not because they can’t. The implication still remains that the possibility is real and it is there. Then give examples of how they could. Again, new owners are not limited by this choice of the old owners.
  • “What can I say to people at my institution who are asking for an “opt-out” for use of their data?”This is the core problem many have with monetization of data: feel free to do it, just give me the option to opt out at least. They say a lot, but don’t really answer that question (which is very concerning).Important to note that they say “Institutions who have access to data about individuals are responsible to not misuse, sell, or lose the data.” Then they say they “hold themselves to that same standard.” Nothing says this couldn’t change with new owners. But how do you “sell” data that is worthless? They seem to think selling data is as possible as misusing it or losing it. It certainly would be a lot easier to say “no one is out there buying or selling educational value.”
  • (While they do say a lot about openness and transparency, many customers have expressed frustration at some lack in those areas.)

(an important side note in support of Watters point earlier is that, in addition to the main products of the company that each could be broken off and sold or things like assets, employees, etc, these data projects and services represent even more parts that could easily be broken off and sold if a PE firm so chooses to at any point)

Then I give you – Canvas Data. The doc for this service is really a whole page of ideas for how to monetize Canvas data, along with the existing tools to do it. Which is really the goal of the project: “customers can combine their Canvas Data with data from other trusted institutions.” What it doesn’t quite clarify here is that these institution include companies that sometimes charge money to create manipulate, and transport student data inside and outside of Canvas. Many people trust Canvas to vet these companies, but sometimes these arrangements are obscure.

I will give one example from an organization that I think is pretty trustworthy – H5P. H5P does integrate for free with Canvas for free. However, some activities designed in H5P generate grades (which is student data). If you want to transport that data back into the Canvas grade book, you need a paid account with H5P. This is just one example of how a company can monetize Canvas student data, even down to one small data point.

Now, while I see no reason to distrust H5P, I can’t force students to trust an organization they don’t know. What if they didn’t want grade generated on all these websites (because H5P is not the only company to do this)? What if they were not comfortable with a company profiting on moving around their grades? Or what if they were concerned about what the change in LMS ownership meant for all of this?

Anyways, all of this information is up on their website because Instructure believes that their data is very valuable, and that it can be sold. Why would they not point out that data is worthless by itself? Why would they talk about all of this if people weren’t asking these questions, if entities weren’t asking to buy data?

(And Instructure is not the only LMS doing this. Even Blackboard’s Ultra is already ready to do more with data, to monetize it today: “We’re not just handing you data. We’re surfacing data that matters when it matters most— to foster more personalized interactions and drive student success.”  In other words, they are not just doing what other LMS have always done and managed (handled) your data to monetize it, they are adding value by surfacing and doing more. They “drive success,” because success sells. If you have ever been to an LMS sales pitch, you know analytics, personalization, success, all these terms used on the pages I shared are key sales terms to convince organizations to sign the contract.)

The Marketplace for Student Data

One of the contentions of the “LMS data has little to no value” side is that no one is buying or selling student data, as in dumps of past records. It seems that there are existing marketplaces for student data, to the point that someone wrote a journal article on the whole thing: Transparency and the Marketplace for Student Data. “The study uncovered and documents an overall lack of transparency in the student information commercial marketplace and an absence of law to protect student information.” Sounds like a pretty good justification for concern over any student data out there, whether currently under consideration for sale or not. Why it that?

Taking the list of student data brokers Fordham CLIP was able to identify, Fordham CLIP sought to determine what data about students these brokers offer for sale and how they package student data in the commercial marketplace. There are numerous student lists and selects for sale for purposes wholly unrelated to education or military service. Also, in addition to basic student information like name, birth date, and zip code, data brokers advertise questionable lists of students, and debatable selects within student lists, profiling students on the basis of ethnicity, religion, economic factors, and even gawkiness.

That is not all.

Under the Radar Data Brokers

Get ready for this one: there is no evidence that educational data is even staying specifically within a dedicated student data marketplace. This article on under the radar data brokers compiled a list of “121 data brokers operating in the U.S. It’s a rare, rough glimpse into a bustling economy that operates largely in the shadows, and often with few rules.” Most of the entries on the list don’t get into the specific data they collect, so the fact that “education” appears three times on the list for the few that do is concerning:

  • BLACKBAUD INC.
    A “supplier of software and services specifically designed for nonprofit organizations. Its products focus on fundraising, website management, CRM, analytics, financial management, ticketing, and education administration.” (Wikipedia)
  • MCH INC. DBA MCH STRATEGIC DATA
    MCH “provides the highest quality education, healthcare, government, and church data.”
  • RUF STRATEGIC SOLUTIONS
    A marketing firm owned by consumer identity management company Infutor with a focus on travel, tourism, insurance, e-commerce, and education.

These are places already in business, already buying and selling data. If you look at the chart of the attributes (types of data) that Acxiom collects, “education” is one. Its not hard to believe that – if they don’t already have it – they would be very interested to add “I completed quiz 2 on such-and-such date” to that massive collection on each person.

So Why $2 Billion for Instructure?

The only concrete answer we have now is “nobody outside those privy to the details knows.” There are many speculations out there – some of which started the Instructure Wars. One of the main ones I haven’t touched on, that probably summarizes one side of the Instructure Wars, is that the data adds little to nothing to the value of Instructure (despite their own claims to the contrary), but it is “simple math” getting from the current market value to $2 Billion.

I think there is a point to be made in the “simple math” argument (although I would be careful calling it “simple” or claiming “people just don’t understand” if they don’t agree). I would say that even basic Math has to account for the value of data (both at the price it can be sold and the value it can add through monetization). Autumm Caines made the comparison that data is the engine to the LMS car, and you don’t really buy one without the other. In fact, those that are claiming that the data have no value are accusing Instructure of being the shadiest used car sales people in the world: “If you will buy this new fancy car, I will throw in the engine for free!”

However, it seems that different Instructure investors are now disagreeing with the $2 Billion price tag, some thinking it is too low. In fact, they think it “significantly” under values the company. I would assume these investors have access to details about the price of the sale, and if the $2 Billion was simple math, I don’t know if there would be much room to disagree. The cost of the code would be tied to revenue it generates, and therefore would be static and easy to calculate. Various aspects like assets, personnel, and the value of the income from investors are all relatively fixed. Even the future revenue is based on various predictive factors that would be hard to argue.

Seeing that the data is the newest priority of the company, and its value is difficult to calculate, might that be the best candidate for the source of this disagreement? Maybe, but the only for really, really complex data that leads to complex calculations that could easily be off. And LMS data is pretty straight forward… right?

Well, not so much. Kin Lane took a look at what the public APIs of Canvas reveal about the underlying data, and its a dozy. That is another article that I could quote the entire thing, so please take time to read it. I know the page looks long, but that is because he lists 1666 data points (!!!) in just the public APIs alone (while pointing out there are many private ones that probably have many more). He also points out how this structure and the value that it brings easily accounts for the $2 Billion price tag and more, especially when combined with the costs of code and people and customers and so on.

Now, of course, I am willing to bet that there are multiple factors that are causing the investors to fight over price. It could be that they think Canvas’ movement into the corporate training space is about to take off. It is probably a combination of many factors, some I have not even touched on here.

But it is just not far-fetched to think that there is a real possibility that the data is driving the prices, either a commodity to be sold, and/or a service to be monetized.

As I finished this post, Jame Luke published a blog post based on the economics side of the issue. While it does expose that many of us (especially me) are using the wrong terms, the basic idea that the PE will not have education’s best interest at heart and that the data is driving the economics here. He touches on some of the reasons why about data I do here, goes into a lot more depth, and shares several plausible scenarios for the future goals of Thoma Bravo, including one that makes the monetization of data very central to the future sales value of the company. A detailed but necessary read as well. I don’t have time or energy to go back and correct what I got wrong based on this post, so feel free to blast me in the comments if you so desire.

The Ferocity of the Battle

Right now, my direct messages on Twitter are booming with multiple people that are all flabbergasted as to why this is so controversial. We get that there would be disagreement, but the level of ferocity that one side has had in this battle is surprising. Especially since we all thought these were people that agreed that data does bring significant monetary value to a company.

“Data is the new oil!” we were told. But now we are told it was only… black paint all along? Something that will be there in every painting, but doesn’t cost much to be there?

Many people have offered thoughts on why some are some determined to fight this fight. If they get shared publicly I will probably come back and add them here. For my part, I just don’t know. People that want to protect student’s from misuse of data I get, but they have really gone to extra levels of fight over this (beyond what they usually do, that is). The real surprise is the shear irritation from the Learning Analytics community. I know – how did that happen? None of this Instructure kerfuffle says anything bad or good about Learning Analytics, yet they are in the thick of the battle at times.

Still, why is the basic message of “we should be vigilant to make sure that a company that has been a bit opaque with data issues recently gets sold to another company that may or may not be more open, because their data has the possibility to be exploited” so controversial right now? Why must so many people be proven right on the exact price of data? I don’t know.

For me, there could be a news report tomorrow that has Instructure stating “yep, the price was all about the data,” and I would just respond with “okay, thought so.” I get the feeling I am going to be buried in a barrage of snide Tweets if the opposite narrative goes in the news.

Which, let’s be honest, that will be the narrative from Canvas. They have to say there isn’t much value to the data no matter what the truth is. If they let on that it has actual, real value, every single school, teacher, and student will immediately want to sue for their share. Even if there is no lawsuit, the public relations nightmare would cause untold damage as people get mad their data had direct value in the massive sale.

Of course, the reality is that it does not matter what comes out. Canvas already bragged about the value of the data they are monetizing. They already are using it in ways that people don’t want. People have a good reason to be upset about the monetization of their data because it is already happening.

Thus ends the accounting of the never-ending Instructure Wars, as best can be summarized near the end of the dread year 2019.

As the wars drag on and alliances are strained, many began to wonder….

Will this ever end….