People Don’t Like Online Proctoring. Are Institutional Admins Getting Why?

You might have noticed a recent increase in the complaints and issues being leveled against online proctoring companies. From making students feeling uncomfortable and/or violated, to data breaches and CEOs possibly sharing private conversations online, to a growing number of student and faculty/staff petitions against the tools, to lawsuits being leveled against dissenters for no good reason, the news has not been kind to the world of Big Surveillance. I hear the world’s tiniest violin playing somewhere.

It seems that the leadership at Washington State University decided to listen to concerns… uhhh… double down and defend their position to use proctoring technology during the pandemic. While there are great threads detailing different problems with the letter, I do want to focus in on a few statements specifically. Not to specifically pick on this one school, but because WSU’s response is typical of what you hear from too many Higher Ed administrations. For example, when they say…

violations of academic integrity call into question the meaningfulness of course grades

That is actually a true statement… but not in the way it was intended. The intention was to say that cheating hurts academic integrity because it messes up the grade structures, but it could also be taken to say that cheating calls into highlights the problem with the meaningfulness of grades because cheating really doesn’t affect anyone else.

Think about it: someone else cheats, and it casts doubt on the meaning of my grade if I don’t cheat? How does that work exactly? Of course, this is a nonsense statement that reals highlights how cheating doesn’t change the meaning of grades for anyone else. Its like the leaders at this institution are right there, but don’t see the forest for the trees: what exactly does a grade mean if the cheaters that get away with it don’t end up hurting anyone but themselves? Or does cheating only cause problems for non-cheaters when the cheaters get caught? How does that one work?

But let’s focus here: grades are the core problem. Yes, many people feel they are arbitrary and even meaningless. Still others say they are unfair, while some look at them as abusive. At the very least, you really should realize grades are problematic. Students can guess and get a higher grade than what they really actually know. Tests can be gamed. Questions have bias and discrimination built in too many times. And so on. Online proctoring is just an attempted fix for a problem that existed long before “online” was even an option.

But let’s see if the writers of the letter explain exactly how one person cheating harms someone else… because maybe I am missing something:

when some students violate academic integrity, it’s unfair for the rest. Not only will honest students’ hard work not be properly reflected…. Proctoring levels the playing field so that students who follow the rules are not penalized in the long run by those who don’t.

As someone that didn’t cheat in school, I am confused as to how this exactly works. I really never spent a single minute caring about other students’ cheating. You knew it happened, but it didn’t affect you, so it was their loss and not yours. In fact, you never lost anything in the short or long run from other student’s cheating. I have no clue how my hard work was not “properly reflected” by other students’ cheating.

(I would also note that this “level the playing field” means that they assume proctoring services catch all “cheaters” online, just like have instructors in the classroom on campus meant that all of the “cheaters” in those classes. But we all know that is not the case.)

I have never heard a good answer for how does supposed “penalization” works. Most of the penalization I know of from classes are systemic issues against BIPoC students that happens in ways that proctoring never deals with. You sometimes wish institutions would put as much money into fighting that as they would spying through student cameras….

But what about the specific concerns with how these services operate?

Per WSU’s contract, the recorded session is managed by an artificial intelligence “bot” and no human is on the other end at ProctorU watching the student. Only the WSU instructor can review the recorded session.

A huge portion of the concern about proctoring has been about the AI bots – which are here presented as an “it’s all okay because” solution…? Much of the real concern many have expressed is with the algorithms themselves and how they are usually found to be based on racist, sexist, and ableist norms. Additionally, the other main concern is what the instructor might see when they do review a recording of a student’s private room. No part of the letter in question addresses any of the real concerns with the bigger picture.

(It is probably also confusing to people whether or not someone is watching on the other side of the camera when there are so many complaints online from students that have had issues with human proctors, especially ones that were “insulting me by calling my skin too dark” as one complaint states.)

The response then goes on to talk about getting computers that will work with proctoring service to students that need them, or having students come in to campus for in-person proctoring if they just refuse to use the online tool. None of this addresses the concerns of AI bias, home privacy, or safety during a pandemic.

The moral of the point I am making here is this: if you are going to respond to concerns that your faculty and staff have, make sure you are responding to the actual concerns and not some imaginary set of concerns that few have expressed. There is a bigger picture as to why people are objecting to these services, which – yes – may start with feeling like they are being spied on by people and/or machines. But just saying “look – no people! (kind of)” is not really addressing the core concerns.

QM and the Politics of the “Unbiased” Bias

So it started off innocent enough, with a Tweet about concerns with the popular QM rubric for course review:

Different people have voiced concerns with the rubric through the years, usually not saying it is all bad or anything, just noting that it presents itself as a “one rubric for all classes” but seems to show a bias for instructor-centered courses with pre-determined content and activities. Which describes many good classes, don’t get me wrong. But there are other design methodologies and learning theories that produce excellent courses as well.

The responses from the QM defenders to the tweet above were about many things that no one was questioning: how it is driven by research (we know, we have done the research as well), lots of people have worked for decades on it (obviously, but we have worked in the field for decades as well; plus many really bad surveillance tools can say the same, so be careful there), QM is not meant to be used this way (even though we are going by what the rubric says), it is the institution’s fault (we know, but I will address this), people who criticize QM don’t know that much about it (I earned my Applying the QM Rubric (APPQMR) certificate on October 28, 2019 – so if don’t understand, then whose fault is that? :) ), and so on.

Now technically I wasn’t “criticizing” QM as much as discussing it’s limitations. Rubrics are technology tools, and in educational technology are told not to look at tools as the one savior of education. We are supposed to acknowledge their uses, limitations, and problems. But every time I want to discuss the limitations of QM, I get met with a wall of responses bent on proving there are no limitations to QM.

The most common response is that QM does not suggest how instructors teach, what they teach, or what materials they use to teach. It is only about online course design. True enough, but in online learning, there isn’t such a clear line between design and teaching. What you design has an effect on what is taught. In fact, many online instructors don’t even call it “teaching” in a traditional sense, but prefer to use words like “delivery” or “facilitate” in place of “teaching.” Others will say things like “your instructional design is your teaching.” All of those statements are problematic to some degree, but the point is that your design and teaching are intricately linked in online education.

But isn’t the whole selling point of QM the fact that it improves your course design? How do you improve the design without determining what materials work well or not so well? How do you improve a course without improving assignments, discussions, and other aspect of “what” you teach? How do you improve a course without changing structural options like alignment and objectives – the things that make up “how” you teach?

The truth is, General Standards 3 (Assessment and Measurement), 4 (Instructional Materials), and 5 (Learning Activities and Learner Interaction) of the QM rubric do affect what you teach and what materials you use. They might not tell you to “choose this specific textbook,” but they do grade any textbook, content, activity, or assessment based on certain criteria (which is often a good thing when bad materials are being used). But those three General Standards  – along with General Standard 2 (Learning Objectives (Competencies)) – also affect how you teach. Which, again, can be a good thing when bad ideas are utilized (although the lack of critical pedagogy and abolitionist education in QM still falls short of what I would consider quality for all). But we do have to recognize that QM does affect the “what” and “how” of online course design, which is the guide for the “what” and “how” of online teaching. That is the whole selling point, and it would be useless as a rubric if it didn’t help improve the course to do this.

So, yes, specific QM review standards require certain specific course structures that do dictate how the course is taught. The QM rubric is biased towards certain structures and design methodologies. If you are wanting to teach a course that works within that structure (and there are many, many courses that do), QM will be a great tool to help you with that structure. However, if you start getting into other structures of ungrading, heutagogy / self-determined learning, aboriginal pedagogy, etc, you start losing points fast.

This is has kind of been my point all along. Much of the push back against that point dives into other important (but not related to the point) issues such as accreditation, burnout, and alignment. Sometimes people got so insulting insulting that I had to leave the conversation and temporarily block and shut it out of my timeline.

QM evangelists are incredibly enthusiastic about their rubric. As an instructional designer, I am taught to question everything – even the things I like. Definitely not a good combination for conversation it seems.

But I want to go back and elaborate on the two points that I tried to stick to all along.

The first point was a response to how some implied that QM is without bias… designed for all courses, and because of this, if some institutions end up using it as a template to force compliance, that is their bias and not QM’s fault. And I get it – when you create something and people misuse it (which no one is denying happens), it can be frustrating to feel like you are being blamed for other’s misuse. But I think if we take a close look at how QM is not unbiased, and how there are politics and bias built into every choice they made, we can see how that has the effect of influencing how it is misused at institutions.

QM is a system based on standardization that was created by specific choices that were chosen through bias, in a method that biases it towards instructor-centered standardized implementation by institutional systems that are known to prefer standardization.

I know that sounds like a criticism, but there are a couple of things to first point out:

  • Bias is not automatically good or bad. Some of the bias in QM I agree with on different levels. Bias is a personal opinion or organizational position, therefore choosing one option over another always brings in bias. There is no such thing as bias-free tech design.
  • The rubric in QM is Ed-Tech. All rubrics are Ed-Tech. That makes QM an organization that sells Ed-Tech, or an Ed-Tech organization.  This is not saying that Ed-Tech is their “focus” or anything like that.

So the thing with QM is that most people understand how it was designed to be flexible. But even those QM design choices had bias in them. All design choices have bias, politics, and context. And when the choices of an entity such as QM are packaged up and sold to an institution, they are not being sold to a blank slate with no context. The interaction of the two aspects causes very predictable results, regardless of what the intent was.

For example, the QM rubric adds up to 100 points. That was a bias point right there. Why 100? Well, we are all used to it, so it makes it easy to understand. But it also connects to a standardized system that we were mostly all a part of growing up, one that didn’t have a lot of flexibility. If we wanted to score higher, we had to conform. When people see a rubric that adds up to 100, that is what many think of first. Choosing a point total that connects with pre-existing systems that also utilize that highest score is a choice that brings in all of the bias, politics, and assumptions that are typically associated with that number elsewhere.

Also, the ideal minimum score is 85. Again, that is biased choice. Why not 82, or 91, or 77? Because 85 is the beginning of “just above average” of the usual “just above average score” (a “B”) that many are used to. Again, this connects to a standardized system we are used to, and reminds people of the scores they got in grade school and college.

In fact, even using points in general, instead of check marks or something else, was another choice that QM made that is also a biased choice. People see points and they think of how those need to add up to the highest number. This mindset affects people even when they get a good number: think of how many students get an 88 and try to find ways to bump it up to a 90. This is another systemic issue that many people equate to “follow the rules, get the most points.”

Then, when you look at how long some of the explanations of the QM standards are, again that was a choice and it had bias. But when combined with an institutional system that keeps its faculty and staff very busy, it creates a desire to move through the long, complicated system as fast as possible to just get it done. This creates people that game the system, and one of the best ways to hack a complex process that repeats itself each course is to create a template and fill it out.

While templates can be a helpful starting place for many (not all), institutional tendency is to do what institutions do: turn templates into standards for all.

This is all predictable human behavior that QM really should consider when creating its rubric. I see it in my students all the time – even though I tell them that there is flexibility to be creative and do things their way, most of them still come back to mimicking the example and giving me a standardized project.

So you can see it all up and down the QM rubric – each point on the rubric is a biased choice. Which is not to say that they are all bad, it’s just that they are not neutral (or even free from political choices). Just some specific examples:

  • General Standard 3.1 is based on “measure the achievement” – which is great in many classes, but there are many forms of ungrading and heutagogy and other concepts that don’t measure achievement. Some forms of grading don’t measure achievement, either.
  • General Standard 3.2 refers to a grading policy that doesn’t work in all design methodologies. In theory, you could make your grading policy about ungrading, but in reality I have heard that this approach rarely passes this standard.
  • General Standard 3.3 is based on “criteria,” which is a popular paradigm for grading, but not compatible with all of them.
  • General Standard 3.4 is hard to grade at all in self-determined learning when the students themselves have to come up with their own assessments (and yes, I did have a class like that when I was a sophomore in college – at a community college, actually). Well, I say hard – you really can’t depending on how far you dive into self-determined learning.
  • General Standard 3.5 seems like it would fit a self-determined heutagogical framework nicely… in theory. In reality, its hard to get any points here because of the reasons covered in 3.4

Again, the point being that it is harder for some approaches like heutagogy, ungrading, and connectivism to pass. If I had time and space, I would probably need to go into what all of those concepts really mean. But please keep in mind that these methods are not “design as you go” or “failing to plan.” These are all well-researched concepts that don’t always have content, assessment, activities, and objectives in a traditional sense.

Many of the problems still come back to the combination of a graded rubric being utilized by a large institutional system. A heutagogical course might pass the QM rubric with, say, an 87 – but the institution is going to look at it as a worse course than a traditional instructivist course that scores a 98. And we all know that institutional interest in the form of support, budgets, praise, awards, etc will go towards the courses that they deem as “better” – this is all a predictable outcome from choosing to use points on a 100 point scale.

There are many other aspects to consider, and this post is getting too long. A couple more rabbit trails to consider:

So, yes, I do realize that QM has helped in many places where resources for training and instructional designers are low. But QM is a rubric, and rubrics always fall apart the more you try to make them become a one-size-fits-all solution. Instead of trying to expand the one Higher Ed QM rubric to fit all kinds of design methodologies, I think it might be better to look at what kind of classes it doesn’t work with and move to create a part of the QM process that identifies those courses and moves them on to another system.

Emergency Educational Measures in a Time of Pandemics, Anti-Racism Protests, and Political Chaos: Is This All Going… Anywhere?

A few months ago, a former co-worker of mine Erika Beljaars-Harris asked me to come talk to the University she now works at – RMIT Australia. They were interested in hearing about the current problems U.S. education is facing and where it is going. While I am not a futurist by training, well, neither are half the people that put the term in their bio. But I think there are at least three obvious trends that are having  – and will continue to have – immediate impacts on education. I also think it is pretty obvious where they are going if you put aside what you would like to see happening and be honest about how difficult it is going to continue to get.

So I came up with the title for the presentation (that I re-used for this post) and a quick 10-minute intro to frame the discussion. I thought I would write this post to capture the basic thoughts I shared in that intro. I covered the three trends I mentioned in the title and where I think we (in the U.S., but many other countries are facing similar issues as well) are going with those, both good and bad.

Pandemic Responses

When I first wrote up the description, it was a month ago and few were really talking Fall seriously. Now a lot of places have started talking, and even releasing some plans. These plans, for the most part, say nothing concrete other than “back to campus! (we hope but won’t commit to fully)” I made a joke about how solid those plans feel:

Campuses need students, faculty, and staff back on campus to generate revenue. So they are going to wait as long as they can to make any decisions other than going back. But coronavirus numbers are growing. Politicians and leaders are not doing enough to reverse that. A small handful of schools have put out some good plans for the Fall (thank you to those that have), but most have released nothing, or even worse, confusing complicated plans that try to hard to get as many students on campus as possible.

So where is this going? Classes are going to be online at some point in the Fall – or even before the Fall. It will probably depend on how much death and sickness we can accept before making the decision. But make no mistake: we will be back online sooner rather than later. And I realize that is not necessarily a good thing for all – but it will happen regardless. As much as you will be told to get ready for on campus or hybrid options, I would put more effort towards getting your classes and yourself ready for online. Make plans to keep yourself and your students as safe as possible. Don’t wait for guidance from leaders, and if it does come, realize you will probably have to go above and beyond anything they plan for.

Anti-Racism and Education

I am not totally sure I used correct English in the title – but the main idea is that we have witnessed sustained protests against systemic racism in several ways following the murder of George Floyd, and many are noting that they seem to be having some effects. I suddenly got a bunch of emails on June 18th telling me about Juneteenth celebrations or events happening the next day – as if a bunch of companies suddenly realized they better do something. Sure, many people will probably lose interest in social justice and equality once the news cycle moves on, but I think we have finally crossed a line to where you need to decide to be anti-racist or not. The lines have been drawn

So where is this going? Don’t look at “diversity” as a thing that you do for a special one day thing in class, or leave to the People of Color in your life. Don’t look at it as something our Black colleague can fill you in on when ever you remember to ask. Don’t ask your Indigenous student to speak for all Indigenous Peoples during a class discussion. You will need to start doing the work to bring marginalized voices into your content from the beginning of class to the end. It will no longer be acceptable to lean on “traditional experts” (aka white males) to make up the main focus of your content. You will need to do the work of searching and learning for yourself who needs to be added to your lessons. You will also need to do the work yourself to examine your own biases and prejudices and how they affect how you teach and interact with students, colleagues, and co-workers. Critical educational practices / pedagogy will need to be practiced by all, not just those that study it as an academic field.

Update: an important aspect of this work is also listening:

Political Chaos

So even beyond academic leaders, it seems that political leaders at all levels and political parties are also falling apart in their response to COVID. Some have done well, but most have not. From reacting too late, to re-opening too soon, to realizing they opened too soon but took too long to reverse course, it has been a failure of partisan politics up and down the board. I am not getting into both side-ism here – one party clearly wanted to follow Science and the other did not (even though even those that wanted to follow Science could have done much better in most cases). Wearing masks, social distancing, supporting the economy after you ordered businesses to shut down, and so on should NOT have ever been political issues to disagree over. And I know that this problem comes from the top and affects the ability of all levels underneath to make decisions. But more leaders at state and local levels should have decided to do the right things regardless of what was happening at the national level. Many did and that helped for the people living in those areas. But many did not. And instead of getting over the worst of it by now as we should be, we are gearing up for the worst of it to still come.

So where is this going? I think we have to face the hard truth that we are on our own for now. You will have to find what ever group you can that will do the right thing and band together to do what you can. Whether a smaller group in the city, at the department level, a subsection of your department, an ongoing Twitter DM group, a Slack channel, a group Facebook Message, whatever it might be – find some people to connect with and band together to help. Whether it is emotional help online or in-person help in your community (socially distanced, of course), find those that are willing to do the right thing and form networks. Try to reach out to as big a level as you can, but don’t count on leaders to get things done. Some leaders may get some things done and that is great – connect with them. If not, find the people that will and band together with them to make it through.

 

Crash Course in Online Teaching 1: Starting With Theory (Wait, Wait – Give It a Chance! Really!)

With several universities now coming to grips with the fact that they will still be online in the Summer (and most likely the Fall), several are turning to how to quickly train their entire faculty in online teaching in a hurry. There really isn’t one ideal way to do this, but I want to offer up the way I would do it if given the chance to design a Crash Course in Online Learning (insert your Budgie/Metallica song parody here).

I would personally start off with a very basic intro to… stay with me here… learning theory. Wait – don’t click away just yet. Many would balk at the idea of starting with theory and not practical tool/building skills, while learning theorists would cry foul at thinking that a basic intro to learning theory is even possible. But I have found that a few basic concepts taught at a very intro level can help faculty not only understand how they have been taught in the past, but also how they can try new ideas and designs they may have never thought of.

So give it chance if you are tired of academia and theory, or give me some wide wiggle room for scaffolding if you are deep into learning theory. I’m going to combine and give examples that will make theory easy to digest, but also blur the lines of complexity that exist. Please keep that in mind as we go forward.

First of all, I will point out that I have published and taught this before. I will give the basics below, but if you have 30 minutes to an hour to dig in, there are slightly more explanatory resources out there. The first is the paper that I published called “From Instructivism to Connectivism: Theoretical Underpinnings of MOOCs.” Yes, it is about MOOCs, but the ideas can be applied to any course. Plus, it comes with a worksheet that you can use to plan your course. The second is a video archive of a training session I led from last year on the paper. If you prefer video more than blog posts or papers, this might work for you:

So, for those that are wanting the summary version, here we go….

Overall Power Dynamics

Out of all the different ways to approach learning theory, I like focus on power dynamics first when it comes to designing a course. So think about the overall power dynamic you want to see happening in your course. This can change from week to week, but in general most courses stick to one for the most part. The question is: who determines what learners will learn in your course, and who directs how it is learned? There are many ways to look at this, but I like to focus on three different -isms:

  • Instructivism: The course is controlled by an expert instructor, determined and directed by that expert.
  • Constructivism: Constructed self-discovery, sometimes determined by the expert, but usually directed by the learner.
  • Connectivism: Learner-determined and directed by the learner, enhanced by networking (connecting) with others (including other learners, the instructor, online resources, etc).

Yes, these can mix, and you can move between different ones. The main thing to think about is who is determining the overall knowledge and/or skills to be learned, and then who is directing how the learners will learn those knowledge and skills and then how they will prove they learned them.

Again, there are many different ways of looking at these terms, many other -isms you can use, and a lot of ambiguity that I am glossing over here. These three -isms (and how I described them) are just a good place to start for those that are new. Generally you pick one, but also think how elements of others might also be utilized in your course.

Methodology of Course Design

Course design methodology often overlaps with power structures. However, within various power structures, there still is room for different design methodologies. For example, even in connectivism there it is still possible to design a course that focuses on transmission of knowledge from experts, even if those experts are not always the instructor.

In this stage, you are thinking about where knowledge and/or skills training comes from, not just who controls the overall power dynamics. Again, there are many, many different ways to look at this. I want to start with three popular -agogies:

  • Pedagogy: many people use this as a catch-all phrase for all teaching design, but in a traditional sense for several centuries it has meant focusing on knowledge transfer from an expert (Update: please note there are many different ways of looking at the term that have gained traction over the past 80-100 years that  hope to cover in upcoming parts – see viewing pedagogy as a philosophy rather than a theory in fields like critical pedagogy).
  • Andragogy: learners draw upon their experience to connect what they already know with new content / knowledge / skills / etc (some have advocated to use the term “Anthropagogy” in place of Andragogy to be more inclusive, but I use the more common term here).
  • Heutagogy: learners focus on learning how to learn about a particular topic rather than just what to learn. Heutagogy is often seen as a critical response to the limitations of other -agogies.

Typically, you see pedagogy matching with instructivism, andragogy matching with constructivism, and heutagogy matching with connectivism. But it is possible for other combinations, such as constructivist heutagogy or connectivivist pedagogy. There is a chart of page 94 of the article above that explains the nine different combination and gives examples.

The main idea is that you choose which combination of the two you want for your class most often (even if if changes from time to time). I tend to advocate for a connectivist heutagogical approach the most often, as that is what more and more people need in the world today. Rather than memorizing expert facts as determined by a the instructor, we need more learners that know how to grow and learn about a topic by connecting with the people and resources that can teach them what they need to know.

At this point, you will start considering what activities and assignments you will be using in your course. It is also good to have some well-written and aligned goals, objectives, competencies, or other standards. I will cover that as a separate post next, but more than likely, you are transitioning a course that already exists on campus into an online course. So I will continue with the theory first, but keep in mind that even if you already have goals and objectives, it would be a good idea to review them after you work through learning theory.

Types of Interaction

Of course, your class will have all types of interaction. However, I have found that once people jump into creating activities and assignments and content first, they leave out interaction until after the bulk of the course activities have been created. At that point, interaction becomes an afterthought or an add-on addition to what has already been created. Which is not ideal.

Thinking through the types of communication that can happen in a course is a good way to proactively plan out different ways to foster interaction as you create content and activities. Most of us think of different types of communication like student-to-student, teacher-to-student, student-to-content, etc. There are already 12 types that have been identified in the literature, but there could be up to 20 emerging. I gave my run-down of communication types that currently exist and how they might change in the future here:

Updating Types of Interactions in Online Education to Reflect AI and Systemic Influence

You will probably want to pick several types of interaction for different parts or times in your course. Again, if you don’t plan for it from the beginning, it may never make it into your goals, objectives, or lesson plans. However, please keep in mind that you can come back and change/add/remove types throughout the design process.

Also, make sure to match the different types of interaction with the methodology and power structure you selected earlier, at least as you initially see them working out in your course. If you don’t have a good match for your previous choices, then you probably need to consider adding some appropriate interaction types.

Communicative Actions

The final theoretical part to think about will probably be something that you consider now, but come back to once you have an idea of what activities you want in your course. But I will cover it here since it is also in the article above, and it helps to think about it from the beginning as well. Once you know the power structure, methodology, and types of interaction you want, you will need to think through the form that various communication acts will take in your course.

There are many different theories of communication – one that I have found works well for instructors is Learning and Teaching as Communicative Actions (LTCA) theory (based on the work of Jurgen Habermas, but created mainly by one of doctoral committee members Dr. Scott Warren for full disclosure). Current LTCA theory proposes four types of communicative actions:

  • Normative communicative actions: communication of knowledge that
    is based on past experiences (for example, class instructions that
    explain student learning expectations).
  • Strategic communicative actions: communication through textbooks,
    lectures, and other methods via transmission to the learner (probably
    the most utilized educational communicative actions).
  • Constative communicative actions: communication through
    discourses, debates, and arguments intended to allow learners to make
    claims and counterclaims (utilizing social constructivism and /or
    connectivism).
  • Dramaturgical communicative actions: communication for purposes
    of expression (reflecting or creating artifacts individually or as a group
    to demonstrate knowledge or skills gained).

As you can see, you will most likely need to mix these during the class – even for each lesson. The goal on this part is not to pick one or two, but to think through how you communicate what is happening in your course. Think through the activities you will have in your course, and then match those with at least one communicative action and power dynamic/methodological combination.

Pulling It All Together

So that is really it for a quick run through some of the basics of theory that can help you begin to design an online course. Like I have said, there are many other theories than those covered here, and deeper/more complex ways of looking at the ones that were covered. This is meant to be a quick guide to just get started, whether you are designing a new online course from scratch or converting an existing on-campus course to an online version. If you looked at the article, you saw that there is a one-page worksheet at the end to help you work through all of these theories in a fairly quick manner. I have also created a Word Doc version, a Google Doc version, and a PDF form version that you can use to fill out and use as you like.

In parts 2 and 3, I want to go back to some topics I have covered before – but for now, here are links to past posts that cover those basics if you can’t wait:

Goals, Objectives, and Competencies

An Emergency Guide (of sorts) to Getting This Week’s Class Online in About an Hour (or so)

Update: I wasn’t clear enough that this is a basic beginner’s way to look at the terms and ideas that are used in learning theory. This will continue to go deeper as I look at other areas of theory in future posts. Some people are not happy that I avoided using the term “Critical Pedagogy” anywhere in the article. I apologize for that. My main thought was that Critical Pedagogy is often classified as an educational philosophy because it puts theory into action, and therefore it would be better to cover that in practical areas like formative evaluation, writing objectives, creating content, etc. Examining power dynamics, who controls communication, and what forms communication take are one of the foundations of being critical about education, so it is still the foundation of everything in this post.

A Template, a Course, and an OER for an Emergency Switch to Online

So the last few weeks have been… something. Many of us found ourselves in the rush to get entire institutions online, often with incredibly limited resources to do so. I’ve been in the thick of this as well. Recently I shared some thoughts about institutions going online, as well as an emergency guide to taking a week of a class online quickly. I would like to add some more resources to the list that we have been developing since those posts.

First of all, I would like to repeat what many have said (and what I tried to emphasize in that first post): take care of your self, your family, and those around you first. Don’t expect perfection from yourself. Practice self-care as much as possible (I know that easier said that done). Then make sure to take care of your students as well. Communicate with them as much as possible, be flexible, remember that many aspects of their lives have been suddenly upended, and above all, make sure to be a voice of care in these times.

I also know that at some point, you will be expected to put your course online and teach something, whether you think it is a good idea or not. So for those that are at that stage, here are some more resources to help.

First of all, I am working with some other educators to put together a free course called Pivoting to Online Teaching: Research and Practitioner Perspectives (I didn’t really like the word “pivot,” but I was overruled). It is a course that you can take for free from edX, but for those that don’t want to register, we have been placing all of the content on an alternative website that requires no sign-up. Lessons are being created in H5P (remixable) and traditional html format. Archives of past events are also being stored here as well. We are halfway through Week 1, so plenty of time to join us.

As part of that course, I created a module template for an emergency switch to online. This is basically a series of pages that work together as a module that you can copy and modify to quickly create course content. It tends to follow many of the concepts we are promoting in the class (Community of Inquiry, ungrading, etc), but it can also easily be modified to fit other concepts as well. I basically went through my earlier post “An Emergency Guide (of sorts) to Getting This Week’s Class Online in About an Hour (or so)” and followed it in making a Geology module. Then I add some notes in red to talk about options and things you should think about if you are new to this. You can find the Canvas or IMS Common Cartridge version in the Canvas Commons that can be imported in Canvas, or downloaded and imported to systems that support IMS. However, since there are also other systems that don’t use either of these formats, I also made a Google Docs version as well as a Microsoft Word version for download.

And finally, the OER – our book Creating Online Learning Experiences:A Brief Guide to Online Courses, from Small and Private to Massive and Open is still available through Mavs OpenPress in Pressbooks (with Hypothes.is enabled for comments as well). I want to highlight a few of the chapters:

Of course, I like the whole book, so it was hard to pick just a few chapters, but those are the ones that would probably help those getting online quickly. When you get more caught up, I would also suggest the Basic Philosophies chapter as one to help guide you think through many underlying aspects to teaching online.

An Emergency Guide (of sorts) to Getting This Week’s Class Online in About an Hour (or so)

With all of the concern the past few weeks about getting courses online, many people are collecting or creating resources for how to get courses online in case of a last minute emergency switch to online teaching. Some are debating whether to call it “emergency remote teaching” (or some variation of that) instead of actual “online teaching.” I agree with the difference, but don’t think that the academic definitions of either one really brings about much change in the practical work of getting online.

There are many problematic issues to address that many are not talking about. Accessibility, student support, and social support structures that schools provide don’t always switch online so well. Some students are even being kicked off of campus, with little mention being made of finding out where they will go if they don’t have a place to go this early, if they can afford to get where they need to go if so, and if the environment they go to can even allow for them learning from online. On top of all of that, few are talking about the difficulty and chaos that going online will create.

Of course, a lot can be said about if closing schools or going online instead of canceling is a good idea. All are good questions to ask. But a lot of people out there have already been forced to go online whether they agree or not, and many more will be forced to do so in the coming weeks. So we have to talk practical steps for those that are in this predicament.

There are many okay to good guides out there for switching online. I see most of them will tell faculty to examine their syllabus to see what can and can’t go online. This is a good first step, but it often ends up being the last step mentioned in this process. There needs to be some quick and blunt guides for what that means to examine your syllabus. So I am going to dive into that here.

Most Instructional Designers will be able to put a week’s worth of a class online in a very short amount of time IF given free reign to apply effective practices focused on the bare minimum needed and a complete set of content based on those principles. Once IDs start getting away from that – adding in time consuming online options that faculty love but that are not absolutely necessary, or waiting for faulty to get them content – the time to create a class increases quickly. However, if you are willing to focus on the bare bones of good online course design, there are many things you can learn from IDs in a pinch.

As I go through this, I will be addressing accessibility issues along the way. The main thing to keep in mind is that media (mainly video, but also images) is easy to make accessible (due to built in alt tags and captioning features), but also the most time consuming. Auto captioning usually doesn’t cut it. You will still need to manually read through and correct any mistakes by hand. The more you can get away from relying on video or video services, the less time it will take to prepare course work (in general).

The first step in going online is to talk with your students about what that would mean before you are forced to make the switch. Talk with them about what it takes to learn online. Have them go through your syllabus and brainstorm ideas for how to transition your objectives to online. Give them the freedom to suggest changes to objectives, or to even think of different activities to meet objectives. Ask them to talk to you privately if they don’t have Internet access at home, of they need other support services. Make sure they all have a way to check in with you (just so you know they are okay), and a back-up method or two in case the main communication method is not working well (or goes down).

If you have already been forced online, or there will be no class meetings between now and when the switch happens, you will need to think through this yourself. Of course, thinking through this yourself might help you guide the discussion with your students – so do it either way.

Content Creation

The first thing to ask yourself is how new information/content/etc will be communicated to students:

  1. One-Way Communication: Typical lecture method, where you share the new information that students learn. Easiest communication to make accessible, but captions could still be time consuming if you are relying on video (especially longer ones). If your goal is one way communication, you don’t need synchronous video tools, even for questions (students can contact you for that, email them, comment if you use a blog, etc). Also, note that this type of communication can be made to work on mobile devices easier.
  2. Discussion Between Instructor and Students: If you really want the ability to interact, and not just answer questions, you do need to look at tools for interaction. Discussion forums are the most accessible (but a little less human), while video conferencing tools are more problematic in regards to accessibility. For example, people with various hearing issues report that Zoom’s accessibility tools start to fall way short of ideal once four people get on a session. So if you really need this, you might want to consider using small group structures that can use a variety of tools (even a phone). Which would bleed over into another communication modality:
  3. Students Communicating With Each Other: Yes, this would include small group discussion. However, also consider how you can encourage students to use your class as a support network. Don’t just lock-down class tools to only be used for class activities. Help students get that human connection they will start to miss once social distancing sets in. This communication modality can be very effective for mobile devices and accessibility needs if you can be flexible about tools and structures.

Here is the thing that will save you the most time: If it were up to me for a class I was teaching, I would not try to schedule meetings for online lectures or even record videos of my lectures to get those online. It is possible to do that well when going online, but time consuming and problematic in regards to accessibility. Even typing out my lecture for the week can take a while. I would go straight to the Internet:

  • With so much out there, you can probably find articles, blogs, websites, etc that contain the content you need in a 15-30 minute search (or less if you already know of some sources).
  • Then use this link to see how to set up a really fast accessibility check tool in your browser. Use this tool on each source you find.
  • Be careful of video sources – make sure they have accurate captions.
  • Then take another 15-30 minutes to create a content page or blog post that lists each source and adds any core concepts you couldn’t find. This will be the most accessible form of content to make, as well as mobile friendly, as long as the services you use are accessible and mobile friendly.
  • This is a great way to get a wider array of perspectives on course topics than a textbook usually provides. But check to make sure you have diverse perspectives – if your list relies heavily on white Western heterosexual cis males, then you will need to change the parameters of your search to be more inclusive.
  • If needed, you can print articles out on paper for those without Internet access (and just hope there is a way to get it to them still functioning).

Obviously, if you have a textbook that you base your lectures on, they you already have a source of content that you can write up some notes on. This won’t necessarily be the most diverse perspective, but it will be quick. Just be sure to think through issues like students that couldn’t afford the textbook (ahem – OER?), if they can access the eBook texts at home, and so on.

Even more advanced quick method: Turn to some student-centered design methodologies to make the course more engaging:

  • Spend the 15-30 minutes creating an activity for your students to go find the content for the week (online, at a library, etc).
  • Towards the end of the week, create a page or blog post collecting those sources with your commentary.
  • Put some time into creating something more than just “go find content!”
  • Think through how to address accessibility issues, as well as how students that don’t have Internet will find content. Be flexible on that last one – not every student can just go to the local library when they want (and what if libraries close?).

Be ready to have to use the mail service for some students if you have to, and don’t worry too much about deadlines if so.

If you really need to use video, you will need to have well-edited captions, This can take a while. There are really three main options you have:

  • Option 1: Record your video, upload for auto-captioning, and then edit the captions for errors. Not all video upload services allow all of this, so check before time. This will probably sound the most natural, but you will also probably be surprised at how much time you waste going “ummm…” or “let me start over….”
  • Option 2: Type out your content and read from the page, without worrying about the way it makes you look “stiff.” You don’t usually write the way you talk, so it will just have to come out that way in crunch. But you will probably focus without too many side tracks. If you keep the video length to about 2-3 minutes, you can probably write and record it in 30 minutes, depending on how fast you write.
  • Option 3: Record your video, upload to YouTube or something else that has auto captioning, download the auto captions, edit for mistakes (not style), and re-record the video reading this script. The fact that it was based on your natural speech will make it sound more natural. Plus you had a built in practice run. A better end result (not perfect), but also more time consuming.

Activity Creation

This part of the course could possibly be the easiest part to create, or the hardest part, depending on your topic. If you have extensive lab requirements that can’t or aren’t simulated online, you will need to get together as a department and figure out how to translate that into the online space. Unfortunately, there is no quick way around that.

Also, keep in mind that you don’t have to come up with a project or test every week. Sometimes projects take more than one week. Sometimes it is good to just relax and learn. Plus, tests are a problematic concept to begin with, and proctoring solutions will be hard to implement when a lot of people start staying home in the same house. So as much as you can move away from high stakes big tests, the better the online experience will be for you and your learners.

Here is the thing that will save you the most time: One thing that is very cliche but effective is to use your LMS test tool to create a low stakes understanding check:

  • 5-10 questions that covers the core things students should have learned for the week.
  • Give students unlimited attempts so that learners can take them over as needed to get all the questions right.
  • This is not the coolest online design method, but it does give students some relief to know they are on the right track.
  • It should only take 20-30 minutes to create 5-10 questions… if your focus is on making sure students have had some contact with core concepts, and not on trick questions or “gotcha!” fake answers.
  • The goal with this kind of activity is not to catch cheaters, but to help students know what you think is important.

Even more advanced quick method: Really what I would focus on is creating authentic / experiential / etc projects that allows learners to engage more deeply with what they are learning:

  • Think of something that would allow learners to apply the course content to their real lives.
  • Think of something that would also let them apply course knowledge to a real world situation.
  • Let students think of how they will communicate what they have learned. Don’t limit them to just what you think they should produce (like a paper).
  • If you are spending more than 15-20 minutes writing out the instructions to this, you are over-thinking it.
  • If you spend less than 5 minutes writing instructions, you are probably not giving students that are possibly new to this level of agency enough guidelines on how to do the project. Remember the students that are new to all of this.
  • Provide 3-4 example ideas of how students could complete the assignment. Don’t worry too much of several students use your idea. Think of some outside the box ideas, like skits, graphics, etc.
  • Remember flexibility and accessibility. If you have to accept a hand-written project sent through the mail – or maybe even one transcribed by a sibling or a spouse – then no problem. Just be glad they are learning.
  • Its best to grade these in more of a general way. Don’t get bogged down in exact point totals for every mistake. Consider ungrading if possible in your institutional structure.
  • For larger classes, have students self-organize into groups based on interests and/or desired artifact to create. Don’t forget that students might need help self-organizing online or at a distance. Again, remember those with accessibility issues, or internet access issues.

At this point, I would have spent about an hour creating my class for the week. This would have been 30 minutes researching and writing a blog post containing the week’s content (FYI – this blog post is waaaay too long for that, so don’t follow my example :) ), and 30 minutes creating a student-centered authentic open-ended project. And this project would take students 3-4 weeks to complete. If you are new to creating classes this way, your first time or two at doing this might take longer (especially if you are new to the tools you will use). Also, if you need other things like video or labs, that will take more time that this. But there is something else to plan for as well.

Course Communications

I have been touching on communication issues through out this post, so I will try not to repeat what I have already stated here. You will need to communicate other things like class norms and online etiquette outside of content and activities. While it may seem like I am against synchronous communication methods here, that is not the case. You can use both. You will just need to consider how to use synchronous tools in ways that addresses accessibility and internet access issues, as well as make sure the tools works well on mobile devices (which is hard to do for all, since a lot of that is personal preference).

Like I mentioned in a previous post, scheduling synchronous sessions can be tricky at best in shutdown / quarantine situations. Yes, you can do the “we will record it and you can watch later if you miss” thing, but that is also problematic. Some of the reasons that cause people to miss – like overwhelmed internet service – will prevent the same people from watching the recorded video. They will also feel a bit left out.

But here is what I would do for communication in a sudden switch to online:

  • Talk to students before hand if possible. Create a plan.
  • Use synchronous tools for open communal office hours.
  • Set-up alternate options for communication – phone, email, even mail if needed.
  • Send out an email, text, mass phone call, or something weekly. This is a good way to humanize your online learning – stick with those principles as much as possible.
  • Don’t totally avoid video if you don’t want to. Sometimes, a quick 1-2 minute “welcome to the remote version of class” can help. Just get the captions right first!
  • Students might be new to learning online. If your school has a Code of Conduct for online interactions, make sure to follow that and make your students aware of it. If your school doesn’t have one, or has one that you feel is not adequate enough, consider creating your own Code of Conduct for your class.

Every time I edit this post, I add a bunch of new sentences all over the place. There is a lot more that could be said, but I will stop here. I hope that this post gives faculty the idea that they can focus their classes on what works in online learning, not just re-creating the face to face class. Also, I hope this empowers you to save some time in the process, without sacrificing effective practices in the name of an emergency. If you have access to an Instructional Designer to help, please talk to that person even if you have read this post. They can give you even more specific advice related to your unique course needs than this post can.

Let’s Talk About “Shifting an Entire University Online as Disaster Preparedness”…

With the rapid spread of Covid-19 (aka “the Coronavirus” in shorthand for now), there has been an explosion of discussions about preparing for quarantines and societal closures of various kinds. Among these discussions are moving conferences, courses, and even entire institutions online. I have been tweeting about this for a few days, so I wanted to collect and expand some of my thoughts on this topic.

Since I have a Masters in Instructional Design and a Ph.D. in Learning Technologies, we spent many assignments in courses for both degrees discussing various benefits and pitfalls of online learning, and yes, switching to online in the case of an emergency was frequently covered. It’s a complicated and problematic idea, so this will be a bit dark and complex / rambling post to make.

First of all, let me start off by stating that no matter how well you plan, switching to online will be more chaotic and hard than you can imagine, and it will cause greater damage to disadvantaged students than you will probably notice. Your first and foremost duty is to consider your disadvantaged learners first, and to work on navigating chaos rather than trying to stop it. Because you won’t avoid chaos. Remember – it is called DISASTER preparedness for a reason.

Many disaster preparedness plans I have seen, as well as many conference and institutional reactions to Covid-19, seem to only focus on able-bodied younger people that are not older, immuno-compromised, living with those that are older or immuno-compromised, already affected by food insecurity, homeless or on the edge of homelessness, affected by digital redlining, dealing with disability, held back by systemic discrimination and intersectionality, and so on. The reactions are only taking into account young able-bodied people living with other young able-bodied people, with maybe a link to an external resource that mentions everyone else (occasionally in passing). This is not going to cut it. Please keep your entire population in mind, not just those that will probably be okay no matter what you do.

Next, you need to realize that just because a conference or university can pivot to online, there are still institutional / organizational barriers to the overall idea of online. Some just won’t pivot because they are against online in general. If an institution does not allow remote work options, they probably do so for reasons that other institutions that do so have already dealt with. Its usually an institutional preference to be against remote work at this point, so that will likely carry over to online courses as well. Therefore, don’t assume some big switch will happen in those situations. Besides, how will institutions switch courses over to online if they don’t already have the procedures in place for their employees to go online?

Where I work for my day job, we already allow remote work, and already have a robust array of online courses. We have also been providing LMS shells for every course section, to be utilized even in on-campus courses. The structure is already there for the switch. Yes, I know LMSs are not hugely popular right now – I’m with you on that. But the important issue is that there is a space online there already for every class. It could be in WordPress blogs or many other tools for all that it matters.

But I guess we should talk first about how there are different types of disasters at different levels. There aren’t really any hard lines between these categories (I kind of made them up on the spot)… but in general, you see a few different kinds (and probably more than these):

  • Individualized/localized disasters: This is anything from one or more person getting sick in an atypical way (for them, at least), to something that affects only certain people in one or some locations. Tornadoes are an example of that – living in Texas, we frequently have to consider plans on how adjust courses based on the fact that a portion of our class will be affected and the other portion won’t. These plans have to often create more flexibility for students. But they can also happen out of the blue, and cause portions of your courses to be out of sync.
  • Displacement disasters: These can affect entire cities, regions, states, or even countries. There might be some warning, but the situation does come about very suddenly. Things like hurricanes, floods, and other mass displacement events. In general, your students’ first priority will not be education. They will need to get out, find shelter, food, water, etc. Usually, this will call for cancellation, postponement, etc. I teach online courses for UT Rio Grande Valley. When hurricanes were heading for the valley, we had to postpone even the online courses. People are usually fleeing something, so don’t plan to switch to online right away – or maybe even ever. Do look for ways to find out where your students are and how they are doing. Don’t just lock down the campus and say “good luck!”
  • Quarantine/lockdown/closed borders disasters: To be honest, this is the one that most of us did not think much about in the U.S. until Covid-19. On a global scale, it is probably more common than we realize. Neighborhoods, cities, states, etc could be quarantined, closed down, blocked off, etc due to disease, civil unrest, climate change displacement, even economic issues. Some of these might happen suddenly, while others might happen on a slower basis with time to prepare. I think some institutions think you can just switch your face-to-face courses to video conference tools and be done with it, but it is really much more complicated than this.

Something to remember: not all disasters bring about changes for all learners. Some are already living in disaster conditions. We tend to make disaster plans with the stereotypical “traditional” student in mind – young, flexible, and financially stable people that are so focused on education that they will skip meals or sleep to study more. These imaginary students are also probably perfectly flexible in our minds, so our first plan is that education has to be switched online right away to help them. The truth is, many of our students are already not sure where they will get their next meal or place to sleep. While you are thinking about how to adjust your class for disaster preparedness, why not consider how those changes could go ahead and happen at your institution – as a way to help those that are already in a disaster situation personally?

So… how to make the switch to online – if you are at one of places that do that? First of all, I’m focusing mostly on higher ed here, which might also work for High School classes as well as maybe some Junior High contexts. Not sure if I would recommend K-6 going online – but if someone has found a way to do that without leaving behind disadvantaged students, I am all ears.

First I want to touch a little bit on what will likely happen IF some try to make the switch. I realize that more schools have some kind of “sudden switch to online” plan than many of us think, so it is possible that some schools might go ahead and make that switch. These kinds of plans were a thing for a while, so I know they are out there. Some of those plans are inadequate (probably mainly from becoming outdated), but also probably based on some popular faux-futurist scaremongering and not true trend analysis. But that depends on a lot of things, so that may not be the case your plan. If the plan focuses on one big, easy solution  – its not going to work that well. Look for one that is realistic with the idea that “its gonna be rough, here are several possible options and ideas.”

Even those well-thought out plans can not account for everything, so a sudden or relatively slow-moving switch to online will be mass chaos whether a school has no plan, has an adequate plan, or has a detailed plan. Its just that the detailed flexible plans will make people realize there will be chaos.

The ability to navigate through the chaos will probably depend on the size of your Distance Education / Instructional Design group in relation to the number of classes. Those that have small DE/ID groups will find that even a great contingency plan falls apart without enough people in their DE/ID department. Those with a large DE/ID group will find the chaos much more manageable (even if they lack a solid contingency plan) thanks to having a group of people that know how the tools work, as well as the theory and research and history of how to use it (and how not to).

Places that have found themselves in need of a mass switch to online have also found that humans can manage chaos to some degree even without a plan, and that the switch can happen… it just won’t be something to brag about. What will likely happen – assuming the typical modest contingency plan & small DE/ID group – is that most faculty will suddenly ditch a lot of what they are doing in their on-campus courses. They will just stick with “the basics” for their switch online (whatever that means to them – more than likely email with attachments and synchronous video conference sessions). Which, of course, raises all kinds of questions about what they really do in their class if so much can be ditched last minute. Many will find out that email will work just the same as the LMS for a lot of what they want to do, which will lead many people to live the reality that the technology should not be the focus of course design, even if they don’t realize it.

(One weird thing down the road that I would predict is that instructors with the overall attitude of “I just do whatever” will probably come out looking like stars, and will probably get several keynotes/TED talks/etc out of it. The lack of structure and planning in their on-campus courses will, unfortunately, work out quite well… for once. Meanwhile, the Instructional Designers and Tech Support People that saved their butts by fixing their mistakes behind the scenes will sit in the crowd, ignored…. but showing up still because that is what we always do…. yes, there is a reason I sound like I am speaking from experience… :) )

But let’s also discuss some ways to at least try and steer things towards some better outcomes, even amidst the chaos. Keep in mind I am talking about possible options that may not work out perfectly every time. Disaster response happens in situations when a lot of things like proper design of online courses (which takes a long time to do properly) has to take place in a very short time frame. Nothing is wrong with going with what just works in the moment. These are just some ideas of what to think about and possibly try to incorporate in your plans.

So, if you really are going to go down the path of planning for a mass switch to online, the biggest issue you need to deal with is convincing professors to switch from synchronous methods to asynchronous as much as possible (I always assume everyone knows what is meant by that, but that is not always the case – here is a good summary if you are not totally sure what I mean here). I can’t stress enough how disaster could cause synchronous to break down.

When a disaster strikes, especially like a quarantine, people lose a lot of the control over their schedule. Issues such as “when food arrives,” “when family are available to check in,” or even possibly “when they are able to use the Internet” (uh-oh!) will possibly be out of their control. All parts of life suddenly have to become coordinated, scheduled, and controlled by others, who themselves are most likely doing the best they can to get supplies out with limited staff and mobility. Students will need the ability to work learning around society that is not fully functioning – not the other way around.

The problem with quarantines is that you have a massive strain suddenly on resources in a matter of hours that last longer than the usual cyclical strains. Whether the quarantine is short or long term, there will be possibly be rationing and rolling stoppages of all kinds of services to offset loads on systems (don’t forget how this could also affect cell phone service when local towers become overloaded). So what are you going to do when a quarter of your students don’t have Internet service when you schedule synchronous sessions, and then when those students have Internet, another quarter is hit by rolling blackouts, and so on? Or what about other issues, like those that suddenly have to change work hours to keep income flowing to themselves or their employer due to societal upheaval?

I know that oftentimes, the solution to this is “schedule the teaching session through Zoom or some other tool, but record it for those that miss.” From experience, I can tell you that more and more will start missing because they can, even if they are available. Then they start to not get the same learning experience as others since they can’t ask questions live, or feel as connected to other learners. I’m not saying “don’t do synchronous meetings ever,” but just consider those are not the best way to do online learning by default. There are additional consequences for those that miss that can harm them in the long run, so why not consider ways to make it equitable for all?

Even in the first disaster scenario I described – the individualized disaster – some people say that they will just get a camera and live stream their class. That might work great, but what if that student is in the hospital, and not in control of their schedule enough to be online when your class meeting happens? Are you expecting the hospital (or family caregivers if they are at home) to re-arrange their schedules around your class? Probably not a priority when someone is getting medical help. Just a thought.

Additionally, don’t forget the technical problems that can arise during mass migration to synchronous sessions (which many will be familiar with because they also happen in non-disaster situations as well).  For example:

or

…just for a few examples. There are hundreds to consider. For asynchronous as well – there are pros and cons to both.

While many know that there are major differences between asynchronous and synchronous course design considerations that have to be accounted for in the switch between those two modalities, sometimes we also forget that even switching from on-campus classrooms to synchronous online sessions is also not that simple.

If you have used Zoom or some other video conferencing tool in an online setting, you know that there are big differences between on-campus and online meetings. Just remember that not all of your students and faculty have used those tools in online learning contexts, so they will need guidance on how to do that. Where I work for my day job, they are already on top of that aspect.

There are also many other issues to consider, and this post is too long as it is. Hopefully you can start thinking through all of the unique complexities that a switch to online would run into. For example, are you prepared to let students that are locked in dorms rooms and bored work ahead in courses/programs to fill their time? Are you going to let people use campus tools to organize supply distribution, news updates, just chat if bored, etc if needed? Are you considering the mental effects of long-term quarantine, and how to address that while still in quarantine? There are so many to think about.

And to those that say “how likely is that to happen?” Well, not very to be honest. But that is kind of the point of emergency disaster preparedness. You never know until its too late, so think about it now.

A final note – I am not trying to draw hard lines between asynchronous and synchronous. Asynchronous does not mean you can’t ever have the option of synchronous class sessions. Its not always an either/or. Like I have said, there are also things you have to do because that is all you can in a pinch. These are just some ideas to consider. Keep in mind that there are many great ideas that mix both, like putting students in groups and having those groups meet synchronously. These options can work out very well (especially since smaller groups are easier to schedule common times). Or another idea – you can (and should) meet with individual students synchronously as well.

Don’t draw hard lines. It will be chaos. Plan to be flexible. Good luck, and Godspeed…

(Top photo by Kelly Sikkema on Unsplash)

Instructure Wars, Private Equity Concerns, and The Anatomy of Monetization of Data

The Annals of the Dark and Dreadful Instructure Wars of 2019

as told by

Matt, the Great FUD Warrior, Breaker of Keyboards, Smacker of Thine Own Head, Asker of Questions That Should Never Be Asked

If you were lucky, you were spared the heartache that came out of nowhere over the announcement that Instructure will possibly be sold to Thoma Bravo, a private equity firm.

Well, it should be stated that the concern from those that always have concerns over these sales announcements was expected. The quick “shush shush – nothing to worry about here” that came in response from people that usually understand the concerns over past Private Equity sales in Ed-Tech (Blackboard being the typical example) was the surprising part.

For the record, the first skirmish actually started when Jon Becker asked what possible outcomes there could be of the sale, Audrey Watters responded with her thoughts on that, and someone made a sexist attack on Audrey’s knowledge of private equity. Also for the record, they initially did not disagree with her points that prices would go up or that Instructure would be broken up and sold off. They stated this was impossible because there is nothing about Instructure that could be broken into parts. Many of us pointed out the sexists problems with the way he expressed his opinion (not his underlying opinion about PE), but he dug his heels in. We also pointed out that there were actually many things within Instructure that could be broken apart, but that was apparently grounds for fighting (even though it is true there are many parts of Instructure that could be broken off and sold, and just because Thoma Bravo has a history of buy-and-build strategy, there is nothing stopping them from still selling off parts they don’t fit their strategy. “Buy-and-build strategy” and “selling off parts” are not mutually exclusive). Within that argument, the idea that the all of the data that Instructure has been bragging about for a few years could either be sold or monetized (more on the important difference there later).

Sometime within the next day, Jesse Stommel made the tweet that really kicked off the main war (I don’t know if he was replying to comments about the value of data in the earlier arguments or it was a coincidence). This is going to be a long post, so I am trying not to embed Tweets here like I usually do. In what was an obvious reference to Instructure bragging that their data was core to their value as a company, Jesse made the comment that we can now know that this value they were bragging about has a price tag of $2 Billion dollars.

Now, can I just say here – I don’t think in any way that Jesse thought that “Instructure data actually cost $2 Billion.” I’m pretty certain he knows that personnel, assets, code, customer payments, etc all are part of that value. Its just that there was a lot of bragging about data being core to the company value, and a huge gap between the market value at the time) and $2 Billion dollars, and that his professional analysis was that data contributed to that in a big way.

Then there was some debate over the value of data in an Ed-Tech company. This was followed by some shooshing and tone policing towards anyone that thought there should be concern over the lack of transparency about data that Instructure has become known for recently, as well as concerns over what could change with new owners. This led to people retreating to their own corners to express their side without having to be interrupted with constant tangential arguments (and there is nothing wrong with this retreating).

Audrey Watters has written her account of the ordeal, which I recommend reading in its entirety first. I am tempted to quote the whole thing here, so really go read it. I’ll wait.

Okay, first I want to clarify something. In my mind, there is a difference between “selling data” and “monetizing data” even though there are obvious overlaps:

  • “Selling Data” is taking a specific set of data (like from a SQL data dump) and selling that to companies that will turn around and sell it to others (which does happen with educational data – more on that later). When someone says “what good is someone knowing that I submitted Quiz 2 back in 2016?”, they are referring to data as a set archive of database rows from a set date. It is kind of looking at data as a crop of apples that were harvested at a specific time. There was concern over this as a possibility, and we will look at that later.
  • “Monetizing Data” is any form of making money directly from creating, manipulating, transporting, etc data. This happens a lot in every day life, and not all instances of it are bad. The core business of most for-profit LMS companies is the monetization of data – nothing in an LMS works without data. Grade Books need data to work. Discussion forums are empty without data. Analytics dashboards show nothing without data. This is kind of looking at data like a the fruits of a field of apple trees that are constantly growing once picked. You could wipe an LMS database of all past data (well, assuming you could find a way to do so without shutting down functionality), but as soon as you turn it on it the code starts generating a massive set of new data. For many, the main concern with the monetization of data is who controls the data and what will they use it for in the LMS? Will I get manipulated by my own data being compared to past learner’s data without either of us knowing about it?

Now, to be fair, most responses were fairly nuanced between the two “sides” of the war. For the record, my “side” in the great Instructure War of 2019 is that “data has the potential to be used in ways that users may not want, which could include monetization, and both Instructure and their potential buyer are not saying enough about what their plans are.” I think that is close to what many others thought as well, but our position was mainly reduced to “all data bad!” While the other side was reduced to “data has no value so stop worrying!”, I do want to examine the idea of whether educational data can have value (to be sold or monetized) outside of either side.

Instructure’s View of Their Own Data

First, I think it is prudent to start with Instructure’s own view on their data. While it would be hard to reference they amount of bragging they have been doing about the value of data at conferences and sales calls, we only have to look at their own words on their own website to see how they view data.

First there is Project Dig. They start off by proclaiming that they have become “passionate about leveraging the growing Canvas data to further improve teaching and learning.” That passion “became our priority, and over the years we’ve provided greater access to more data and designed new, easy-to-understand (and act on) course analytics.” How can a priority of the company not be a huge factor in what they are worth? This is all under the banner of “we’ve been focused on delivering technology that makes teaching and learning easier for everyone.” Obviously, as an LMS, that focus is their main revenue maker as well. And now data is the priority for that focus.

FYI – the target launch date for their tools that will “identify and engage at-risk students, improve online instruction, and measure the impact of teaching with technology.” is…. 2020. Conveniently after the proposed sales date it seems. Again, how could this priority focus of the company that is improving what they offer to customers not be a huge factor in the current sales price?

But they do recognize that there are some problems with digging into data. What word gets mentioned A LOT in the FAQs about potential issues (hint: it involves a word combined with data that starts with “s” and rhymes with “felling”).

Some key highlights from the FAQs:

  • “Will your practices be consistent with your data privacy or security policies?”They say they “are not selling or sharing institutions’ data” – but only because they choose not to. It important to note that the question about selling is there because they feel they can do just that if they want. But they assure us they won’t. Of course, new owners can change that.
  • “Is this really just my data, monetized?”Basically, they say it is not an example of monetizing your data just because… they choose not to, not because they can’t. The implication still remains that the possibility is real and it is there. Then give examples of how they could. Again, new owners are not limited by this choice of the old owners.
  • “What can I say to people at my institution who are asking for an “opt-out” for use of their data?”This is the core problem many have with monetization of data: feel free to do it, just give me the option to opt out at least. They say a lot, but don’t really answer that question (which is very concerning).Important to note that they say “Institutions who have access to data about individuals are responsible to not misuse, sell, or lose the data.” Then they say they “hold themselves to that same standard.” Nothing says this couldn’t change with new owners. But how do you “sell” data that is worthless? They seem to think selling data is as possible as misusing it or losing it. It certainly would be a lot easier to say “no one is out there buying or selling educational value.”
  • (While they do say a lot about openness and transparency, many customers have expressed frustration at some lack in those areas.)

(an important side note in support of Watters point earlier is that, in addition to the main products of the company that each could be broken off and sold or things like assets, employees, etc, these data projects and services represent even more parts that could easily be broken off and sold if a PE firm so chooses to at any point)

Then I give you – Canvas Data. The doc for this service is really a whole page of ideas for how to monetize Canvas data, along with the existing tools to do it. Which is really the goal of the project: “customers can combine their Canvas Data with data from other trusted institutions.” What it doesn’t quite clarify here is that these institution include companies that sometimes charge money to create manipulate, and transport student data inside and outside of Canvas. Many people trust Canvas to vet these companies, but sometimes these arrangements are obscure.

I will give one example from an organization that I think is pretty trustworthy – H5P. H5P does integrate for free with Canvas for free. However, some activities designed in H5P generate grades (which is student data). If you want to transport that data back into the Canvas grade book, you need a paid account with H5P. This is just one example of how a company can monetize Canvas student data, even down to one small data point.

Now, while I see no reason to distrust H5P, I can’t force students to trust an organization they don’t know. What if they didn’t want grade generated on all these websites (because H5P is not the only company to do this)? What if they were not comfortable with a company profiting on moving around their grades? Or what if they were concerned about what the change in LMS ownership meant for all of this?

Anyways, all of this information is up on their website because Instructure believes that their data is very valuable, and that it can be sold. Why would they not point out that data is worthless by itself? Why would they talk about all of this if people weren’t asking these questions, if entities weren’t asking to buy data?

(And Instructure is not the only LMS doing this. Even Blackboard’s Ultra is already ready to do more with data, to monetize it today: “We’re not just handing you data. We’re surfacing data that matters when it matters most— to foster more personalized interactions and drive student success.”  In other words, they are not just doing what other LMS have always done and managed (handled) your data to monetize it, they are adding value by surfacing and doing more. They “drive success,” because success sells. If you have ever been to an LMS sales pitch, you know analytics, personalization, success, all these terms used on the pages I shared are key sales terms to convince organizations to sign the contract.)

The Marketplace for Student Data

One of the contentions of the “LMS data has little to no value” side is that no one is buying or selling student data, as in dumps of past records. It seems that there are existing marketplaces for student data, to the point that someone wrote a journal article on the whole thing: Transparency and the Marketplace for Student Data. “The study uncovered and documents an overall lack of transparency in the student information commercial marketplace and an absence of law to protect student information.” Sounds like a pretty good justification for concern over any student data out there, whether currently under consideration for sale or not. Why it that?

Taking the list of student data brokers Fordham CLIP was able to identify, Fordham CLIP sought to determine what data about students these brokers offer for sale and how they package student data in the commercial marketplace. There are numerous student lists and selects for sale for purposes wholly unrelated to education or military service. Also, in addition to basic student information like name, birth date, and zip code, data brokers advertise questionable lists of students, and debatable selects within student lists, profiling students on the basis of ethnicity, religion, economic factors, and even gawkiness.

That is not all.

Under the Radar Data Brokers

Get ready for this one: there is no evidence that educational data is even staying specifically within a dedicated student data marketplace. This article on under the radar data brokers compiled a list of “121 data brokers operating in the U.S. It’s a rare, rough glimpse into a bustling economy that operates largely in the shadows, and often with few rules.” Most of the entries on the list don’t get into the specific data they collect, so the fact that “education” appears three times on the list for the few that do is concerning:

  • BLACKBAUD INC.
    A “supplier of software and services specifically designed for nonprofit organizations. Its products focus on fundraising, website management, CRM, analytics, financial management, ticketing, and education administration.” (Wikipedia)
  • MCH INC. DBA MCH STRATEGIC DATA
    MCH “provides the highest quality education, healthcare, government, and church data.”
  • RUF STRATEGIC SOLUTIONS
    A marketing firm owned by consumer identity management company Infutor with a focus on travel, tourism, insurance, e-commerce, and education.

These are places already in business, already buying and selling data. If you look at the chart of the attributes (types of data) that Acxiom collects, “education” is one. Its not hard to believe that – if they don’t already have it – they would be very interested to add “I completed quiz 2 on such-and-such date” to that massive collection on each person.

So Why $2 Billion for Instructure?

The only concrete answer we have now is “nobody outside those privy to the details knows.” There are many speculations out there – some of which started the Instructure Wars. One of the main ones I haven’t touched on, that probably summarizes one side of the Instructure Wars, is that the data adds little to nothing to the value of Instructure (despite their own claims to the contrary), but it is “simple math” getting from the current market value to $2 Billion.

I think there is a point to be made in the “simple math” argument (although I would be careful calling it “simple” or claiming “people just don’t understand” if they don’t agree). I would say that even basic Math has to account for the value of data (both at the price it can be sold and the value it can add through monetization). Autumm Caines made the comparison that data is the engine to the LMS car, and you don’t really buy one without the other. In fact, those that are claiming that the data have no value are accusing Instructure of being the shadiest used car sales people in the world: “If you will buy this new fancy car, I will throw in the engine for free!”

However, it seems that different Instructure investors are now disagreeing with the $2 Billion price tag, some thinking it is too low. In fact, they think it “significantly” under values the company. I would assume these investors have access to details about the price of the sale, and if the $2 Billion was simple math, I don’t know if there would be much room to disagree. The cost of the code would be tied to revenue it generates, and therefore would be static and easy to calculate. Various aspects like assets, personnel, and the value of the income from investors are all relatively fixed. Even the future revenue is based on various predictive factors that would be hard to argue.

Seeing that the data is the newest priority of the company, and its value is difficult to calculate, might that be the best candidate for the source of this disagreement? Maybe, but the only for really, really complex data that leads to complex calculations that could easily be off. And LMS data is pretty straight forward… right?

Well, not so much. Kin Lane took a look at what the public APIs of Canvas reveal about the underlying data, and its a dozy. That is another article that I could quote the entire thing, so please take time to read it. I know the page looks long, but that is because he lists 1666 data points (!!!) in just the public APIs alone (while pointing out there are many private ones that probably have many more). He also points out how this structure and the value that it brings easily accounts for the $2 Billion price tag and more, especially when combined with the costs of code and people and customers and so on.

Now, of course, I am willing to bet that there are multiple factors that are causing the investors to fight over price. It could be that they think Canvas’ movement into the corporate training space is about to take off. It is probably a combination of many factors, some I have not even touched on here.

But it is just not far-fetched to think that there is a real possibility that the data is driving the prices, either a commodity to be sold, and/or a service to be monetized.

As I finished this post, Jame Luke published a blog post based on the economics side of the issue. While it does expose that many of us (especially me) are using the wrong terms, the basic idea that the PE will not have education’s best interest at heart and that the data is driving the economics here. He touches on some of the reasons why about data I do here, goes into a lot more depth, and shares several plausible scenarios for the future goals of Thoma Bravo, including one that makes the monetization of data very central to the future sales value of the company. A detailed but necessary read as well. I don’t have time or energy to go back and correct what I got wrong based on this post, so feel free to blast me in the comments if you so desire.

The Ferocity of the Battle

Right now, my direct messages on Twitter are booming with multiple people that are all flabbergasted as to why this is so controversial. We get that there would be disagreement, but the level of ferocity that one side has had in this battle is surprising. Especially since we all thought these were people that agreed that data does bring significant monetary value to a company.

“Data is the new oil!” we were told. But now we are told it was only… black paint all along? Something that will be there in every painting, but doesn’t cost much to be there?

Many people have offered thoughts on why some are some determined to fight this fight. If they get shared publicly I will probably come back and add them here. For my part, I just don’t know. People that want to protect student’s from misuse of data I get, but they have really gone to extra levels of fight over this (beyond what they usually do, that is). The real surprise is the shear irritation from the Learning Analytics community. I know – how did that happen? None of this Instructure kerfuffle says anything bad or good about Learning Analytics, yet they are in the thick of the battle at times.

Still, why is the basic message of “we should be vigilant to make sure that a company that has been a bit opaque with data issues recently gets sold to another company that may or may not be more open, because their data has the possibility to be exploited” so controversial right now? Why must so many people be proven right on the exact price of data? I don’t know.

For me, there could be a news report tomorrow that has Instructure stating “yep, the price was all about the data,” and I would just respond with “okay, thought so.” I get the feeling I am going to be buried in a barrage of snide Tweets if the opposite narrative goes in the news.

Which, let’s be honest, that will be the narrative from Canvas. They have to say there isn’t much value to the data no matter what the truth is. If they let on that it has actual, real value, every single school, teacher, and student will immediately want to sue for their share. Even if there is no lawsuit, the public relations nightmare would cause untold damage as people get mad their data had direct value in the massive sale.

Of course, the reality is that it does not matter what comes out. Canvas already bragged about the value of the data they are monetizing. They already are using it in ways that people don’t want. People have a good reason to be upset about the monetization of their data because it is already happening.

Thus ends the accounting of the never-ending Instructure Wars, as best can be summarized near the end of the dread year 2019.

As the wars drag on and alliances are strained, many began to wonder….

Will this ever end….

Is Learning Analytics Synonymous with Learning Surveillance, or Something Completely Different?

It all started off simply enough. Someone saw a usage of analytics that they didn’t like, and thought they should speak up and make sure that this didn’t cross over into Learning Analytics:

The responses of “Learning Analytics is not surveillance” came pretty quickly after that:

But some disagreed with the idea, feeling they are very, very similar:

(a couple of protected accounts that I can’t really embed here did come out and directly say they see Learning Analytics and Learning Surveillance as the same thing)

I decided to jump in the conversation and ask some questions about the difference between the two, and see if anyone could given definitions of the two that explained their difference, or perhaps prove they are they same.

My main point was that there is a lot of overlap between the two ideas. Both Learning Analytics and Learner Surveillance collect a lot of student data (grades, attendance, click stream, demographics, etc). If you look at the dictionary definition of surveillance (“close watch kept over someone or something (as by a detective)”), the overlap between the two only grows. Both rely on the collection of data to detect, keep watch, and predict future outcomes, all under the banner of being about the learning itself. Both Learning Analytics researchers and Learning Surveillance companies claim they do their work for the greater good of helping us to understand and optimize learning itself and/or the environments we learn in. The reality is that all surveillance (learning or otherwise) is now based on data that has been analyzed. If we don’t define the difference between Learning Analytics and Learner Surveillance, then the surveillance companies will continue to do what they want with Learning Analytics. Just saying “they are not the same” or “they are the same” without providing quantitative definitions of how they are or are not the same is not enough.

It seems that the questions that were raised in replies to my thread showcase how there is not a clear consensus on many aspects of this discussion. Some of the questions raised that need to be acknowledged and hashed out include:

  1. What counts as data, especially throughout the history of education?
  2. What exactly counts as surveillance and what doesn’t?
  3. Is surveillance an inherently evil, oppressive thing; a neutral force that completely depends on the answer to other questions here; or a benign overall positive force in society? Who defines this?
  4. Does the purpose of data collection (which is driven by who has access to it and who owns it) determine it’s category (analytics or surveillance)?
  5. Does the intent of those collecting data determine it’s category?
  6. Does consent change the nature of what is happening?
  7. Is Learning Analytics the same, similar in some ways but not others, or totally different than Learning Surveillance?
  8. What do we mean by the word “learning” in Learning Analytics?
  9. Are the benefits of Learning Analytics clear? Who gets to determine what is a “benefit” or not, or what counts as “clear”?

I am sure there are many other questions (feel free to add in the comments). But lets dig into each of these in turn.

The Historical Usage of Data in Education

There have been many books and papers written on the topic of what data is, but I got the sense that most people recognize that data has been used in education for a long time. Many took issue with equating Learning Analytics with collecting one data point:

This is a good point. Examining one data factor falls well short of any Learning Analytics goal I have ever read. Seeing that certain data points such as grades, feedback, attendance, etc have always been used in education, at what point or level does the historically typical analysis of information about learners become big data or Learning Analytics? If someone is just looking at one point of data, or they are looking at a factor related to the educational experience but not at learning itself, do we count it as “Learning Analytics”? If not, at what point does statistical information cross the line into becoming data that can be analyzed? How many different streams of data does one have to analyze before it becomes learning analytics? How close does the data have to be to the actual educational process to be considered Learning Analytics (or something else)? Does Learning Analytics even really ever look at actual learning? (more on that last one later)

What is Surveillance Anyways?

It seems there is a range of opinions on this, from surveillance meaning only specific methods of governmental oppression, to the very broad general definition in various dictionaries. Some would say that if you make your data collection research (collected in aggregate, de-identified, and protected by researchers), then it is not surveillance. Others say that analytics requires surveillance. Others take those ideas in a different direction:

I don’t know if I would ever go that far (and if you know George, this is not his definitive statement on the issue. I think.), or if I even feel the dictionary definition is the most helpful in this case. But you also can’t disagree with Miriam-Webster, right? Still, there are some bigger questions about what exactly is the line between surveillance and other concepts:

Oversight, supervision, corporate interest, institutional control, etc… don’t they all affect where we draw the line between analytics and surveillance (if we even do)? Or even deeper still….

Is All Surveillance Evil?

It seems there is an assumption that all surveillance is evil in some corners. Some even equate it with oppression and governmental control. However, if that is what everyone thinks of the idea, then why do grocery stores and hotels and other businesses blatantly post signs that say “Surveillance in Progress“? My guess is that this shows there are a lot of people that don’t see it as automatically bad, and even more that don’t care that it is happening. Or do they really not care, or just think there is nothing they can do about it? Either way, these signs would be a PR disaster for the companies if there was consensus that all surveillance is evil. Then again, I’m not so sure many would be so accepting of surveillance if we really knew all of the risks.

However, many do see surveillance as evil. Or at least, something that has gone too far and needs to stop:

But taking attendance and tracking bathroom breaks for points are two different things, right? So does that mean that…

Does the Purpose of Data Collection Change Anything?

Many people pointed out that the purpose for why data was collected would change whether we label the actions “Learning Analytics” or “Learning Surveillance.” Of course, the purpose of data collection is also driven by who has access to the data, who owns it, and what they need the data for (control? make money? help students? All of the above?). There is sometimes this assumption that research always falls into the “good” category, but that would ignore the history of why we have IRBs in the first place. Research can still cause harm even with the best of intentions (and not everyone has the best of intentions). This is the foundation of why we do the whole IRB thing, and that is not a perfect system. But the bigger view is that research is all about detective work, watching others closely to see what is going on, etc. Bringing the whole “purpose” angle into the debate will just cause the definition of Learning Analytics to move closer to the dictionary definition of surveillance.

On the other hand, a properly executed research project does keep the data in the hands of the researchers – and not in the hands of a company that wants to monetize the data analysis. Does the presence of a money making purpose cross the line from analytics to surveillance? Maybe in the minds of some, but this too causes confusion in that some analytics researchers are making sell-able products from their research. They may not be monetizing the product itself, but they may sell their services to help people use the tools. And its not wrong to sell your expertise on something you created. But many see a blurry line there. Purpose does have an effect, but not always a clear cut or easy to define one. Plus, some would point out that purpose is not as important as your intentions…

The Road to Surveillance is Paved With Good Intentions

Closely related to purpose is intent – both of which probably influence each other in most cases. While some may look at this as a clear-cut issue of “good” intentions versus “bad” intentions, I don’t personally see that as the reality of how people view themselves. Most companies view themselves as doing a good thing (even if they have to justify some questionable decisions). Most researchers see themselves as doing a good thing with their research. But we have IRBs and government regulation for a reason. We still have to check the intentions of researchers and businesses all the time.

But even beyond that – who gets to determine which intentions are good and which aren’t? Who gets to says what intentions still cause harm and which ones don’t? The people with the intentions, or the people affected by the intentions? What if there are different views among those that are affected? Do analytics researchers or surveillance companies get to choose who they listen to? Or if they listen at all? And are the lines between “harmful good intention” and “positive results of intention” even that clear? Where do we draw the line between harm and okay?

Some would say that the best way to deal with possibly harmful good intentions is to get consent….

Does the Line Between Analytics and Surveillance Change Due to Consent?

Some say one of the lines between Learning Surveillance and Learning Analytics is created by consent. Learning Analytics is research, and ethical research can not happen without consent.

Of course, the surveillance companies would come back and point to User Agreements and Terms of Service. So they are okay with consent, right?

Well, no. Who really reads the Terms of Service, anyways? Besides, they typically don’t clearly spell out what they do with your data anyways, right?

While this is often true, we see the same problem in research. We often don’t spell out the full picture for research participants, and then don’t bother to check to see if they really read the Informed Consent document or not. To be honest, consent in research as well as agreement with Terms of Service is more of a rote activity than a true consent process. We are really fooling ourselves if we think these processes count as consent. They really count more as a legal “covering the buttocks” than anything else.

Of course, many would point out that Learning Surveillance is often decided at the admin level and forced on all students as a condition of participating in the institution. And sadly, this is often the case. Since research is always (supposed to be) voluntary, there is some benefit to Informed Consent over Terms of Service, even if both are imperfect. But after all of this…

So, For Real, What is the Difference Between Analytics and Surveillance?

I think some people see the difference as:

Learning Analytics: informed consent, not monetized, intending to help education/learners, based on multiple data points that have been de-identified and aggregated.

Learning Surveillance: minimal consent sought from end users (forced by admin even), monetized, intending to control learners, typically focused on fewer data points that can identify individuals in different ways.

…or, something like that. But as I have explored above, this is not always the clear-cut case. Learning Analytics is sometimes monetized. Learning Surveillance often sells itself as helping learners more than controlling. De-identified data can be re-identified easier and easier as technology advances. Learning Surveillance can utilize a lot of data points, while some Learning Analytics studies focus in on a very small number. Both Learning Analytics and Learning Surveillance have consent systems that are full of problems. Learning Analytics can be used to control rather than help. And so on.

And we haven’t even touched on the problem of Learning Analytics not really even analyzing actual “learning” itself…

Learning Analytics or Click Stream Analytics?

Much of the criticism of Learning Surveillance focuses on how these tools and companies seek to monitor and control learning environments (usually online), while having very little effect on the actual learning process. A fair point, one that most Surveillance companies try to downplay with research of their own. That’s not really an admission of guilt as much as it is just the way the Ed-Tech game goes: any company that wants to sell  a product to any school is going to have to convince the people with the money that there is a positive affect on learning. Some how.

But does Learning Analytics actually look at learning itself?

So while Learning Analytics does often get much closer to examining actual learning than Learning Surveillance usually does, it is generally still pretty far away. But so is most of educational research, to be honest. It is not possible yet to tap into brains and observe actual learning in the brain. And a growing number of Learning Analytics papers are taking into account the fact that they are looking at artifacts or traces of learning activities, not the learning activities themselves or the actual learning process.

However, the distinction that “Analytics is looking at learning itself” and “Surveillance is looking at factors outside of learning” still comes apart to some degree when you look at what is really happening. Both of them are examining external traces or evidence of internal processes. This leaves us with the idea that there has to be a clear benefit to one or other if there is a true difference between the two….

What is Clear and What is a Benefit Anyways?

Through the years, I have noticed that many say that the benefits of analytics and/or surveillance are clear. The problem is, who gets to say they are clear, or that they are beneficial? All kinds of biases have been found in data and algorithms. If you are a white male, there are fewer risks of bias against you… so you may see the benefits as clear. To those that see a long history of bias being programmed into the systems… not so much. Is it really a “benefit” if it leaves out large parts of society because a bias was hard-coded into the system?

Where some people see benefits of analytics, other see reports tailored for upper level admin that tells them what we already know from research. Having participated in a few Learning Analytics research projects myself, I know that it takes a lot of digging to find results, and then an even longer time to explain to others what is there. And then, if you create some usable tool out of it, how long does it take to train people to use those results in “user-friendly” dashboards? Obviously, in academia we don’t have a problem with complex processes in and of themselves. But we should also be reluctant to call them “clear” if they are time-consuming to discover, understand, communicate, and make useful for others.

Then, on top of all of this, what we have had so far is a bunch of instructors, admins, and researchers arguing over whether analytics is surveillance, and if either one of them are okay or not. Do the students get a say? When are we going to take the time to see if students clearly understand what all this is about (and then clearly explain it to them if they don’t), and then see what they have to say? Some already understand the situation very well, but we need to get to place where most understand it fairly well, and then include their voice in the discussion.

So Back to the Question: How Do You Define These Two?

Like many have stated, analytics and surveillance have existed for a long time, especially in formal educational settings:

If you really think about it, Instructivism has technically been based on surveillance and analysis all along. This has kind of been baked into educational systems from the beginning. We can’t directly measure learning in the brain, so education has traditionally chosen to keep close watch over students while searching for evidence that they learned something (usually through tests, papers, etc). Our online tools have just replicated instructor-centered structures for the most part, bringing along the data analysis and user surveillance that those structures were known for before the digital era. Referring to teachers as “learning detectives” is an obscure trope, but one that I have heard from time to time.

(Of course, there are those that choose other ways of looking at education, utilizing various methods to support learner agency. This is outside the focus of this rambling article. But it is also the main focus of the concepts I research, even when digging into data analytics.)

So if you are digging through large data sets of past student work and activity like a detective, in order to find ways to improve educational environments or the learning process…. am I describing Learning Analytics, or Learning Surveillance?

Yes, I intentionally choose a sentence that could easily describe both on purpose.

To be honest, I think if we pull back too far and compare any type of data analysis in learning with any form of student surveillance in learning, there won’t be much difference between the two terms. And some people that only work occasionally with either one will probably be okay with that.

I think we need to start looking at Learning Analytics (with capital L-A) vs. analytics (little a), and Learning Surveillance (capital L-S) vs. surveillance (little s). This way, you can look at the more formal work of both fields, as well as general practices of the general ideas. For example, you can look at the problems with surveillance in both Learning Analytics as well as in Learning Surveillance.

However, if I was really pressed, I would say that Learning Analytics (with capital L-A) seeks to understand what is happening in the learning process, in a way that utilizes surveillance (little s) of interface processes, regardless of monetary goals of those analyzing the data. Learning Surveillance (capital L-S) seeks to create systems that control aspects of the learning environment in a way that monetizes the surveillance process itself, utilizing analytics (little a) from learning activities as a primary source of information.

You may look at my poor attempt at definitions and feel that I am describing them as the exact same thing. You may look at my definitions and see them as describing two totally different ideas. Maybe the main true difference between the two is in the eye of the beholder.

Dealing With Educators That Are Not Interested in Legal Issues

If you have been in Instructional Design / Learning Technology / etc for any length of time, you have no doubt run into the various laws and rules that govern all areas of the work we do in this space. Hopefully, you have even been well-trained on these legal issues as well (but I know that is not always the case). And unfortunately, you have probably run into educators that dismiss certain legal issues out of hand.

I say “educators” even though most people probably think primarily of professors when they think of people that ignore the law. This is because there really are people from all sectors of education that are known to dismiss any legal questions or concerns that may apply to what they are doing.

For me, it seems that Accessibility and Fair Use end up being the most oddly contentious issues to address in education (again, not just with professors, but people from all levels of education and non-teaching staff as well). Its not that people think Accessibility or Fair Use are bad per se. Its usually that dealing with Accessibility can “wait till later,” and that concerns over what counts as Fair Use (or more often, what doesn’t fall under Fair Use) are all just “semantics.”

With accessibility, what typically happens is that the instructional designer will point out the need to take the time to make sure all course content is accessible from the beginning. The common counter argument that is made is that accessibility is important, but it can wait until an actual accessibility request is made… “if” it is ever made. Usually with an extreme amount of emphasis and eye-winking placed on the word “if,” because many seem to believe there just aren’t that many students with accessibility needs out there.

Of course, when those accessibility requests do come in, usually much quicker that expected, the ensuing rush to meet requirements becomes a panicked realization that you can’t just adjust an entire course to be accessible in a few days or even a couple of weeks (the typical time frame given for compliance).

But at least they try. The one that seems to get the least effort to even respond to is our concern over what counts as “Fair Use.” I still find it weird that people use “just semantics” as a dismissive trump card, like it is a reason to give up making a point. Many of our laws are built on semantics. Semantics are usually an important issue to discuss, not a “go away with your petty concerns” push back. It never comes across as helpful or respectful disagreement, even if meant that way. People use “its just semantics” as a way to shut someone else down and “win” a disagreement.

I get that many people don’t want to discuss Fair Use because it can be confusing and hard to define. But if you have had someone warn you about going over the line, chances are that if you end up in court over it, you stand little chance of winning. If an Instructional Designer thinks you went over the line, well, we often go over the line ourselves as well. So don’t expect a court to be more liberal with the line than IDs are. And don’t expect me to put my neck on the line when you ignored my concerns in the first place.

Of course, these two aren’t the only areas that end up seeing contentious arguments over legal definitions. Artificial Intelligence, Learning Engineering, Virtual Reality, OPMs, and many other current hot topics all have critics that raise legal concerns, and defenders that dismiss those concerns out of hand. But these two are the biggest ones I have dealt with in day-to-day course design work.

If I could count the number of times a faculty member came back to me and told me they wished they had listened to me about legal issues….

But what to do about it ahead of time? Just keep bringing it up. Show them this blog post if you need to. Bookmark stories that you read about the issues – from legal or educational experts weighing in the idea, to actual news accounts of legal action. Show them that story about the school in Florida that got caught showing Disney films illegally on a school field trip because a Disney lawyer happened to be driving by the school bus.

You are going to have to make a case and defend it as if you are already in court in many cases. And this is just if you actually have the organizational power to push back. Many IDs do not. In that case, make sure you have an electronic trail suggesting to people that either specific classes or entire departments or schools should take these laws seriously.