Building a Self-Mapped Learning Pathways Micro-Lesson: H5P vs Twine

One of the issues that I often bemoan in relation to creating Self-Mapped Learning Pathways lessons is how there really isn’t simple technology that will let you quickly build non-linear, interactive, open-ended content. I have been keeping my eye on H5P, and building a few things with Twine or SAP Chatbots, so I decided to take them all out for a spin in trying to build something that allows for learners to build their own learning pathway.

So how did it turn out? In general, there were some interesting affordances of the tools, but they still don’t get me to where I would like to be with the lesson design. And none of them really did much for the open-ended part. But I did create some OERs that you can use if you like (details at the end). First, some of the process.

SAP Chatbots have some pretty robust tools for creating interactive chats. In theory, I think I could have built everything within a bot, but didn’t get around to it this time because it would have taken some deep dives. I’m also not convinced that a chatbot interface is the way to go, but more about that later. I decided to use a chatbot at a specific point in the learning pathways lesson to help learners think through modality options.

With H5P, I used the Branching Scenario and Course Presentation tools mainly. With H5P, you get a more intuitive interface that looks nice (and we are told is completely accessible), but very little options for customizing anything. I couldn’t change the look, program variables, or embed things like the SAP Chatbot anywhere into the lesson. So I came up with a way to get around that. It seems to be a good basic option for those that don’t want to get into the weeds of programming variables, but it still is mainly a way to create a Choose Your Own Adventure book. Which is what some call “personalized” these days, even though its really not.

With Twine, there were many options to customize, add variables, manipulate code, and embed what you want. I am not sure how accessible everything in Twine is, but it does give you a lot more flexibility for customization. Also, the option to set variables means you can let learners choose some options that would reformat what they see based on their selections. I did a little bit of that, but I need to dig into this some more. Since I could embed more things in Twine, I was able to build the entire lesson from beginning to end in Twine (with a chatbot embedded near the beginning, and an H5P assessment embedded at the end of one modality).

So I ended up with two different versions of the same lesson that will allow you to compare the two options. Before I share those, a few thoughts on building the lesson.

It took a long time to think through the options and build the simple choices that I did below. A lot of this could be attributed to the fact that I was building an entire lesson from scratch. I decided to dig some into Goals, Objectives, and Competencies because so many of my students struggle with these concepts. Someone that already has a complete lesson built would probably save a lot of time on that front.

Also, I will say that I ran out of time to re-record the videos. There are some mistakes and poorly chosen words here and there (like me saying “behaviorist” when I mean “behavioral”). Maybe I will fix that in the future.

Ultimately, it a took a lot of time to build the options and think through how to navigate the options, while also trying to find ways to get people who choose to take their own path to the tools they need. This is the open ended part I still struggle with. It really comes down to this: learners will step out on their own into the garden, or not. I can’t do much to pre-program those options into a system. I could be there in person to discuss their pathways if they needed it, but that is hard to pre-design for. You would spend hours creating the ability for each option, and then maybe have one or two people choose it.

I should point point out that this lesson uses a modification of the course metaphor idea that asks learners to choose between the “sidewalk” or the “garden” (or to mix both if they like). The metaphor is based on the botanical garden concept, where sidewalks guide those on pre-determined paths to show the highlights of the garden, while the gardens themselves can be explored as you like by leaving the sidewalk. The sidewalk represents the instructor-centered pathway, while the garden represents the student-centered, heutagogical option.

What I don’t like is the modular way all of these parts feel. I wish there was a way to combine all of the elements so that learners only see one page that re-loads new content based on their input. In other words, instead of a chatbot that tries to mimic human conversation (which some like, but others don’t), why not have a conversational interface that would ask questions and then supply new content, videos, activities, etc based on the learner input?

Plus, chatbots tend to be cloud-based, meaning everything you put in them is stored on someone else’s computer. Why can’t that be a local tool that protects your privacy better?

Anyways, these lesson are some basic ideas of what a self-mapped learning pathways micro-lesson could look like. I still feel there is more that could be done with the garden pathway in using the coding/variables option in Twine. I also utilized some tools like and Wakelet in the garden modality (just because I like them), but I need to ponder more about how those tools can be utilized as a mapping space themselves.

So here is what I have:

Goals, Lesson, and Competencies Self-Mapped Learning Pathways micro-lesson in Twine


Goals, Lesson, and Competencies Self-Mapped Learning Pathways micro-lesson in H5P

The H5P tool does use plain html pages for the first three pages – you will see when the switch happens. Also, the Twine tool still uses some H5P activities for the sidewalk modality assessments at the end of that modality. Since this is a stand-alone lesson, I needed some kind of assessment option and decided to re-use what I had created already.

A few design notes: The Sidewalk modality is designed so that there is always a main option to choose from for those that need the most guidance, but also links to other options for those that want to skip around. My goal is to always encourage non-linear thinking and learner choice in small or large ways whenever possible. In the Twine version of the lesson, if you choose the Sidewalk option, that is what you see. If you go to the Sidewalk + Garden option, then there is code that inserts links back to the Garden section into the Sidewalk. This is some of the customization I would like to explore more in the future.  Also, the Garden and Sidewalk + Garden option have some examples and ideas for learners to choose from (basically, custom links to Twitter, Wikipedia, etc to show specific evolving searches there). This obviously isn’t much, but it is a self-determined option and therefore I didn’t want to offer too much. But maybe its not enough?

But, this is a full micro-lesson, and I am designating it as an OER with a CC Attribution-NonCommercial-ShareAlike 4.0 International license for those that want to use it:

  • The videos are on YouTube if you just want to use those.
  • I have created a zip file with all of the html files that you can download and edit. The Twine file in that zip archive (“goals-objectives-competencies.html”) can be loaded into Twine itself and edited as you need.
  • You can also download and update the two H5P files by going either to the full lesson or the assessment portion and clicking on the “Reuse” link in the bottom left corner.
  • The chatbot itself can even be forked and customized by creating an account with SAP and using the fork function on the main page for the bot.

I may even create a badge for those that complete the lesson – who knows? If you want to send a few people through the lesson, feel free to do so with the links above. If you want to send a lot of people through it, maybe consider hosting it on your server. :)

So What Do You Want From Learning Analytics?

If you haven’t noticed lately, there is a growing area of concern surrounding the field of learning analytics (also sometimes combined with artificial intelligence). Of course, there has always been some backlash against analytics in general, but I definitely noticed at the recent Learning Analytics and Knowledge (LAK) conference that it was more than just a random concern raised here and there that you usually get at any conference. There were several voices loudly pointing out problems both online and in the back channel, as well as during in-person conversations at the conference. Many of those questioning what they saw were people with deep backgrounds in learning theory, psychology, and the history of learning research. But its not just people pointing out how these aspects are missing from so much of the Learning Analytics field – it is also people like Dr. Maha Bali questioning the logic of how the whole idea is supposed to work in blog posts like Tell Me, Learning Analytics…

I have been known to level many of the current concerns at the Learning Analytics (LA) field myself, so I probably should spell out what exactly it is that I want from this field as far as improvement goes. There are many areas to touch on, so I will cover them in no particular order. This is just what comes to mind off the top of my head (probably formed by my own particular bias, of course):

  • Mandatory training for all LA researchers in the history of educational research, learning theory, educational psychology, learning science, and curriculum & instruction. Most of the concerns I heard voiced at any LAK I have attended was that these areas are sorely missing in several papers and posters. Some papers were even noticed as “discovering” basic educational ideas, like students that spend more time in a class perform better. We have known this from research for decades, so… why was this researched in the first place? And why was none of this earlier research cited? But you see this more than you should in papers and posters in the LA field – little to no theoretical backing, very little practical applications, no connection to psychology, and so on. This is a huge concern, because the LAK Conference Proceedings is in the Top 10 Educational Technology journals as ranked by Google. But so many of the articles published there would not even go beyond peer review in many of the other journals in the Top 10 because of their lack of connection to theory, history, and practice. This is not to say these papers are lacking rigor for what they include – it is just that most journals in Ed-Tech require deep connections to past research and existing theory to even be considered. Other fields do not require that, so it is important to note this. Also, as many have pointed out, this is probably because of the Computer Science connection in LA. But we can’t forego a core part of what makes human education, well… human… just because people came from a background where those aspects aren’t as important. They are important to what makes education work, so just like a computer engineer that wants to get into psychology would have to learn the core facets of psychology to publish in that area, we should require LA researchers to study the core educational topics that the rest of us had to study as well. This is, of course, something that could be required to change many areas in Education itself as well – just having an education background doesn’t mean one knows a whole lot about theory and/or educational research. But I have discussed that aspect of the Educational world in many places in the past, so now I am just focusing on the LA field.
  • Mandatory training for all LA researchers in structural inequalities and the role of tech and algorithms in creating and enforcing those inequalities. We have heard the stories about facial recognition software not recognizing black faces. We know that algorithms often contain the biases of their creators. We know that even the prefect algorithms have to ingest imperfect data that will contain the biases of those that generated it. But its time to stop treating equality problems as an after thought, to be fixed only when they get public attention. LA researchers need to be trained in recognizing bias by the people that have been working to fight the biases themselves. Having a white male instructor mention the possibility of bias here and there in LA courses is not enough.
  • Require all LA research projects to include instructional designers, learning theorists, educational psychologists, actual instructors, real students, people trained in dealing with structural inequalities, etc as part of the research team from the very beginning. Getting trained in all of the fields I mentioned above does not make one an expert. I have had several courses on educational psychology as part of my instructional design training, but that does not make me an expert in educational psychology. We need a working knowledge of other fields to inform our work, but we also need to collaborate with experts as well. People with experience in these fields should be a required part of all LA projects. These don’t all have to separate people, though. A person that teaches instructional design would possibly have experience in several areas (practical instruction, learning theory, structural inequality, etc). But you know who’s voice is incredibly rare in the LA research? Students. Their data traces DO NOT count as their voice. Don’t make me come to a conference with a marker and strike that off your poster for you.
  • Be honest about the limitations and bias of LA. I read all kinds of ideas for what data we need in analytics – from the idea that we need more data to capture complex ways learning manifests itself after a course ends, to the idea that analytics can make sense of the word around us. The only way to get more (or better) data is to increase surveillance in some way or form. The only way to make more sense is to get more data, which means… more surveillance. We should be careful not to turn our entire lives into one mass of endless data points. Because even if we did, we wouldn’t be capturing enough to really make sense of the world. For example, we know that click stream data is a very limited way to determine activity in a course. A click in an online course could mean hundreds of different things. We can’t say that this data tells us what learners are doing or watching or learning – only just what they are clicking on. Every data point is just that – a click or contact or location or activity with very little context and very little real meaning by itself. Each data point is limited, and each data point has some type of bias attached to it. Getting more data points will not overcome limitations or bias – it will collect and amplify them. So be realistic and honest with those limitations, and expose the bias that exists.
  • Commit to creating realistic practical applications for instructors and students. So many LA projects are really just ways to create better reports for upper level admin. Either that, or ways to try and decrease drop-outs (or increase persistence across courses as the new terminology goes). The admin needs their reports and charts, so you can keep doing that. But educators need more than drop-out/persistence stuff. Look, we already have a decent to good idea what causes those issues and what we can do to improve them. Those solutions take money, and throwing more data at them is not going to decrease the need for funding once a more data-driven problem (which usually look just like the old problems) is identified. Please: don’t make “data-driven” become a synonymy for “ignore past research and re-invent the wheel” in educators eyes. Look for practical ways to address practical issues (within the limitations of data and under the guiding principle of privacy). Talk to students, teachers, learning theorists, psychologists, etc while you are just starting to dig into the data. See what they say would be a good, practical way to do something with the data. Listen to their concerns. Stop pushing for more data when they say stop pushing.
  • Make protecting privacy your guiding principle. Period. So much could be said here. Explain clearly what you are doing with the data. Opt-in instead of opt-out. Stop looking for ways to squeeze every bit of data out of every thing humans do and say (its getting kind of gross). Remember that while the data is incomplete and biased, it is still a part of someone else’s self-identity. Treat it that way. If the data you want to collect was actual physical parts of a person in real life – would you walk around grabbing it off of them the way you are collecting data digitally now? Treat it that way, then. Or think of it this way: if data was the hair on our heads, are you trying to rip or cut it off of peoples’ heads without permission? Are you getting permission to collect the parts that fall to the floor during a haircut, or are you sneaking in to hair cutting places to try and steal the stuff on the floor when no one is looking? Or even worse – are you digging through the trash behind the hair salon to find your hair clippings? Also – even when you have permission – are you assuming that just because the person who got the hair cut is gone, that this means the identity of each hair clipping is protected… or do you realize that there are machines that can identify DNA from those hair clippings still?
  • Openness. All of what I have covered here will require openness – with the people you collect data from, with the people you report the analytical results to, with the general public about the goals and results, etc. If you can’t easily explain the way the algorithms are working because they are so complex, then don’t just leave it there, Spend the time to make the algorithms make sense, or change the algorithm.

There are probably more that I am missing, or ways that I failed to explain the ones I covered correctly. If you are reading this and can think of additions or corrections, please let me know in the comments. Note: the first bullet point was updated due to misunderstandings about the educational journal publishing system. Also see the comments below for good feedback from Dr. Bali.

The Learning Styles™ Industry Versus Learning Preferences

Learning Styles are a contentious issue in education. Just read the room at any conference when some presenter brings them up as a good idea. Hostility and argument are sure to ensue in the Q&A time (if not sooner). Some people love Learning Styles, while others hate them.

Add to this this that too many people like to frame this debate as being between people who believe learning styles exist, and those that believe that learning styles do not exist. Unfortunately, this is not really true (even though there are many debates that seem to devolve into this argument).

Learning Style skeptics do not contend that “there are no learning styles.” We believe that there is no proof of the pre-dominant Learning Styles™ Industry claim that people learn better mostly or only in their preferred learning style. This is a huge difference.

But first, I want to back up and clarify why I say Learning Styles™ Industry. Some people like to clarify the difference between the Learning Styles™ Industry and the general idea of learning styles by calling the general idea things like “learning preferences” and “styles of learning.” If you haven’t ever been a teacher subjected to days of professional development seminars on Learning Styles™ and then unfairly reviewed based on your ability to implement Learning Styles™, let me explain what it is like for a minute.

Yes, in many schools across the nation, teachers are taught that there are three/four/six Learning Styles™ that all learners fall into, and the research proves they learn “best” in this style (visual, kinesthetic, etc.). Therefore, you must administer certain tests to help learners figure out which Style they are and create four-six different lessons for each class topic (the number of learning styles seems to vary from time to time; usually when sales slow, companies have to add or subtract some so they can go back and re-sell the training to everyone again). Then during evaluations, you would be rated based on how well you accomplished the implementation of your Learning Styles™ plan.

Of course when I was in school, learning styles were presented much, much differently. At most, there was a poster on the wall that told you to try different ways of learning beyond the regular “read and memorize” method. It was about exploring variety and find preferences for helping you to learn, not about re-writing lessons over and over again for each style. Somewhere between when I was teenager and when I went into teaching myself, there was a huge shift in what it looked like to implement learning styles in the classroom as Learning Styles™.

For people like me that always come out as different “styles” on different days – which one lesson Style does your instructor pick for you? Do they need to rate percentages of each and then make sure you get that exact mix throughout the year? Or does that mix need to happen daily? Do we keep re-testing kids for Learning Styles™ to see if they change?

Speaking of change, what about students that test as “auditory” learners and then suffer hearing damage? Will their grades suffer for the rest of their life because they can no longer hear?

How do we test learners with disabilities for Learning Styles™? (heaven forbid that we should recognize the ablest nature of Learning Styles™ to begin with….)

How do we know there are only three/four/five styles? Why can’t there be all kinds of weird and random styles?

The whole idea of Learning Styles™ really kind of falls apart when you really start examining the practical ways to implement them as an instructivist instructor (which is the context for many attempts). As I have learned from experience, you quickly run into situations like learners with sight issues that test as visual learners (it really did happen once). You start to wonder what kind of weird research went into this idea… until you find out that there was little research into the idea that students learn better in their Learning Style™. Opps.

Then there is the irony that so many “Learning Style™ Assessments” are completely text based…

FYI – I came out as a visual learner by the test above. Most of my answers to the questions were “it depends,” but because I am an artist and I do like to draw, I got Visual. But I hate most of the study tips they gave me.

Like many people, I like to do different things for learning different content at different times. If you wanted to say that there are learning preferences that 1) don’t follow rigid categories (Auditory, Tactile, etc), and 2) change for each learner depending on what/when/how they study – I would agree with that concept. The idea that there is one main Learning Style™ that each learner must figure out and then have every lesson tailored for them or by them to that style in order for them to learn best? I don’t see much evidence for that.

In order to become a self-determined learner, we all need to learn how the content is presented to us, and then tailor it to what we need at that moment. Learning Styles™ are just too rigid for that to happen – at least in the way they are most often implemented.

So if, you were to ask “what is the difference between Learning Styles and something like Universal Design for Learning?” For me, even when taking into account the less rigid version of learning styles from my youth, it is:

Learning Styles: People are different, so here are some easy boxes to keep them in and contain the complexity.

Universal Design for Learning: People are different, but its complicated, so let’s design something to release that complexity. Even if we don’t fully get it.

The idea of complex and changing learner preferences is why I continue work on self-mapped learning pathways (also known as “dual-layer” and “customizable pathways”). Getting locked into one “style” and then having that one style handed to you as a “personalized” lesson each day is just another form of instructivism that removes much of the need for self-regulation and all of the need for self-determination.

Ed-Tech Retro-Futurism and Learning Engineering

I don’t know what I am allowed to say about this yet, but recently I was recorded on an awesome podcast by someone that I a big fan of their work. One of the questions he asked was what I meant on my website when I say “Ed-Tech Retro-Futurist.” It is basically a term I made up a few years ago (and then never checked to see if someone else already said it) in response to the work of people like Harriet Watkins and Audrey Watters that try to point out how too many people are ignoring the decades of work and research in the educational world. My thought was that I should just skip Ed-Tech Futurism and go straight to Retro-Futurism, pointing out all of the ideas and research from the past that everyone is ignoring in the rush to look current and cool in education.

(which is actually more of what real futurists do, but that is another long post…)

One of the “new” terms (or older terms getting new attention) that I struggle with is “learning engineering.” On one hand, when people want to carve out an expert niche inside of instructional design for a needed subset of specific skills, I am all for that. This is what many in the field of learning engineering are doing (even though having two words ending in “-ing” just sounds off :) ). But if you go back several decades to the coining of the term, this was the original goal: to label something that was a specific subset of the Ed-Tech world in a way that can help easily identify the work in that area. Instructional Technology, Learning Experience Design and other terms like that also fall under that category.

(And for those that just don’t like the idea of the term “engineering” attached to the word “learning” – I get it. I just don’t think that is a battle we can win.)

However, there seems to be a very prominent strain of learning engineer that are trying to make the case for “learning engineering” replacing “instructional design” / “learning experience design” / etc or becoming the next evolution of those existing fields. This is where I have a problem – why put a label that already had a specific meaning on to something else that also already had a specific meaning, just in the pursuit of creating something new? You end up with charts like this:

Which are great – but there have also been hundreds of blog posts, articles, and other writings over several decades with charts almost exactly like this that have attributed these same keywords and competencies to instructional design and instructional technologist and other terms like that. I have a really dated Master’s Degree portfolio online that covers most of these except for Data Scientist. Data Science was a few years from really catching on in education, but when it did – I went and got a lot of training in it as an instructional designer.

There are also quotes like this that are also frequently used for instructional designers as well:

And also tongue-in-cheek lists exactly like this for IDs:

(except for #4 – no instructional designer would say that even jokingly because we know what the data can and can’t do, and therefore how impossible that would be :) )

One of the signs that your field/area might be rushing too fast to make something happen is when people fail to think critically about what they share before they share it. An example of this would be something like this:

Did the person that created this think about the significance of comparing a fully-skilled Learning Engineer to “white” and a totally unskilled Learning Engineer to “black”? We really need a Clippy for PowerPoint slides that asks “You put the words ‘Black’ and ‘White’ on a slide. Have you checked to make sure you aren’t making any problematic comparisons from a racial standpoint?”

But there are those that are asking harder questions as well, so I don’t want to misrepresent the conversation:

There are also learning engineers that get the instructional design connection as well (see the Ellen Wagner quote on the right):

Although as an instructional designer, I would point out we aren’t just enacting these – we were trained and given degrees in these areas. The systems we work for currently might not formerly recognize this, but we do in our field and degree programs. Of course, instructional designers also have to add classroom management skills, training others how to design, convincing reluctant faculty, mindfulness, educational psychology, critical pedagogy, social justice, felt needs, effects of sociocultural issues such as food insecurity, and many other fields not listed in the blue above to all of those listed as well. Some might say “but those are part of human development theory and theories of human development and systems thinking.” Not really. They overlap, but they are also separate areas that also have to be taken into account.

(Of course, there is also the even larger field of Learning Science that encompasses all of this and more. You could also write a post like this about how instructional designers mistakenly think they are the same as learning scientists as well. Or how Learning Science tried to claim it started in 1990s when it really has a longer history. And so on.)

I guess the main problem I have is that instructional design came along first, and went into all of these areas first, and still few seem to recognize this. To imply that instructional design is a field that may also enact what learning engineers already have could possibly be taken as reversing what actually happened historically. I am still not clear if some learning engineers are claiming to have proceeded ID, to be currently superseding ID, or to have been the first to do what they do in the Ed-Tech world before ID. If any of those three, then there are problems – and thus the need for Ed-Tech Retro-Futurism.

Learning -Agogies Updated

A few years ago, I created a list of learning -agogies as a reference for myself and anyone else interested. I didn’t have time to finish it and left some of the non-epistemological -agogies defined. So I decided to make a more completed and updated list, but housed on a page that I can update as needed in the future. Making a blog post every time someone proposes a new -agogy would just end up being confusing. So if you want to make any additions to this page, let me know:

Learning -agogies

As you can see, I added learnagogy, dronagogy (which I still say should be dronology), and several of the other words I mentioned but didn’t define in the original post.

Why Trust Google’s Algorithms When You Can Teach?

You have heard it said “If you can Google it, why teach it?”, but I want to ask “why trust Google’s algorithms when you can teach?” I Google things all the time, so I am not saying to stop using Google (or your preferred search engine). But is it really safe to let our learners of any age just Google it and let that be it? I want to push back against that idea with some issues to consider.

When we say “Google it,” we need to be clear that we are not really searching a database and getting back unfiltered results from complete data curated by experts (like you would get in, say, a University library), but allowing specific Google algorithms to filter all the web content it can find everywhere for us and present us with content based on their standards. There is often little to anything guaranteeing those results are giving us accurate information, or even trying to, say, correct a typo we don’t notice that gets us the wrong information (like adding the word “not” when you don’t realize it). But how often do people think through the real differences between Google and a library when they refer to Google as the modern day global library?

We have all heard the news stories that found everything from promotion of neo-Nazi ideals to climate change denial within Google search and auto-correct results. Things like that are huge problems within themselves, but the issues I am getting at here are how Google search results are designed to drive clicks by giving people more of what they want to hear, regardless of whether it is factual or not. Even worse, most internet search engines are searching through incomplete data that is already biased and flawed, adding to existing inequalities when it uses that data to produce search results. People with more money and power can add more content from their viewpoint to the data pool, and then pay to multiply and promote their content with search engines while diminishing other viewpoints. Incomplete, biased, flawed… all are terms that really don’t do the problem they describe justice here.

When you are an educator of learners at any level – why leave them to navigate through a massive echo-chamber of biased and incomplete search results for any information about your field? Why not work with them to think through the information they find? And when they do need to memorize things (because not every job will let you Google the basics on the spot), why not look into research on how memorization before application helps things like critical thinking and application? To be honest, as many, many others have pointed out, Google has only increased the need to teach rather than “just Google it.” But can we change the societal narrative on this on before it is too late?

So You Want to Go Online: OPMs vs In-House Development

As the Great OPM Controversy continues to rage, a lot is being said about developing online courses “in-house” (by hiring people to do the work rather than paying a company to do so). This is actually an area that I have a lot of experience in at various levels, so I wanted to address the pros and cons of developing in-house capacity for offering online programs. I have been out of the the direct instructional design business for a few years, so I will be a bit rusty here and there. Please feel free to comment if I miss anything or leave out something important. However, I still want to take a rough stab at a ballpark list of what needs consideration. First, I want to start with three given points:

  1. Everything I say here is assuming high-quality online courses, not just PowerPoints and Lecture Capture plopped online. But on the other hand, this is also assuming there won’t be any extra expenses like learning games or chat-bots or other expensive toys… errr… tools.
  2. In most OPM models, universities and colleges still have to supply the teachers, so that cost won’t be dealt with here, either. But make sure you are accounting for teacher pay (hopefully full time teachers more than adjuncts, and not just adding extra courses to faculty with already over-full loads).
  3. All of these issues I discuss are within the mindset of “scaling” the programs eventually to some degree or another, but I will get to the problems with scale later.

So the first thing to address is infrastructure, and I know there are a wide range of capacities here. Most universities and colleges have IT staff and support staff for things like email and campus computers. If you have that, you can hopefully build off of that. If you don’t…. well, the OPM model might be the better route for you as you are so far behind that you have to catch up with society, not just online learning. But I know most places are not in this boat. Some even already have technology and support in place for online courses – so you can just skip this part and talk directly with those people about their ability to support another program.

You also have to think about the support of technology, usually the LMS and possibly other software. If you have this in place, check to make sure the existing tools have capacity to take on more (they usually have some). If you have an IT department – talk with them about what it would take to add an LMS and any other tools (like data analysis tools) you would like to add. If you are talking one online program, you probably don’t need even one full time position to support what you need initially. That means you can make this a win/win for IT by helping them get that extra position for the ____ they have been wanting for a while if they can also share that position with online learning technology support part-time.

This is, of course, for a self-hosted LMS. All of the LMS providers out there will offer to host for you, and even provide support. It does cost, but shop around and realize there are vendors that will give you good service for a good price. But there are also some that won’t deal with you at all if you are not bringing a large numbers of courses online initially, so be careful there.

Then there is support for students and teachers. Again, this is something you can bundle from most LMS providers, or contract individually from various companies. If you already have student and faculty tech support of some kind on campus, talk with them to see what it would take to support __ number of new students in __ number of new online courses. They will have to increase staff, but since they often train and employ student workers to answer the calls/emails, this is also a win/win for your campus to get more money to more students. Assuming your campus fairly treats and pays its student workers, of course. If not, make sure to fix that ASAP. But keep in mind that this can be done for the cost of hiring a few more workers to handle increased capacity and then paying to train everyone in support to take online learning calls.

Then there will be the cost of the technology itself. Typically, this is the LMS cost plus other tools and plug-ins you might want to add in (data analytics, plagiarism detection, etc). Personally, I would say to avoid most of those bells and whistles at the beginning. Some of them – like plagiarism detection – are surveillance minded and send the wrong message to learners. Hire some quality instructional designers (I’ll get to that in a minute) and you won’t even need to use these tools. Others like data analytics might be of use down the line, but you might also find some of the things they do underwhelming for the price. With the LMS itself, note that there are other options like Domain of One’s Own that can replace the LMS with a wider range of options for different teachers and students (and they work with single sign on as well). There are also free open-source LMS if you want to self host. Then there are less expensive and more expensive LMS providers. Some that will allow you to have a small contract for a small program with the option to scale, others that want a huge commitment up front. Look around and remember: if it sounds like you are being asked to pay too much, you probably are.

So a lot of what I have discussed is going to vary in cost dramatically, depending on your needs and current capacity. However, if you remain focused on just what you need, and maybe sharing part of certain salaries with other departments to get part of those people’s time, and are also smart about scaling (more on that later), you are still looking at a cost that is in the tens of thousands range for what I have touched on so far. If you hit the $100k point, you are either a) over-paying for something, b) way behind the curve on some aspect, or c) deciding to go for some bells and whistles (which is fine if you need them or have people at your institution that want them – they usually cost extra with OPMs as well).

The next cost that almost anyone that wants to go online will need to pay for no matter what you do is course development. Many people think they can just get the instructors to do this – but just remember that the course will only be as good as their ability/experience in delivering online courses. You may find a few instructors that are great at it, but most at your school probably won’t fall into that category. I don’t say that as a bad thing in this context per se – most instructors don’t get trained in online course design, and even if they do, it is often specific to their field and not the general instructional design field. You will need people to make the course, which is where OPMs usually come in – but also in-house instructional designers as well.

With an average of 6-8 months lead time with a productive instructor, a quality instructional designer can complete 2-3 three quality 15 week online courses per semester. I know this for a fact, because as an instructional designer I typically completed 9 or so courses per year. And some IDs would consider that “slow.” More intense courses that are less ready to transition to online could take longer. But you can also break out of the 15 week course mindset when going online as well – just food for thought. If you are starting up a 10 course online program, you would probably want three instructional designers, with varying specialties. Why three IDs if just one could handle all ten courses in two years easily? Because there is a lot more to consider.

Once you start one online program, other programs will most likely follow suit fairly quickly. It almost always happens that way. So go ahead and get a couple more programs in the pipeline to get going once the IDs are ready. But you also need to build up and maintain infrastructure once you get those classes going. How do you fix design problems in the course? When do you revise general non-emergency issues? What about when you change instructors? And who trains all of these instructors on their specific course design? What about random one-off courses that want to go online outside of a program? Who handles course quality and accreditation? And so on. Quality, experienced instructional designers can handle all of these and more, even while designing courses. Especially if you get one that is a learning engineer or that at least specializes in learning engineering, because these infrastructural questions are part of their specialty.

The salary and benefits range of an instructional designer is between 50K-100K a year depending on experience and the cost of living where you are located. These are also positions that can work remotely if you are open to that – but you will want at least one on campus so they can talk to your students for feedback on the courses they are designing. But remote work is something to keep in mind because you also have to consider the cost of finding an office and getting computers and equipment for each new person you want to hire (either as IDs or the other positions described). Also don’t forget about the cost of benefits like health care, which is pretty standard for full-time IDs.

Another aspect to keep in mind is accreditation – that will take time and people, but that will be the case even if you go with an OPM as well. You will need to pull in people from across the campus that have experience with this, of course – but you will also have to find people that can handle this aspect regardless of what model you choose. And it can be a dozy, just FYI.

Another aspect to consider is advertising. This is a factor that will always cost, unless you are focused solely on transitioning an existing on campus program into an online one (and not planning on adding the online option to the on-campus one). But even then, if you want it to scale – you will need to advertise. Universities aren’t always the best at this. If yours is, then skip ahead. If not, you will need to find someone that can advertise your new program. Typically, this is where OPMs tend to shine. But it is also getting harder and harder to find those that will just let you pay for advertising separate from the entire OPM package.

I can’t really say what you need to spend here – but I will say to be realistic. Cap your initial courses at manageable amounts – not just for your instructors, but also for your support staff. I can’t emphasize enough that it is better to start off small and then scale up rather than open the floodgates from the beginning. Every course that I have seen that opens up the first offerings to massive numbers of students from the beginning has also experienced massive learner trauma. Don’t let companies or colleges gloss over those as “bumps in the road.” Those were actual people that were actually hurt by being that bump that got rolled over. Hurt that could have been avoided if you started small and scaled up at a manageable pace.

So while we are here, let’s talk scale. Scale is messy, no matter how you do it. Even going from one on-campus course to two on-campus courses has traditionally led to problems. All colleges have wanted to increase enrollments as much as possible since the beginning of academia, so its not like OPMs were the first to talk or try scale. However, we need to be real with ourselves about scale and the issues it can cause.

First of all, not all programs can scale. Nursing programs scale immensely because the demand for nurses is still massive. Also, nurses work their tails off, so Nursing instructors often personally take care of many problems of scale that some business models cause. I’m still not sure if the OPMs involved in those programs have even realized that is true yet. But not all programs can scale like a Nursing program can. Not all fields have the demand like Nursing does. Not all fields have the people with the mindset like Nurses have (no offense hopefully, but many of you know its true and its okay – I’m not sure if Nurses ever sleep).

All that to say – if you are not in Nursing, don’t expect to scale like Nursing can. Its okay. Just be realistic about. Also, be honest about any problems that are happening. Glossing over problems will only cause more problems in no time. Always have your foot on the brake, ready to stop the scaling before issues spiral out of hand.

Remember: education is a human endeavor, and people don’t react well to being herded like cattle. I feel like I have only touched the surface and left out so much, but I am as tired of typing as you probably are of reading. Hopefully this is giving some food for thought for the people that have been wondering about in-house program development.

So why go in-house development rather than OPM? Well, I have been making the case for the cost-saving benefits plus capacity-building benefits as well. Recently I read about an OPM that wanted to charge $600,000 to build one 10 course program. All that I have outlined here plus stuff I left out would easily half of that for a high-quality program. And I am one of those people that usually advocates for how expensive online courses can be to do right. But even I am thinking “Whoa!’ at $600K.

Look, if you are wanting to build a program in a field like Nursing that can realistically scale, and you want to deal with thousands of students being pushed through a program (along with all the massive problems that will bring), then you are probably one of five schools in the nation that fit that description and OPMs are probably the best bet for you. For the other 3000-4000+ institutions in the nation, here are some other factors to consider:

  • Hiring people usually means some or all of those people will live in your community, thus supporting local economies better.
  • Local people means people on your campus that can interact with your students and get their input and perspective.
  • Having your people do things also typically means more opportunities to hire students as GTAs, GRAs, assistants, etc – giving them real world skills and money for college.
  • When your academics and your GRAs are part of something, they usually research it and publish on it. The impact on the global knowledge arena could be massive, especially if you publish with OER models.
  • Despite what some say, HigherEd is constantly evolving. Not as fast as some would like, but it is happening. When the next shift happens, you will have the people on staff already to pivot to that change. If not, that will be another expensive contract to sign with the next OPM.

The last point I can’t emphasize enough. When the MOOC craze took off, my current campus turned to some of its experienced IDs – myself and my colleague Justin – to put a large number of MOOCs online. Now that analytics and AI are becoming more of a thing in education (again), they are turning to us and other IDs and people with Ed-Tech backgrounds on campus as well. For people that went the OPM route, these would all be more (usually expensive) contracts to buy. For our campus, it means turning to the people they are already paying. I don’t know what else to say to you if that doesn’t speak for itself.

Also, keep in mind that those who are not in academia don’t always understand the unique things that happen there. Recently I saw a group of people on Twitter upset about a college senior that couldn’t graduate because the one course they needed wasn’t offered that semester. The responses to this scenario are those that many in academia are used to hearing: “bet there is a simple software fix for this!” “what a money making scam!” “if only they cared to treat the student like a customer, they wouldn’t make this happen!” The implication is that the problem was on the University’s side for not caring about course scheduling enough to make graduation possible. Most people in academia are rolling their eyes at this – it is literally impossible for schools to get programs accredited if they don’t prove that they have created a pathway for learners to graduate on time. It makes good business sense that not all courses can be offered every semester, just like many business do not sell all products year round (especially restaurants). Plus, most teachers will tell you it is better to have 10 students in a course once a year than 2-3 students every semester – more interaction, more energy, etc. But schools literally have to map out a pathway for these variable offerings to work in order to just get the okay for the courses in the first place. Those of us in academia know this, but it seems that, based on what I saw on Twitter recently, many in the OPM space do not know this. We also know that there is always that handful of students that ignore the course offering schedules posted online, the advice of their advisers, and the warnings of their instructors because they think they can get the world to bend to their desires. I remember in the 90s telling two classmates they wouldn’t graduate on time if they weren’t in a certain class with me. They scoffed, but it turns out they in fact did not graduate on time. So something to keep in mind – outside perspectives and criticism can be helpful, but they can also completely misunderstand where the problems actually lie.

And look, I get it – there will always be institutions that prefer to get a “program in a box” for one fee no matter how large it is. If that is you, then more power to you. There are a few things I would ask if you go the OPM route: first of all, please find a way to be honest and open about the pros and cons of working with your OPM. They may not like it, but a lot of the backlash that OPMs are currently facing comes from people just not buying the “everything is awesome” line so many are pushing. The education world needs to know your successes as well as your failures. Failure is not a bad thing if you grow from it. Second, please keep in mind that while the “in-house” option looks expensive and complicated, going the OPM route will also be expensive and complicated. They can’t choose your options for you, so all the meetings I discuss here will also happen within an OPM model, just with difference people at the table. So don’t get an inflated ego thinking you are saving time or money going that route. Building a company is much different from building a degree program, so don’t buy into the logic that they are saving you start-up funds. They had to pay for a lot of things as a for-profit company that HigherEd institutions never have to pay for.

Finally, though, I will point out how you can also still sign contracts with various vendors for various parts of your process while still developing in-house, like many institutions have for decades. This is not always an all-or-nothing, either/or situation (see the response from Matthew Rascoff here for a good perspective on that, as well as Jonathan D. Becker’s response at the same link as a good case for in-house development). There are many companies in the OPM space that offer quality a la carte type services for a good price, like iDesign and Instructional Connections. Like I have said on Twitter, I would call those OPS (Online Program Support) more than OPM. Its just that this term won’t catch on. I have also heard the term OPE for Online Program Enablers, which probably works better.

The Great OPM Controversy

So if you have been following OPMs for a while, you are probably asking yourself “which particular controversy are you referring to?” Good point. Over the past week, there has been some controversy over an article by Kevin Carey that takes a harsh look at the pricing and income from online courses, especially related to OPMs. I took issue with the way the article throws all OPMs into the same bucket – Carey mentions 2U and iDesign in the same sentence, but doesn’t cover the massive differences between the two companies. Personally, I have concerns over even labeling companies like iDesign as OPMs, because they don’t offer to take over the entire online program creation process. They serve more as a specialty service for contract, a type of company that has existed for a long time in HigherEd and that adds great value when priced right.

(also, full disclosure: I have worked for iDesign in the past as a part-time side gig, and still would if their current employment model allowed for work on nights and weekends).

Carey also falls for the assumption that online courses should be cheaper, something that Matt Reed effectively discusses in his own response (just ignore where he briefly falls into the “MOOC attrition rate” misunderstanding). Despite these two points of disagreement, Carey does raise some legitimate hard questions about OPMs that we as a field should discuss.

Of course, with all of this attention, 2U was bound to respond. Today their CEO Chip Paucek wrote an article for Inside HigherEd. While I am glad that Paucek wants to have a constructive dialogue, there were problems with his response as well. Paucek starts off (after selling his company some) by stating that any real conversation about cost or value in online education has to be “grounded” in four specific principles: quality, access, outcomes, and sustainability (personally, I would add ethics and privacy concerns as well). But those are four good ones, and Paucek states that Carey’s article did not focus on those.

Okay, the quality aspect – as related to costs – he did miss. But access, outcomes, and sustainability are all important aspects of the cost of online access – and by addressing cost Carey is also focusing on those three aspects. I think it would be more accurate to say that Paucek felt that Carey did not focus on those aspects the way he wanted him to. They were still there, just not in a format that Paucek recognized maybe? Hard to say. But I felt that point was too forced in Paucek’s response. You can’t separate any discussion of cost from access, outcomes, and sustainability.

Paucek goes on to point out that face-to-face returning students typically have to quit their job and lose income to get a degree while still paying for living expenses. Which is still the case in some places, but not as much as it used to be. For example, I earned my Ph.D. while still working full time because the traditional on-campus program I was a part of adjusted their courses to be on nights and weekends. But the point by Paucek is:

Most master’s and doctorate-level students are working adults who historically had to quit a job, and often move, in order to attend a top-tier university for graduate school…. the average actual cost and debt burden of attending a 2U-powered program is significantly less once you factor in ongoing income and the room and board savings, which in some cities can be as high as 25 to 40 percent of tuition.

Which is true – for all online programs. If the 2U partner schools had built their own online programs, this statement would still be true. Its a bit disingenuous for an OPM to claim a historical benefit of all online / distance education as their own like this. It would be like a website designer claiming they personally are saving clients money by using WordPress, even though WordPress was free long before they started a web design company. Paucek also does this again by claiming that, on average, their partner programs students “are more diverse from both a race and gender perspective than students in comparable on-campus programs.” Again, that was typically true of many online programs long before OPMs came along.

Paucek also goes on the attack against schools that want to build in-house capability for online programs, because he sees this as being wasteful of institutional funds. This is partially true and partially not true. Paucek’s point is that

“…it’s also critical to discuss whether it’s reasonable, rational and appropriate for that investment and risk capital to be shouldered exclusively by schools or in collaboration with a strategic partner like 2U…. each one of our program partners would need to invest their own scarce capital and hire in-house talent to expertly deliver what we deliver.”

Yes, it is true that it takes a lot of money to build online programs in-house. But it also takes a lot of money to hire an OPM like 2U. However, here is the counterpoint: you can hire people that are already experts in online course design, online program management, accessibility, privacy, cybersecurity, etc. You don’t have to start from scratch even if you go the in-house route. I know this, because I am one of many, many experts out there that has the ability to do so. And we are not as expensive as one would think :)

And while Paucek tried to make it seem like it takes 10 years and nearly a billion dollars to develop a quality online program, the truth is that a lot of that went towards building a company – which is different from building an online program. Yes, it does take a lot of time and money to build a quality online program, but it takes a whole lot more to build a national / international company – and those are mostly costs that HigherEd programs will not have to shoulder. To be cliche, it is comparing apples to oranges to make this point. There is some financial overlap between building an OPM and building an online program at an existing institution, but there is a lot that is extra to build a company from scratch.

There are also many other important benefits to building programs in-house that few are talking about. Usually, these programs are built in-house by hiring local talent, which helps local economies. Then there are all of the schools that hire GRAs, GTAs, student assistants of all kinds to help build and administrate and even teach the courses. This helps to empower students by giving them valuable life and employment skills of all kinds. Then there are all of the research articles, blog posts, think pieces, etc that various instructors, staff, and students produce while participating in the process. When these are published through OER models, the additions to the global knowledge space of online learning are immense. Some OPMs participate in some of these benefits, but many keep the whole process behind closed doors to protect proprietary processes and products.

Of course, Paucek’s overarching points that creating quality online courses is expensive, and that we need to have open conversations about the process, are both important. However, I am of the opinion that OPMs should not be the one’s hosting this conversation (as Paucek suggests), as the points outlined in this article make apparent. We as the education community have been hosting it, and all disagreements aside, we have been doing a pretty good job of doing so.

Updating Types of Interactions in Online Education to Reflect AI and Systemic Influence

One of the foundation concepts in instructional design and other parts of the field of education are the types of interaction that occur in the educational process online. In 1989, Michael G. Moore first categorized three types of interaction in education: student-teacher, student-student, and student-content. Then, in 1994, Hillman, Willis, and Gunawardena expanded on this model, adding student-interface interactions. Four years later, Anderson & Garrison (1998) added three more interaction types to account for advances in technology: teacher-teacher, teacher-content, and content-content. Since social constructivist theory did not quite fit into these seven types of interaction, Dron to propose four more types of interaction in 2007: group-content, group-group, learner-group, and teacher-group. Some would argue that “student-student” and “student-content” still over these newer additions, and to some degree that is true. But it also helps to look at the differences between these different terms as technology has advanced and changed interactions online – so I think the new terms are also helpful. More recently, proponents of connectivism have proposed acknowledging patterns of “interactions with and learning from sets of people or objects [which] form yet another mode of interaction” (Wang, Chen, & Anderson, 2014, p. 125). I would call that networked with sets of people or objects.

The instructional designer within me likes to replace “student” with “learner” and “content” with “design” to more accurately describe the complexity of learners that are not students and learning designs that are not content. However, as we rely more and more on machine learning and algorithms, especially at the systemic level, we are creating new things that learners will increasingly be interacting with for the foreseeable future. I am wondering if it is time to expand this list of interactions to reflect that? Or is it long enough as it is?

So the existing ones I would keep, with “learner” exchanged for “student” and “design” exchanged for “content”:

  • learner-teacher (ex: instructivist lecture, learner teaching the teacher, or learner networking with teacher)
  • learner-learner (ex: learner mentorship, one-on-one study groups, or learner teaching another learner)
  • learner-design (ex: reading a textbook, watching a video, listening to audio, completing a project, or reading a website)
  • learner-interface (ex: web-browsing, connectivist online interactions, gaming, or computerized learning tools)
  • teacher-teacher (ex: collaborative teaching, cross-course alignment, or professional development)
  • teacher-design (ex: teacher-authored textbooks or websites, teacher blogs, or professional study)
  • group-design (ex: constructivist group work, connectivist resource sharing, or group readings)
  • group-group (ex: debate teams, group presentations, or academic group competitions)
  • learner-group (ex: individual work presented to group for debate, learner as the teacher exercises)
  • teacher-group (ex: teacher contribution to group work, group presentation to teacher)
  • networked with sets of people or objects (ex: Wikipedia, crowdsourced learning, or online collaborative note-taking)

The new ones I would consider adding include:

  • algorithm-learner (ex: learner data being sent to algorithms; algorithms sending communication back to learners as emails, chatbot messages, etc)
  • algorithm-teacher (ex: algorithms communicating aggregate or individual learner data on retention, plagiarism, etc)
  • algorithm-design (ex: algorithms that determine new or remedial content; machine learning/artificial intelligence)
  • algorithm-interface (ex: algorithms that reformat interfaces based on input from learners, responses sent to chatbots, etc)
  • algorithm-group (ex: algorithms that determine how learners are grouped in courses, programs, etc)
  • algorithm-system (ex: algorithms that report aggregate or individual learner data to upper level admin)
  • system-learner (ex: system-wide initiatives that attempt to “solve” retention, plagiarism, etc)
  • system-teacher (ex: cross-curricular implementation, standardized teaching approaches)
  • system-design (ex: degree programs, required standardized testing, and other systemic requirements)

Well… that gets too long. But I suspect that a lot of the new additions list would fall under the job category of what many call “learning engineer” – so maybe there is a use for this? You might have noticed that it appears as if I removed “content-content” – but that was renamed “algorithm-design,” as that is mainly what I think of for “content-content.” But I could be wrong. I also left out “algorithm-algorithm,” as algorithms already interface with themselves and other algorithms by design. That is implied in “algorithm-design,” kind of in the same way I didn’t include learners interacting with themselves in self-reflection as that is implied in “learner-learner.” But I could be swayed by arguments for including those as well. I am also not sure how much “system-interface” interaction we have, as most systems interact with interfaces through other actors like learners, teachers, groups, etc. So I left that off. I also couldn’t think of anything for “system-group” that was different from anything else already listed as examples elsewhere. And I am not sure we have much real “system-system” interaction outside of a few random conversations at upper administrative levels that rarely trickle down into education without being vastly filtered through systemic norms first. Does it count as “system-system” interaction in a way that affects learning if the receiving system is going to mix it with their existing standards before approving and disseminating it first? I’m not sure.

So – that is 20 types of interaction, with some more that maybe should have been included or not depending on your viewpoint (and I am still not sure we have advanced enough with “algorithm-interface” yet to give it it’s own category, but I think we will pretty soon). Someone may have done this already and I just couldn’t find it in a search – so I apologize if I missed others’ work. None of this is to say that any of these types of interactions are good or bad for learners – they just are the ones that are happening more and more as we automate more and more and/or take a systems approach to education. In fact, these new levels could be helpful in informing critical dialogue about our growing reliance on automation in education as well.

Artificial Intelligence and Knowing What Learners Know Once They Have “Learned”

One of the side effects – good or bad – of our increasing utilization of Artificial Intelligence in education is that it brings to light all of the problems we have with knowing how a learner has “learned” something. This specific problem has been discussed and debated in Instructional Design courses for decades – some of my favorite class meetings in grad school revolved around digging into these problems. So it is good to see these issues being brought to a larger conversation about education, even if it is in the context of our inevitable extinction at the hands of our future robot overlords.

Dave Cornier wrote a very good post about the questions to ask about AI in learning. I will use that post to direct some responses mostly back to the AI community as well as those utilizing AI in education. Dave ends up questioning a scenario that is basically the popular “Netflix for Education” approach to Educational AI: the AI perceives what the learners choose as their favorite learning resource by likes, view counts, etc, and then proposes new resources to specific learners to help them learn more, in the way Netflix recommends new shows to watch based on the popularity of other shows (which were connected to each other by popularity metrics as well).

This, of course, leads to the problem that Dave points out: “If they value knowledge that is popular, then knowledge slowly drifts towards knowledge that is popular.” Popular, as we all learn at some point, does not always equal good, helpful, correct, etc. However, people in the AI field will point out that they can build a system that relies on the expertise of experts and teachers in the field rather than likes, and I get that. Some have done that. But there is a bigger problem here.

Let’s back up to the part from Dave’s post about how AI accomplishes recommendations by simplifying the learners down to a few choices, much in the same way Netflix simplifies viewing choices down to a small list of genres. This is often true. However, this is true not because programmers wanted it that way – this is the model they inherited from education itself. Sure, it is true that in an ideal learning environment, the teacher talks to all learners and gets to make personal teaching choices for each one because of that. But in reality, most classes design one pathway for all learners to take: read this book, listen to these lectures, take this test, answer this discussion question while responding to two peers, wash, rinse, repeat.

AI developers know this, and to their credit, they are offering personalized learning solutions that at least expand on this. Many examinations of the problems with AI skip over this part and just look at ideal classrooms where learners and instructors have time to dig into individual learner complexities. But in the real world? Everyone follows the one path. So adding 7 or 10 or more options to the one that now to exists (for most)? Its at least a step in right direction, right?

Depends on who you ask. But that is another topic for anther day.

This is kind of where a lot of what is now called “personalized education” is at. I compare this state to all of those personalized gift websites, where you can go buy a gift like a mouse pad and get a custom message or name printed on it. Sure, the mouse pad is “personalized” with my name… but what if I didn’t need a mouse pad in the first place? You might say “well, there were only a certain set of gifts available and that was the best one out of the choices that were there.”

Sure, it might be a better gift than some plain mouse pad from Walmart to the person that needed a mouse pad. But for everyone else – not so much.

Like Dave and many have pointed out – someone is choosing those options and limiting the number of them. But to someone going from the linear choice of local TV stations to Netflix, at first that choice seems awesome. However, soon you start noticing the limitations of only watching something on Netflix. Then it starts getting weird. If I liked Stranger Things, I would probably like Tidying Up with Marie Kondo? Really?

The reality is, while people in the AI field will tell you that AI “perceives the learner and knowledge in a field,” it is more accurate to say that the AI “records choices that the learner makes about knowledge objects and then analyzes those choices to find patterns between the learner and knowledge object choices in ways that are designed to be predictive in some way for future learners.” If you just look at all that as “perceiving,” then you probably will end up with the Netflix model and all the problems that brings. But if you take a more nuanced look at what happens (it’s not “perceiving” as much as “recording choices” for example), and connect it with a better way of looking at the learner process, you will end up with better models and ideas.

So back to how we really don’t have that great of an idea of how learning actually happens in the brain. There are many good theories, and Stephen Downes usually highlights the best in emerging research in how we really understand the actual process of learning in the brain. But since there is still so much we either a) don’t know, or b) don’t know how to quantify and measure externally from the brain – then we can’t actually measure “learning” itself.

As a side note: this is, quite frankly, where most of the conversation on grading goes wrong. Grades are not a way to measure learning. We can’t stick a probe on people’s heads and measure a “learning” level in human brains. So we have to have some kind of external way to figure out if learning happens. As Dr. Scott Warren puts it: its like we are looking at this brick wall with a few random windows that really aren’t in the right spot and are trying to figure out what is happening on the other side of the wall.

Some people are clinging to the outmoded idea that brains are like computers: input knowledge/skills, output learning. Our brains don’t work like that. But unfortunately, that is often the way many look at the educational process. Instructors design some type of input – lectures, books, training, videos, etc – and then we measure the output with grades as way to say if “learning happened” or not.

The reality is, we technically just point learners towards something that they can use in their learning process (lectures, books, videos, games, discussions, etc), they “do” the learning, and then we have to figure out what they learned. Grades are a way to see how learners can apply what they learned to a novel artifact – a test, a paper, a project, a skill demonstration, etc. Grades in no way measure what students have learned, but rather how students can apply what they learned to some situation or context determined by someone else. That way – if they apply it incorrectly by, say, getting the question wrong – we assume they haven’t learned it well enough. Of course, an “F” on a test could mean the test was a poor way to apply the knowledge as much as it could say that the learner didn’t learn. Or that the learner got sidetracked while taking the test. Or, so on….

The learning that happens in between the choosing of the content/context/etc and the application of the knowledge gained on a test or paper or other external measurement is totally up to the learner.

So that is what AI is really analyzing in many designs – it is looking at what choices were made before the learning and what the learner was able to do with their learning on the other side of the learning based on some external application of knowledge/skills/etc. We have to look at AI something that affects and/or measures the bookends to the actual learning.

Rather than the Netflix approach to recommendations, I would say a better model to look to is the Amazon model of “people also bought this.” Amazon looks at each thing they sell as an individual object that people will connect in various ways to other individual objects – some connects that make sense, others that don’t. Sometimes people look at one item and buy other similar items instead, sometimes people buy items that work together, and sometimes people “in the know” buy random things that seem disconnected to newbies. The Amazon system is not perfect, but it does allow for greater individuality in purchasing decisions, and doesn’t assume that “because you bought this phone, you might also want to buy this phone as well because it is a phone, too.”

In other words, the Amazon model can see the common connections as well as the uncommon connections (even across their predefined categories), and let you the consumer decide which connections work for you or not. The Netflix model looks for the popular/common connections within their predefined categories.

I would submit that learners need ways to learn that can look at common learning pathways as well as uncommon pathways – especially across any categories we would define for them.

Of course, Amazon can collect data in ways that would be illegal (for good reason) in education, and the fact that they have millions of transactions each day means that they get detailed data about even obscure products in ways that would be impossible at a smaller scale in education. In no way should this come across as me proposing something inappropriate like “Amazon for Education!” The point I am getting at here is that we need a better way to look at AI in education:

  • Individuals are complex, and all systems need to account for complexity instead of simplifying for the most popular groups based on analytics.
  • AI should not be seen as something that perceives the learner or their knowledge or learning, but one that collects incomplete data on learners choices.
  • The goal of this collection should not just be to perceive learners and content, but to understand complex patterns made by complex people.
  • The categories and patterns selected by the creators of AI applications should not become limitations on the learners within that application.
  • While we have good models for how we learn, the actual act of “learning” should still be treated as a mysterious process (until that changes – if ever).
  • AI, like all education, does not measure learning, but how learning that occurred mysteriously in the learner was applied to an external context or artifact. This will be a flawed process, so the results of any AI application should be viewed within the bias and flaws created by the process.
  • The learners perception of what they learned and how well they were able to apply it to external context/artifact is mostly ignored or discarded as irrelevant self-reported data, and that should stop.