What I Tell Every New Course Reviewer
cThis post was prompted by Carla Swanson, an online course quality consultant. She had completed an OSCQR certification, and was looking for examples of completed online course quality reviews, or resources that might help improve the quality and reliability of her reviews. And was particularly interested in examples that demonstrate effective commenting and “scoring.”
I’d recommend our OSCQR Online Course Reviewer webinar and certification to any new instructional designer, or course reviewer. You can view a recent recording of that session on our youtube channel, or register for one of our free sychronous webinars.
What I usually tell new online course reviewers is that the more online courses you review, the better at it you will get, and the more you will learn about what works and what needs improvement in online course designs. The OSCQR website is also full of suggestions and ideas for addressing and improving every OSCQR standard, if you are looking for evidence-based best practices and suggestions.
“It is also important to note that OSCQR is not a scorecard.
You can’t “fail” OSCQR.”
It is also important to note that OSCQR is not a scorecard. This is an important distinction between OSCQR and other online course quality rubrics. You can’t “fail” OSCQR. OSCQR is a tool for online instructional designers (IDs) and faculty to self-assess, or review what is working in the online course design, and what needs to be improved. It helps with identifying areas for improvement, helps with prioritizing things to fix, but most importantly supports a culture of continuous online course design improvements. The standards should be introduced to new online faculty formatively as they design their first online course, followed by a self-assessment, and provide the opportunity for informal, or more formal, consistent, and systematic reviews of courses by IDs, peers, or multidisciplinary groups of reviewers.
Giving Effective Feedback
Learning how to give feedback takes practice. You can bake a cake — it starts with flour and ends with a fork and a cuppa. Or you can spend a lifetime learning to bake. A good ID knows how to give warm and cool feedback,. A great ID knows how to assess and how to present that feedback in a way that is most persuasive and tailored to the individual instructor. Everyone is different and may need different words or approaches to understand suggestions — especially when those suggestions feel counterintuitive, challenge long-held assumptions, or push against familiar practices.
Sometimes it won’t matter if the suggestion is theoretically-supported with heaps of evidence backing it up. That’s when the ID needs to check their ego at the door and tap into their superpowers: understanding the instructor’s perspective, becoming a chameleon, and adapting their approach to uncover and respond to the underlying fear, concern, discomfort, bias, or belief behind the resistance.
They also need to stay open to the idea that the instructor might be right — and be willing to let them try it their way, even if you’re convinced it won’t work. You have to know which battles to pick. At the end of the day, it’s their course. They’re the ones teaching it, interacting with students, and learning as they go. Just like in baking, every course — and every instructor — has their own flavor. And while you can suggest a recipe, it’s their cake. It’s their practice. And like any good baker, they’ll learn over time — through trial, error, and experience — how to make it their own, and how to make it better.
Here are some examples of some online course reviews, which are just examples… not intended to suggest that you do the reviews in that manner, but just to give you an idea of one person’s review style: Course reviewer feedback and examples.
Understanding Continuous Improvement vs. Perfection
One thing I’ve noticed with new instructional designers — and even seasoned online instructors who are new to quality rubrics — is that they often want to “perfect” their course before doing a self-assessment or review. And I get it. There’s this natural instinct to perceive the review like a judgment, rather than what it really is: a tool for improvement. Some folks feel like they need to have everything just right before letting anyone else take a look, especially if they’re used to being the expert in the room. Others may not fully understand that the rubric isn’t meant to grade them — it’s there to help guide improvements. Sometimes it’s just a matter of pride and ownership; they’ve put a lot into their course and want it to look polished before anyone else sees it. And occasionally, they’re just trying to be helpful — thinking it’ll make the review quicker or easier if everything’s “fixed” ahead of time.
But the truth is, trying to perfect things before the process can actually get in the way. The real power of the OSCQR rubric and the process comes when we use it to help us figure out how to prioritize in terms of improvements, and what to tackle first, not everything all at once. If we can help instructors see the review as a collaborative, supportive process, it takes the pressure off — and that’s when the best improvements and ideas can really happen.
Then there are those who genuinely believe their course doesn’t need any improvement — and maybe it was great when it was first built. But things change. Technology evolves. Content gets updated. Learners’ needs shift. And our understanding of what works — grounded in research and practice — keeps growing. That’s why course design is never really “done.” It’s an iterative process, part of a culture of continuous improvement. There’s always something to revisit, rethink, and refine. OSCQR can help.
I recommend that instructors/reviewers spend time looking deeply into the suggestions and resources found on pages on the OSCQR website to see what resonates. For example,
- What is one way you could improve the ability for learners to understand and navigate your course? Do you have a video/screencast that introduces you AND that gives a tour of the important areas of your course?
- What could you do to improve the trust and sense of community in the course? Do you have a forum for informal learner networking/ conversation?
- What one thing could you do to help learners develop self-regulated learning skills? Do you provide a tip of the week announcement to help learners set goals, organize, and create a structured environment? Do you have a forum set up to facilitate study groups or where students can ask/answer questions for each other?
- What one thing could you do to support the development of their sense of self-efficacy? Do you send a note or notes of encouragement helping learners to believe that they can succeed and that you believe in them? An individual email at midterm, and/or at the end of the course is one very effective strategy. Did you use the workload estimator to check your course workload? And compare with information to help calrify time on task.
- What is one thing you could adjust to be more learner-centered? Can you let go of control of something? Have you asked your students what they think about your due dates, assignments, workload? Have you considered offering options for how learners demonstrate master or their /learning that the learners decide on? Do you offer the opportunity of Student-led discussions? Peer reviews? Assignment/Assessment Rubrics?
- Is there anything you do to improve the accessibility of the materials for your course?
- What could you add to your course to demonstrate compliance with the RSI regulation?
- Are there any critical reflection activities you yourself could undertake to interrogate your own implicit biases? https://implicit.harvard.edu/implicit/takeatest.html
- What one thing could you do in the design or facilitation of your course that would address what you learn from that critical activity to address and improve inclusivity and a sense of class community?
- What is one thing you could do in your course to help learners feel like they belong?
These are just some ideas. All this stuff is in the OSCQR website. There is always something that can be improved, right? ☺ It is important for the reviewer to really deeply understand that, so that you can guide faculty who may think their course is already good and done…
As a reviewer, this understanding is essential. The point of the OSCQR is not to rubber stamp the quality of an already well-designed course, but to consider what could be improved in the course even if it is already well designed – to support and encourage a culture of continuous improvement.
I’d be happy to answer any questions you may have and to share anything we have to assist you in your journey as an online course reviewer. And, I invite you to join our OSCQR user group to continue this conversation, and to share any examples you may have of course reviews you have conducted.