Exam Reviews

A contentious topic between instructors, students, and PACE administration is exam reviews.  How much is too much?  How explicit should they be?   It can be a difficult topic to deal with as students put pressure on instructors for help as they focus on grades and passing a course.

As a general guideline, PACE does not expect instructors to provide explicit questions and answers to students.  That’s a pretty easy starting point. A good method to use in answering these questions is to give students a breakdown by topic or chapter by building your exam with a blueprint.

Using a blueprint is a method to check the distribution of your content, and type of questions that you are asking.  I build mine by textbook chapter, recall questions (like remembering a definition; what is …? Or why did …?), applying the material (a thinking question;  how would you compare…?  How would you classify…?), or creating (going to a higher level of problem solving with the question; what facts support …?  What is the relationship between…? How would you improve…?):

Chapter M/C Short Answer Recall Apply Create Marks
1 1   1     1
1   1 1     1
2   1   1   2
2   1     1 4
Totals 1 3 2 1 1 8

 

As I create a draft of my exam, I can see how many questions I have from each chapter to ensure I’m covering all the course topics.  I can tell how many marks I’ve allocated for each area, and how many of those marks are from recalling information versus applying it.  When I’m finished, I can remove or add questions to ensure enough weight is given to the important areas, and to ensure there is an appropriate mix of question types.

In giving students a ‘heads up’ on the exam, I can tell them what chapters are covered and what percentage of the exam is from what area.  Using the above blue print:

The exam will cover chapters 1 and 2.  There is 1 multiple choice question, and 3 short answer.  50% of the questions are on chapter 1; 505 of the questions are on chapter 2.

Or I could tell the students by marks:

The exam covers chapters 1 and 2.  There is 1 multiple choice question and 3 short answer questions.  25% of the marks are on materials in Chapter 1; 75% of the marks are on the material in chapter 2.

As a student, you know that you should be putting an emphasis for studying on Chapter 2.

Here’s an actual example of one I recently posted for a course:

The exam is worth 25% of the final grade and consists of 12 multiple choice questions (worth 1 mark each) and 27 short answer questions (worth marks as indicated) for a total of 100 marks. 

The exam questions are distributed across the materials we covered as follows:

Chapter Number of Questions
1  6
2  8
3  4
4  2
5  4
6  5
7  5
8  5

Of the questions, 55% are just demonstrating your ability to recall material; 38% of them are displaying that you can apply the material given a set of facts; 5% of the questions are asking you to do something with the information supplied. 

I hope this gives an idea of how to answer students’ questions, without giving away the exact answers and questions.

Advertisements

Rubric Repair

Looking for some tips on how to make your rubrics better?  Check out this episode from the Cult of Pedagogy podcast: https://www.cultofpedagogy.com/rubric-repair/

(There’s also a transcript there if you prefer just to read it. 😉 )

Interviewing an experienced educator and author, the episode offers five tips to use when building a rubric:

1) Measure what really matters

2) Weigh the criteria appropriately

3) Check your math

4) Can do rubrics, not can’t do

5) Models

It’s worth checking out and then having a look at your rubrics to see how they stack up to the tips offered.

Cheers!

 

Using the Whiteboard for More Impact

Using the whiteboard in a classroom seems pretty easy, but some pre-planning over how you are going to use it can improve the delivery of your lesson.

There are some basics ideas that it’s good to keep in mind:

-never  talk to the board, always write the point and then turn so you can address the class when you talk

-print large enough that everyone can see

-check the markers and erasers at the start of class (just ask the Registration Desk if you need replacements)

In using the board, plan out the location of materials that you are going to put on the board:

-assignment instructions, if being listed on the board, should stay visible for the whole class

-make use of a “parking lot” – a space for ideas or questions that don’t quite fit the moment but are worth coming back to later.

-have an active space for materials that you are currently discussing, then clearing off for the next piece – but erase it only have you are sure everyone got the points down!

-terms or learning outcomes can be listed on the board either at the start of the class or as they are encountered so students can see the progression during the day

-put the agenda for the day up, students can see progress over the class and queue to the topic if they miss something said aloud

-identify items that need to be put up in advance, like tables or formulas, so that you aren’t loosing time or momentum during the class

-write student responses to questions on the board to ensure everyone knows the answer and that you heard the answer correctly

-think about a structure for the board in advance, divide the space up so that you can use it effectively and consistently week to week.  Like consistency in slide design, students will come to recognize the connection between items if they appear in groups on the board from class to class

-think about using colours for a purpose: terms are in green, formulas in red for example

The whiteboard is a teaching aid.  Like any teaching aid, thinking about it’s use in the class and how it improve the lesson delivery is an important part of prepping for class.

How Hard Should Your Test Be?

When I started as an instructor I struggled with the idea of how hard to make my tests and assignments.  Even with time, it is still a bit of a guess.  This past month, the Scientific American posted a blog summarizing some recent research.

In short, you don’t want your course or exam to be too easy, then students are challenged and true learning doesn’t occur, nor do you want the test to be so hard that no one can pass it.  The ideal sweet spot for learning is 85%.  At that point, students are being optimally challenged and are still getting enough correct answers to engage their interest and keep them motivated.

The theory is derived from the work of a team lead by Robert Wilson for the University of Arizona, whose team studied the ideal point of difficulty to enhance the learning of material and how the level of difficult impacted that learning.

If you want to read the blog post, click here:

https://blogs.scientificamerican.com/observations/how-wrong-should-you-be/

If you want to read the research paper that it was based on, you can read it here:

https://www.biorxiv.org/content/biorxiv/early/2018/01/27/255182.full.pdf

 

 

Are Learning Styles Real?

When I was learning to be an instructor, I was taught two key points that I was told to ensure all of lessons plans addressed.  One was the Learning Pyramid and the other was learning styles; it was stressed that to be a good instructor, these needed to be in every lesson.

The Learning Pyramid is a theory that people learn only a small part of a lecture, learn 10% from reading, learn 20% from audio/visual, and so one till you get to the base of the pyramid that says we learn much more by ‘doing’.

If you’ve heard of that theory, I hope you’ve also heard that it’s false.  There are a number of articles out there that show it’s not true, including some that show the supposed source never put out the stats that the pyramid claims:

https://www.washingtonpost.com/news/answer-sheet/wp/2013/03/06/why-the-learning-pyramid-is-wrong/?utm_term=.c1f9a1dd7d90

https://acrlog.org/2014/01/13/tales-of-the-undead-learning-theories-the-learning-pyramid/comment-page-1/

I quickly picked that up and have dropped it from my materials and teaching plans.  Now it seems the other ‘must do’ I was taught is also false; teaching styles are myth says a lot of research, and yet it persists:

https://www.insidehighered.com/news/2019/01/09/learning-styles-debate-its-instructors-vs-psychologists?utm_source=Academica+Top+Ten&utm_campaign=d835e28cab-EMAIL_CAMPAIGN_2019_01_09_05_51&utm_medium=email&utm_term=0_b4928536cf-d835e28cab-51944493

Rather than try to cater to every learners’ style, it’s more important to be clear and gain understanding by teaching material in a variety of ways. The type of material and understanding being sought will play a larger role than ‘learner style’ – teaching someone to do something (like play soccer) needs tactile / hands on work, while teaching someone to be able to define concepts will lean more to discussion or lecture.

The Forgetting Curve

Ever had that moment where you’ve met someone and two minutes later, you can’t recall their name?  Or read a passage in a book, but you can’t remember what it said?  Think of your students, how much do you think they remember after the class is done?  Would you believe they can forget as much as 80% by the day after your class?

Back in the 1800’s, Hermann Ebbinghaus, a German psychologist, did a study on how people forget what they’ve learned.  It was a limited study, but the theory has been replicated and become known as the Forgetting Curve.

The Forgetting Curve is exponential.  The student walks out of the lecture knowing 100% of the material, by the day after 50%-80% is gone!  Each day after that gets progressively worse.  By day 7, students have forgotten significant amounts of material.  A month after a lecture, students may only retain 2% – 3% of what they learned!

Graphically, the forgetting curve has been portrayed like this:

forgettingcurve_figure1

(source: https://www.growthengineering.co.uk/what-is-the-forgetting-curve/ )

A huge part of beating the Forgetting Curve is on the student.  Doing recall exercises the next day, ie., studying.  But as instructors, we also play a role in beating the Forgetting Curve:

-Over the duration of a course, or even during a lesson, build in repetition.  Reinforcement is a big part of beating the Forgetting Curve.  During the lesson, build in repetition, to help ensure the material is being learnt.  Then in subsequent classes, come back to it, reinforce the material and ensure that it is being retained.

-Make the lesson, and the material, memorable.  Teach the lesson in a way that students will remember it; give the lesson meaning so students know why they need the lesson, show the teaching point matters.  Deliver it in a way that works for students and will be memorable – be that a game, a story, an example, an activity.

-Share memory tricks.  Is there a mnemonic or memory device that can help to learn the material? Share it!  (How many of us still recite 30 days hath September, April, June, and November….?)

-Be clear in your teaching.  The worse piece is delivering teaching points in a way that can’t be easily followed.  If students struggle to get the material in the first place, they won’t be able to recall it later.  So ensure that the lesson you’ve created is clear TO THE STUDENT.  Too often it’s clear to the instructor, but it’s the student that matters!

Want to know more about the Forgetting Curve, and how easy it is to beat with studying, check out the University of Waterloo post:  https://uwaterloo.ca/campus-wellness/curve-forgetting

Want to see some more ideas about how to improve delivery to beat the Curve, this post is specific to online learning (and was the inspiration for this post), but the ideas can be adopted:  https://elearningindustry.com/forgetting-curve-combat

Grade Insurance

Welcome to 2019!

When I was studying for university, I had a professor who offered grade insurance on day one of the class.  By doing an extra assignment, students could get insurance worth one mark toward their final grade.  If a student was one mark short of passing or one mark short of the next letter grade their assignment was read and, if it met the minimum requirement, it earned the mark to bump a student up so they didn’t fail or earned them the next letter grade.

To be eligible, students had to submit the extra assignment BEFORE receiving their first mark.  Much like insurance on your car, you can’t buy it after you’ve had an accident.

To be read, students had to be one mark short of the next letter grade.  Not two, not three, one mark and one mark only.  Students were told up front that if they fell one mark short and hadn’t taken out ‘insurance’, there would be no discussion about adjusting or raising their grade.

To qualify for the mark, the submission had to meet an established and posted rubric.  This was standard to any assignment.  It included components on grammar, content, writing skills, etc.  To earn the mark, students had to earn a C or better on the assignment.  It was not a case of submitting the extra work and automatically getting the grade as the submitted assignment still had to meet a minimum level of effort and quality.

Like any assignment, this extra insurance assignment had assignment instructions that spelled out the ‘question’ to be answered in the assignment, the format, the length of the response, and the due date and time.

Given that a portion of marking is subjective, no matter how good a rubric is designed, this grade insurance is a way for the instructor to head off debates about the subjective qualities and to alleviate the pressure from students looking to raise their grade.

Next class I teach, I’m going to give it a try.

Gold plating in the Classroom: A Lesson from Project Management

Gold plating is a term in project management that refers to the project team putting extras into the project.  Extras that are not asked for by the client, extras that are not being paid for by the client.  In general, it’s considered a bad thing and should be avoided.  The classroom is the same as a project, gold plating should be avoided. In short, it’s about trying to teach too much.  A common theme in a lot of literature aimed at college and university instructors.

In the classroom context, gold plating is adding in extra teaching points, expanding the lesson beyond the learning outcomes, or adding in trivia to expand the class.  In general, it’s a bad thing for our students.

With most of our classes at 36 hours, with students in the full time program taking courses five days a week.  Students rush to finish assignments, cramming for tests, and juggling any home commitments they have.

Gold plating puts extra pressure on students to learn more points, to try and retain ideas and concepts that aren’t needed.

It’s not that ideas and concepts shouldn’t be expanded on to ensure understanding, adding personal stories and examples that help students fully understand and apply the main lesson.  Those should always be there, but adding in extra material that takes the course further, that should be viewed with skepticism.

As an instructor, strive to find the balance between delivering what the student needs, what PACE wants, and adding in too many extras.

Improving Writing Skills

October’s Workshop Wednesday was a presentation by Scott Poole, EALTCP Coordinator in the UWinnipeg’s English Language Program.  Scott talked about what instructors can do to help improve the writing skills of our students.

There is no magic bullet that will make students write better.  Writing is a skill that takes time and feedback in order to improve.  Scott offered a number of ways that instructors can help with.  To highlight just three:

  1. Conferences: one on one coaching with students is the best way for students to get meaningful feedback and for instructors to assess that the feedback has been heard and understood.
    While not easy with the numbers in some of our classes, this may be done selectively with those students that need more help than others.
  2. Rework: for students to truly learn and get the opportunity to improve, students need repetition.  In the ideal world, that means allowing students to rework submissions so that they can build on their skills and get more feedback.
    As our classes generally run over very short periods of time, this may not always be possible.  A variation of this can be achieved by laddering assignments, where they build or come together so that students get that opportunity to apply feedback and try to learn and grow in their skills.
  3. Peer editing: even when peers’ skills may be weak, getting a second set of eyes to look at a document can help both people to grow their skills.
    This does not mean that the peer rewrites the paper, and the topic may itself need some coaching and explanation in our classes as this is a new concept for many.  The benefits though can be tremendous.

In general, we run an Instructor Workshop session the first Wednesday of most months.  They are always voluntary, we encourage you to come and share experiences, meet other instructors, and pick up some tips.