How Can I Prepare for the CASPer, What is CASPer and More

Session 303

Today, Dr. Kelly Dore, who is one of the brilliant minds who developed and researched CASPer, will talk about what it is, how it’s scored, and hopefully, listening to this will help relieve some of your anxiety and fear over this test. Over 30 schools now in the U.S. as well as in Canada are using CASPer in place of or in addition to the secondary essays. CASPer stands for Computer-Based Assessment for Sampling Personal Characteristics.

Check out all our other podcasts on MedEd Media Network including The MCAT Podcast, where we talk about MCAT prep. The OldPreMeds Podcast is for nontrad premeds where we take questions from the forums. The Specialty Stories features physicians as they share their specialty. We also have the Short Coat Podcast from the University of Iowa Carver College of Medicine and the TMDSAS Podcast for those applying to Texas medical schools. Please help us spread the word and share this podcast to even just one person today!

New CASPer Webinar: May 10th, 2019! Register: http://www.takecasper.com/webinar-may10

[02:40] A Little Background About Kelly

Kelly is a full-time faculty member at McMaster University in the Department of Medicine and the McMaster Education Research, Innovation and Theory Program in Canada. She did her undergraduate and PhD at McMaster University, specifically in Cognitive Psychology and Health Research Methodology, where she specializes in Cognitive Psychology and Medical Education or Health Professions Education.

During her time at McMaster, she worked closely with several medical school groups. She had a role in the medical school admissions committee where she had to look at the equivalent of what would have been their secondary examination. She would look at the autobiography submissions of students who wanted to come to McMaster for medical school where they would write short essays.

[Tweet “”I got involved in seeing the limitations and problems with that process, trying to figure out a different way.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

Seeing the limitations and problems related to the secondary essays they were giving to students, she worked with another colleague to figure out a solution that’s better. Hence, CASPer was born at McMaster University.

[04:21] The Inherent Problems of Secondary Essays

[Tweet “”The secondaries, these references, letters, and personal statements, they just don’t tell us anything about how the applicant’s going to perform in the future.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

When Kelly was reading the essays, she found it hard to differentiate applicants. They had probably studied the school’s website so they came in so prepared to give the key talking points. So at the end of the day, it was difficult for them to identify who the applicant actually was and separate applicants from each other. This is the main reason they needed an additional piece to the process. They didn’t want to eliminate applicants from the pool by just using grades because she admits their secondaries cannot really predict too much.

Kelly points out the importance of leading people with a sense of who you are and why you want to do this, not what you just think is on the website or what you googled is the right answer because this doesn’t set you apart.

[Tweet “”Lead people with a sense of who you are and why you want to do this, not what you just think is on the website.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

[07:15] Behind the CASPer Idea and Developing the Test

Kelly recalls figuring out this process began back in 2004-2005 and they didn’t really implement anything at McMaster until 2010, as part of their local solution. So they took that time to figure out what the better answer would be.

First, they looked at what was going on. And at that time, they saw there was a huge push for social accountability. They wanted to ensure that people graduating were more than just competent, knowledgeable physicians, but people who could communicate with everybody. They wanted people who were the holistic practitioners who would put themselves in other people’s shoes.

At that time too, Maxine Papadakis’ work came out, where she did an incredible review of graduates from UCSF medical school in California. She looked at those graduates who had professionalism problems in practice. She tracked that back by looking how they were performing in school. And she found a strong correlation between those problems in training and professionalism problems in practice. So, McMaster wanted to pay attention to that. They weren’t just creating an assessment of nonacademic attribute to tick a box. So they wanted to create something evidence-based so they can be informed. As a result, they were graduating practitioners who aren’t just smart but who could also adapt and function within the complex healthcare environment.

[Tweet “”We wanted to make sure that we weren’t just creating an assessment of nonacademic attribute to tick a box.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

[09:45] Situational Judgment Questions vs Behavioral Descriptive Questions

Additionally, they were looking at the way they were asking questions. Kelly describes a lot of the secondaries tend to be about behavior descriptive questions, like “tell me about the time when you…” But these are questions you can easily google. And they found that those questions tend to create the superhero phenomenon. People had to create a complex scenario that they then solved. But that doesn’t tell you about the applicant.

Instead, they looked at situational judgment test questions or SJTs, which unlike behavioral descriptive questions, they give you a scenario you need to respond to. This then positively levels the playing field. You’re not giving an advantage to people who didn’t have the opportunity to volunteer elsewhere. For instance, applicants coming out of university, even if they had no healthcare experience, could respond to these scenarios.

[Tweet “”You can construct these SJTs in a way that actually allows you to get a sense of the applicant and how they respond.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

[11:22] Moving to an Online Model

As with the online nature of the exam, Kelly admits this has taken awhile actually. After they started to create the test in 2010, the technology was barely there to create an online platform. But they knew they needed to push this online test. But along with the high cost that comes with the idea of putting this test in a testing center, they also know they didn’t want to disadvantage applicants coming from remote regions to have to travel to testing centers in major cities and have to incur hundreds or thousands of dollars in additional cost just to write a test.

They also tried audio recordings as well as having longer test times. They looked at all the nuances to try and create this final version of what they think the best CASPer test is.

[13:20] What is CASPer?

CASPer is an online, primarily video-based situational judgment test. Out of the 12 scenarios, 8 of them are video-based. You get to watch a 60- to 90-second video that presents the scenario. In the other 4 scenarios are text-based questions. After seeing the scenario, you answer three open-ended questions about your response to that particular scenario. You have 5 minutes for each scenario to type your response to the three questions. After the 5 minutes is up, it then moves you up to the next scenario.

The scenarios have been developed in a way that there’s no requirement of the healthcare system. It also blinds you in terms of needing to have special knowledge or skills or access to be able to answer these. Hence, it really doesn’t require any knowledge of any of the individual’s schools or any of the nuances of what’s expected of early medical students.

The test was developed in a way that leverages the value of a bell ringer or multiple mini interviews where you get multiple independent assessments of an applicant’s performance. Kelly adds that this is great for the applicants in that your performance on one scenario has no influence over how you’re going to be scored on the other scenarios. So if you feel you didn’t do well on one of them, that doesn’t influence any of the other scores.

So if the applicant is nervous or doesn’t feel like they gave a good answer to a question, it doesn’t impact you, as much as a traditional interview or panel interview where your answer to one question can actually bias the way your other responses are seen.

[Tweet “”Your performance on one scenario has no influence over how you’re going to be scored on the other scenarios. ” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

And because this test is done online, there is no need for students to go to a test center. The CASPer is aimed at assessing your nonacademic qualities complementary to things like GPA and MCAT, to give the admissions committees a larger sense of who you are as an applicant.

[16:00] MMI and CASPer Both Come From McMaster!

Kelly was also involved in the Multi Mini Interview at the very beginning of its implementation and development. She has also been involved in a lot of research around it subsequent to its implementation.

Their admissions chair was engaged and excited to push the envelope in terms of thinking. He wanted to make sure there was evidence behind everything they were using. That said, Kelly describes McMaster as the perfect petri dish for the development of these things. They were creative in looking at these things, aside from the like-minded individuals working there who were keen and excited to try and develop something better than the status quo.

[17:15] What They Look For in Students

Kelly explains the reason for CASPer’s implementation is they want to have good psychometrically strong, evidence-based measures of cognitive ability. The wanted to have the same on the other side for these measures of nonacademic abilities. So they hope to give added value to the information you get from applicants outside their academic performance.

They can tell when somebody is strong academically, but they weren’t seeing the same from a nonacademic professionalism side. Now, the degree or the nuance of that would already depend on each individual school.

[Tweet “”The hope is that we’re bringing in applicants now who have strength in both areas.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

[18:35] How CASPer is Scored

The schools would get a Z score which tells the admissions program how the applicant has scored relative to the other applicants who wrote that test. So you end up with a cumulative Z scores and profile of your applicants.

First off, they have data that demonstrates minor spelling and grammar mistakes don’t count in terms of CASPer score. So this is not something to worry about, as Kelly points out. They also have information that the faster you type doesn’t improve your CASPer score. They encourage you to use bullet points.

[Tweet “”There is no influence of minor spelling and grammar mistakes in terms of your total CASPer score.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

Meanwhile, Kelly breaks down the different CASPer raters. A CASPer rater is somebody different than some admissions committees. They’re not all faculty members or academics. They want to make sure their raters are representative of the population that the applicant may one day hopefully be serving when they become a health professional. This being said, their raters are a combination of demographically diverse raters – from community members to stakeholder groups, advocacy groups, faculty members. etc. The goal is they have different ages, backgrounds, and different perspectives they’re bringing in.

[Tweet “”We want to make sure their raters are representative of the population that the applicant may one day hopefully be serving when they become a health professional.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

[21:13] The Onboarding Process for the Raters

Raters would have a large onboarding process. They’d have to write their own CASPer test to give them a sense of what it’s like. They’re only also seeing your response to a single scenario. They’d also be given a background in theory. This will give the raters a sense of what they’re looking for within that particular scenario. Kelly also reassures that in every scenario, there’s no single thing they’re looking for, or single right or wrong answer. They convey this both to the raters and applicants. So both applicants may say they’d do completely different things but both for very appropriate reasons. Raters are trained to know that there are diverse ways to approach the problem. It’s not just what you’re going to do but why you’re going to engage in that course of action, which is valuable to understand.

[Tweet “”The value of having open-ended responses is that it’s not a cookie-cutter. It’s not a single type of response.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

In CASPer, they’re using the theory of planned behavior to try and understand the values and belief systems of the applicant. This way, it would give them a better sense of how and what the applicant is going to do in the future in these complex scenarios.

[23:40] What Are CASPer Scenarios Like?

Kelly describes a good scenario is one that she wouldn’t want to answer personally since it’s hard, tricky, and complex. It forces you to think through it. And the reason being is that these are the scenarios people are going to struggle with later on. The applicants are always given a role so they would know how they’re supposed to engage in the scenario.

Here’s one example of a scenario. Somebody is trying to return something to a store. They don’t have the receipt but they need it. This is against policy. But the manager is away. Your role in the scenario is the more experienced employee. The junior employee is at the desk trying to manage the situation and the expectation of the customer who wants a refund. And it turns out she needs to buy her daughter a prescription so she needs to have a refund immediately and needs it today. She can’t wait. Since it’s against policy, what are you going to do? The junior employee turns to you and asks you what to do.

Then there would be three probing questions, not what you’d do in that scenario, but why. It’s the process of being able to express the reasoning behind it to make sure you’re not making assumptions. Additionally, this enables you to give answers they would never have thought of. This is the very reason Kelly loves open-ended responses to see the things applicants come up with.

[25:40] Fixed Response SJTs vs. Open-Ended Response

Although not in the CASPer test, other forms of SJTs would, for instance, give five options and then you’d have to rank them in terms of appropriateness. The limitation with this is that what if your answer isn’t one of those. So you’re forced to say this is the closest to what you would have done. So here, you’re trying to read the mind again of what you think the admissions committee would want you to do. As opposed to actually talking about yourself and what you really would do. Kelly adds that raters are also able to adjust their scores. Sometimes, when you do your early scoring, you may think later on that it has to be higher. So they give raters that flexibility to go back and make sure they’re rating applicants relative to each other.

[Tweet “”Some of these answers that we’re getting blow you absolutely out of the water and shock you. But unless you came from that person’s background and their context, you would never have thought of it.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

[27:27] Adjusting for Individual Biases

Kelly explains there’s always individual bias that’s going to come into play in terms of any assessment. Even if it was just a pool of faculty members evaluating the test, there would still be a variation in their perspectives and biases.

[Tweet “”What they strive for as an assessment is to be as objective as possible but there’s always subjectivity in everything we do.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

All this being said, there’s a large amount of quality assurance done at the moment for their raters. After a test has been written and sent to the raters to be scored, they also have to score a variety of CASPers that are old ones before they’re actually released into the test itself. So they’re not able to move on to scoring a CASPer test until they’re ready and have achieved a certain level of thoughtfulness in their marking.

As part of the quality assurance, they also overserve responses to make sure their raters are scoring properly. They also look for raters who are spending enough time on responses. They also try to remediate raters who they don’t think are performing well.

[29:35] Background in Theory

In helping raters with their scoring, they’re given a background in theory. For example, they’d know what is hoped for to be generally covered such as empathy, communication, collaboration, or whatever the combination is for that particular scenario. They also explain why this thing is important in medicine. So they want to make sure it’s not just the word “empathy” that’s shining brightly to them. But they also have to understand how that plays in creating the next generation of physicians who have strong nonacademic qualities. They want applicants to always make sure they’ve unpacked their thinking in their response. This is one of the tips they give to each applicant.

[Tweet “”It’s the process of knowing where these constructs fit within the bigger scheme.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

[30:55] Preventing Leakage of Scenarios

Students will be made to sign a confidentiality agreement so by posting the scenarios, they would be breaching that. But if they see them posted elsewhere, they would put forward a cease and desist. Ultimately, each CASPer test is comprised of new content. So they make sure it’s blueprinted to a general test pattern but each test has its unique videos, questions, etc.

So preparing for a test using old content would not help you, Kelly says. In fact, they’ve actually done a study around this. One year, she released four videos to everybody writing CASPer. On test day, she used two of them for their CASPer test but she changed the associated probing questions. So they had the chance to see the video but they’ve switched the content around. They found that applicants even did worse on those previously seen videos than they did on those they haven’t seen.

[Tweet “”Because you only have a five-minute response time, it’s very hard if you have pre-prepared responses to get that cognitive flexibility to actually answer the question.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

Pre-preparing the materials actually can do you more harm than good. They encourage people that if you want to practice and take notes, feel free. But just find a quiet space. Sit back. Pay attention to the scenarios and just be yourself answering it and you’re going to do better.

[34:00] Preparing for the Test: Would It Help to Go to a Test Prep Company?

In 2010, when McMaster implemented CASPer for the first time, there was already a test prep company that popped three months before. While unavoidable, they want to make sure they’re able to level the playing field.

[Tweet “”We don’t feel that there’s any advantage to these coaching test prep companies, but we’re going to make sure you have access to the way to prepare for the test.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

On the CASPer site, when you sign up for the test, they’re outlining a few things. First, as you go through the technical requirement test, it will take you to the actual CASPer test itself. There are also questions online for you to practice that are CASPer scenarios. Some scenarios she has seen from other companies are nothing like the CASPer scenarios so Kelly warns not to use those.

Kelly adds that on the actual website, you can sign up and you can take the practice test as many times as you want. This way, you can become familiar with the format, even spend a thousand hours if you want to. But Kelly suggests you write your responses then go back and look at them. Answer the response within the five-minute time constraint. Then go back and look at it and do a personal reflection on your own response.

Before you write your test, everybody gets an email about maximizing your success on CASPer. There are helpful tips they’ve created in terms of actual evidence they found on test. First, take the full five minutes to respond. They found that people who were only spending 3-4 minutes aren’t doing as well as those spending longer on their scenario. SO make sure the full amount of test time.

They’ve also done a webinar to unpack some of the nuances of CASPer people can go to for helpful tips and preparation.

[36:50] The Effect of Typing Speed on Getting Their Ideas Out

Kelly explains that once people reach a certain minimum threshold, they’re also looking for a Grade 7 reading level. After you hit that level, your score doesn’t improve. They think it’s important that when you think about not how you’re saying it but what you’re saying. They don’t need a master’s level English pros to write the test. Kelly recommends using bullet points. Just make sure you’re organized in your thoughts. You can answer them in any order you want.

[Tweet “”Take a few moments and become familiar with the test format. Take the practice test to see what it’s like to get your thoughts down.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

Give yourself additional time as to what else you didn’t think of within those five minutes. Again, ultimately, Kelly stresses it’s finding that quiet spot to do it.

And because they let the raters also take the CASPer test, they know what it’s like. So don’t panic if your thought gets cut off. Don’t worry about it. Take a deep breath and move on to the next scenario. As previously mentioned, you have a completely unbiased rater who’s ready to look at your next response.

[38:50] U.S. Medical Schools Using CASPer and Registration Basics

As of the moment, there are 35 U.S. medical schools using CASPer and they estimate that about 80% of the applicants would be writing a CASPer test. In Canada, they’ve got 8 of the 17 schools using CASPer. They also have a few DO programs using it as well.

[39:27] Registration Basics and Some Things to Consider

Kelly explains that there are set test windows that you can sign up for. For U.S. medicine, the test runs between May and February each year. But note particular schools you want to apply for. Some of them have smaller test windows. Make sure you’re going in early and that you can write in enough time.

There are multiple test slots each month. Go in and register for a CASPer test. Create an account. If you wake up on test day and you’re ill, you can reschedule your test up until you’ve started it. Just make sure you don’t apply too late to give yourself enough window for that in case you do get sick.

Once signed up for your test window, they encourage applicants to do that technical requirements test to make sure your computer meets all the technical requirements.

Other things to think about is that when you’re doing the test, make sure your family members won’t be downloading videos at that particular time as well. There are also live help chats both before and during the test.

You can only write the test once a year. They’ve shown no advantage to writing the test multiple times. Also, your results are shared with the programs you choose to distribute it to. If you want to update that, you can always go back in. If you apply for the FAP (Fee Assistance Program), all CASPer fees are waived as well.

[Tweet “”You can only write the test once a year.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

Generally, be sure your speakers are working, as well as your webcam since it will be turned on during the test. When you register, you’re required to present a valid ID that’s authenticated. Before you start your CASPer test, a photograph is taken and facial recognition software is used to make sure you are you. If you do things like wearing a hoodie or sunglasses or a mask, it won’t let you start the test. Or if you do any of that during the test, the test stops since the webcam is running the whole time.

They also found no advantage to group test taking. In fact, people who had done the test previously, prefer to write it independently the next time.

[44:55] Other Things to Know About

Kelly underlines the fact that part of the reason they developed this was a sense of fairness. They didn’t want to advantage people with a healthcare background, neither go through the process of double-counting. The raters wouldn’t know all the other aspects of your portfolio (GPA, MCAT score, etc.) so that it’s just the CASPer test they’re looking at. Students usually receive their score in 2-3 weeks. The raters are all online so they complete the test online by the time they do quality assurance.

Ultimately, Kelly’s advice to students who are still hesitant about taking the test is to avail yourself of all the resources to reassure yourself about the value of the test.

[Tweet “”The more you take a deep breath and be yourself, that’s going to make you shine through in this test.” https://medicalschoolhq.net/pmy-303-how-can-i-prepare-for-the-casper-what-is-casper-and-more/”]

Links:

Get the Podcast Free!

Leave us a Review and Rating!

Just like Yelp reviews or IMDB ratings help you choose your next restaurant or movie, leaving a 5 star rating and/or a written review is very valuable to The Premed Years. It allows us to be able to share our information with more people than ever before.

I am so incredibly thankful to those who have recently gone into our listing in iTunes to provide a five start rating and a written review of The Premed Years.