High-Yield Concepts in Psychology Experiment Design

Session 156

Experiment design has the potential to appear in three of the four MCAT sections, which makes it super high yield. Follow along with the handout!

Once again, we’re joined by Clara from Blueprint MCAT (formerly Next Step Test Prep), one of the best MCAT test prep companies in the country. Visit their site and fill out your name and contact information to have a free consultation with their team. Please let them know you’ve heard about them here on the podcast.

Listen to this podcast episode with the player above, or keep reading for the highlights and takeaway points.

[02:10] Why Experimental Design is Important

In the Psych section, virtually every passage you see will have some mentioning of experimental design. And most passages will have multiple questions about how the researchers set up an experiment.

'These principles can come up in the other science sections too.'Click To Tweet

High Yield Topics in Psych: Experimental Design

[03:22] Question 22

Several statisticians notice an interesting relationship between a high-carb diet and a particular visual deficiency. In all populations, consumption of large amounts of carbohydrates appears to worsen the deficiency. However, in Europe, a high-carb diet has a much stronger effect on disease outcome than in Mexico, where it only slightly appears to impair patients’ vision. In this scenario, which type of variable is location?

(A) Independent

(B) Moderating

(D) Confounding

Clara’s insights:

The correct answer here is B. In this case, you really have to know the definition of these terms.

Moderating affects this relationship. And if you think of it like moderating variables affect the strength of a relationship, this can be helpful.

In the first sentence, they mentioned that there is this relationship between high carb diets and this visual deficiency. What happens based on location is that in one place, it has a really strong relationship between the diet and disease outcome. Whereas in the other location, there’s only this slight relationship.

Since we see here that location affects the strength of the relationship, then the relationship exists no matter where we are from what we know. But it’s a lot stronger relationship in Europe.

Hence, B is the correct answer here.

Let’s dig in to get a better understanding of the other answer choices. If we graphed our relationship, independent is that part where the experimenters alter or impact. And dependent is what’s being measured.

Mediating variables are the ones that explain a relationship. Where A impacts B because there’s C where A directly affects C, and C directly affects B. For instance, summer leads to more ice cream sales because it’s hot in the summer. You can rephrase this to make heat as your variable.

Confounding variable is a variable wherein we’re not actually intending to study that has a measurable effect on what we’re trying to measure on a dependent variable. Confounding variables are the things that mess up our study.

[10:33] Question 23

Which of these experimental setups most clearly lacks external validity?

(A) A study that measured athletic ability based on soccer talent, but entirely neglected other forms of athletic skill

(B) A protocol that established an apparent causal relationship, but calculated a p-value of >0.10

(C) A procedure that found a relationship between infants in specific laboratory conditions, but may or may not apply to infants in typical households

(D) A survey for which researchers could only obtain a very small number of participants

Clara’s insights:

The correct answer is C. You need to know what these terms mean and you can actually deduce what it means by its name.

External validity is a measure of a study that says how applicable are its results to external situations. This is a big problem a lot of studies have. The study is very strictly controlled and every possible confounding variable is minimized. Hence, it could end up with a really good study design but it doesn’t apply to real life.

The answer is C because it says that it may or may not apply to infants in typical households. Therefore, it lacks external validity.

Basically, internal and external validity are the two types of validity that you’re going to see on the MCAT oftentimes. A is just too narrow of a definition. B looks like a statistical error. For answer choice D, small sample sizes will have a lot of negative impacts on studies. But in itself, it’s not necessarily a lack of any kind of validity.

For example, several studies on very rare diseases are always going to have a small number of participants. And if they’re well-constructed, they could still be good studies.

[16:21] Question 24

An experimental protocol that relies entirely on surveys and questionnaires is especially vulnerable to which for of bias?

(A) Social desirability bias

(B) Confirmation bias

(D) The availability heuristic

Clara’s insights:

The answer here is A. Social desirability bias is exactly why surveys and questionnaires are not great methods. This is because there is this tendency for people to frame how they want other people to think of them as they’re responding.

For the answer choices, confirmation bias is where we tend to favor pieces of information that fit in with beliefs that we already have.

Stereotype threat is highly MCAT-relevant but not relevant here. This is the idea that if you’re aware of a certain stereotype, then you’re more likely to act in certain ways that are affected.

A great example of this would be is the stereotype of girls not being as good at math. Then someone tells a girl this before she takes a math test. It then makes her super nervous and she would then actually perform worse. This is because the girl was affected by stereotype threat.

The availability heuristic is a mental shortcut where we tend to favor pieces of information that are more readily available. Rather than thinking about every relevant piece of information you might have.