MCAT CARS Asks: Can Computers Learn Common Sense?

Session 81

Some of the smartest machines on earth can’t tell you what would happen if you dropped a match on a stack of logs—so, are they really that smart?

Once again, I’m joined by Jack Westin. The new MCAT dates are opening up they’re getting ready to start. May 29 is the next batch of dates. And the AAMC surprisingly released the actual structure of the new shortened test.

Jack Westin has created a Chrome extension that you can download. With a click of a button, you can change the look of the practice products to the actual MCAT design interface.

Listen to this podcast episode with the player above, or keep reading for the highlights and takeaway points.

Link to the article:

https://www.quantamagazine.org/common-sense-comes-to-computers-20200430/

One evening last October, the artificial intelligence researcher Gary Marcus was amusing himself on his iPhone by making a state-of-the-art neural network look stupid. Marcus’ target, a deep learning network called GPT-2, had recently become famous for its uncanny ability to generate plausible-sounding English prose with just a sentence or two of prompting. When journalists at The Guardian fed it text from a report on Brexit, GPT-2 wrote entire newspaper-style paragraphs, complete with convincing political and geographic references.

Marcus, a prominent critic of AI hype, gave the neural network a pop quiz. He typed the following into GPT-2:

What happens when you stack kindling and logs in a fireplace and then drop some matches is that you typically start a …

Surely a system smart enough to contribute to The New Yorker would have no trouble completing the sentence with the obvious word, “fire.” GPT-2 responded with “ick.” In another attempt, it suggested that dropping matches on logs in a fireplace would start an “irc channel full of people.”

Marcus wasn’t surprised. Common-sense reasoning — the ability to make mundane inferences using basic knowledge about the world, like the fact that “matches” plus “logs” usually equals “fire” — has resisted AI researchers’ efforts for decades. Marcus posted the exchanges to his Twitter account with his own added commentary: “LMAO,” internet slang for a derisive chortle. Neural networks might be impressive linguistic mimics, but they clearly lacked basic common sense.

[05:16] Paragraph 1, Sentence 1

One evening last October, the artificial intelligence researcher Gary Marcus was amusing himself on his iPhone by making a state-of-the-art neural network look stupid.

Jack says:

We have Gary Marcus, an artificial intelligence researcher. So he’s a smart guy, apparently. And it’s saying that he’s using his iPhone to make a neural network look stupid.

Read and understand while you’re reading. Don’t just read it and trying to figure out the topic and then plan to read it again. Because if you do that, you’re gonna spend way more time reading. It’s going to cause panic concern because you’re going to feel like you’re not getting it.

'When you when you start off a passage, especially the first sentence, take your time.'Click To Tweet

There’s a lot going on here, neural networks, artificial intelligence – all this stuff is happening. If you’re not comprehending it in a coherent kind of fashion, you will get lost later.

It’s really important to understand the basics. So basically, artificial intelligence has probably something to do with computers. If you have a general understanding of that, then you’re fine. They said the guy’s name Marcus was amusing himself on his iPhone. So he’s playing with his iPhone. Try to visualize all of this.

You don’t need to know what a neural network is. It’s state of the art. It’s probably high tech. It’s probably really advanced, really smart. But what is Marcus trying to do here? He is trying to make it look stupid, trying to make it look bad.

Maybe he doesn’t like it. Maybe it’s trying to prove a point. We don’t really know. But if you’re going to try to make something look stupid, you’re mocking it. You’re making fun of it. So you probably are not in favor of it. But we don’t need to assume too much. We just have to know what’s happening.

[08:10] Paragraph 1, Sentence 2

Marcus’ target, a deep learning network called GPT-2, had recently become famous for its uncanny ability to generate plausible-sounding English prose with just a sentence or two of prompting.

Jack says:

Now, we know what he’s trying to make look stupid, this deep learning network. So maybe neural network and deep learning network are synonyms. It’s telling us what GPT-2 is, it’s able to make a prose from a prompt.

[09:11] Paragraph 1, Sentence 3

When journalists at The Guardian fed it text from a report on Brexit, GPT-2 wrote entire newspaper-style paragraphs, complete with convincing political and geographic references.

Jack says:

You don’t need to know The Guardian, but it is italicized. So if you see anything in italics, you should think it’s probably a title of a book, movie, a journal.

Now they could represent a title of a ship, a captain, but typically, it’s a piece of work. It’s an example. It’s just showing us what GPT-2 does. They probably brought up GPT-2 to show us how Marcus will use this to make it look stupid.

“Don't forget the first sentence. Always ask yourself, why are we talking about all this stuff? Because you can use that to predict what's coming up.”Click To Tweet

A lot of students see a disconnect between that first sentence and the second sentence of this paragraph. Just try to understand where we are, what the author is trying to talk about, who’s saying what.

Marcus is not saying anything. Marcus is just doing something and the author is explaining what Marcus is trying to do. There are millions of ways to tell a story. There are millions of ways or billions really to tell to explain a passage. So this is the way the author decided to do it. Let’s go ahead.

[12:57] Paragraph 2, Sentence 1

Marcus, a prominent critic of AI hype, gave the neural network a pop quiz.

Jack says:

Now we’re given a little bit more information on who this Marcus guy is because in the first sentence, it said he’s an artificial intelligence researcher. And now this sentence saying that he’s kind of a critic of AI. So he’s not just a researcher, he is a critic of all the hype.

He’s probably trying to make the pop quiz look stupid now. That’s kind of the direction the author wants us to go in.

[13:46] Paragraph 2, Sentences 2-3

He typed the following into GPT-2: What happens when you stack kindling and logs in a fireplace and then drop some matches is that you typically start a …

Jack says:

This is the prompt that Marcus is giving GPT-2.

[14:37] Paragraph 3, Sentence 1

Surely a system smart enough to contribute to The New Yorker would have no trouble completing the sentence with the obvious word, “fire.” GPT-2 responded with “ick.”

Jack says:

The author here is saying that GPT-2 should spit out fire. That’s what should come next. Obviously, it didn’t respond with the right thing.

[15:23] Paragraph 3, Sentence 2

In another attempt, it suggested that dropping matches on logs in a fireplace would start an “irc channel full of people.”

Jack says:

Not the right answer either. So it just made the GPT-2 look very stupid.

[16:04] Paragraph 4, Sentence 1

Marcus wasn’t surprised.

Jack says:

Marcus trying to make it look stupid did make it look stupid and he wasn’t surprised.

[16:12] Paragraph 4, Sentence 2

Common-sense reasoning — the ability to make mundane inferences using basic knowledge about the world, like the fact that “matches” plus “logs” usually equals “fire” — has resisted AI researchers’ efforts for decades.

Jack says:

This is the author’s point of view, not Marcus’s point of view, but the author saying, this is why this neural network failed because they can’t make these inferences that humans can make.

The author probably agrees with Marcus because why would we be talking about Marcus and his point of view, and his experiment? It’s something that the author probably agrees with unless the author dramatically says no.