Google apps
Main menu

Post a Comment On: Understanding Society

"Analytic philosophy of meaning and smart AI bots"

1 Comment -

1 – 1 of 1
Blogger jed said...

Overall an interesting post. I share most of your criticisms of analytic theories of language. However your analysis of GPT-3 (and similar systems) falls short of the mark. Chomsky was using examples including "Colorless green ideas sleep furiously" to argue that utterances could be well formed but nonsensical -- and GPT-3 certainly proves his case! On the other hand Chomsky has always argued that people must have language specific mechanisms in their minds, with a specific sort of recursive structure (what structure exactly has changed over the years). GPT-3 clearly disproves that claim, and as such should be a major epistemic issue for Chomsky-aligned linguists, but I have not seen any of them respond to that aspect of GPT-3 -- either to accept the results or to dispute them.

The point about semantics deserves a little more discussion. Clearly GPT-3 has some sort of "surface semantics" -- it puts words together much more coherently than in Chomsky's example. It can manage local semantic agreement in "Winograd sentences" where the resolution of pronouns depends on some "world knowledge" (or we can now see more accurately "how people talk about the world knowledge").

But GPT-3 doesn't have -- and wasn't claimed to have or designed to have -- any deep knowledge about how we do things, how things work, what things look like, or even how words sound (so it isn't good with rhymes or puns). Those kinds of knowledge are somewhat modular and in humans depend on learning by other parts of the brain. We can reasonably expect that some language engine like GPT-3 can be linked to similar engines that have learned to hear and produce speech, use hands, recognize objects, etc. (All things that other deep learning systems do reasonably well.)

Probably such linkages would give the language produced by GPT-3 the deep semantics of those additional domains. Of course it would not have all the deep semantics humans have -- surgeons, carpenters, ethnomethodologists and so forth all have their own deep semantics learned by interacting with their special domains.

September 5, 2020 at 12:20 AM

You can use some HTML tags, such as <b>, <i>, <a>

Comment moderation has been enabled. All comments must be approved by the blog author.

You will be asked to sign in after submitting your comment.
Please prove you're not a robot