In reading Carnie's discussion of how we learn to formulate sentences both syntactically and semantically, I thought about how these rules could translate to computers--and how, in many ways, they already have. However, these attempts to teach computers Language are based off a model of Language, not the entirety of it; as Carnie explains, linguistics is more psychological than neurological. As such, some of the results of offering a very simple Language model can be pretty humorous. The writer/ coder Jamie Brew (https://twitter.com/jamieabrew) posts, from time to time, snippets of text produced by feeding text samples to a program, having it analyze the syntactic distribution (which words most often come before/after other words) and spit out something similar. This process is called "predictive text." It produces sentences that are very often syntactically well-formed, even quite advanced, but semantically ridiculous. For example, when fed recipes from The American Woman's Cook Book (1938), the program returns an imitation, including instructions such as "in an opportunity thick enough to taste, pour custard over each person who cannot drink your refrigerator." Semantically, ridiculous, but syntactically? Pretty good. You can do a similar thing by opening a predictive text keyboard on your phone and continually tapping the suggestions of the keyboard program, to produce output like, "I am going through my resume to the inbox to get back with a few weeks back I will have to get back to you...," etc.
These are but small examples of how code can be taught to understand or imitate Language, however, and there have been great advances in the field. For instance, the neural networking system MetaMind (https://www.wired.com/2015/06/ais-next-frontier-machines-understand-language/) is actually able to read texts and answer comprehension questions about the content. This provides what the predictive text above lacks: semantic understanding. MetaMind is able to utilize the rules of language, such as pronouns having antecedents, because if there's one things computers are good at, it's doing what you tell them. That's how a robust analysis of the rules of language can be transformed into an amazing advance in machine learning and language processing, from simple rules like word frequency, all the way to an advanced model of both semantics and syntax.
I found it really interesting how you specifically pointed to this nice example of syntax being distanced from semantics. Sure, Carnie makes the point that some parts of Language must be innate, because it's infinitely generative; but, as you've shown, it's possible to teach computers nearly perfect syntax - even if meanings are totally ignored. And this is, as it seems, without any models of syntax other than the sentence itself (e.g. like directly giving a driving-route-finding algorithm a picture of a map instead of the usual contingency table.)
ReplyDeletePerhaps, then, if human minds could work as well as dynamic programming algorithms used in machine learning, all we'd need innately to learn syntax would be some sort of algorithm and the hardware in our brains to run it?