This week’s
readings accentuated the many layers of language, as not only do languages possess
literal and figurative meanings but also the many nuances in between. Nuances
in language, in word choice or stylistic changes, can help identify the
speaker, such as the African American woman complaining about the heat or the “burnouts”
and the “jocks” of the high school scene. They can also make all the difference
in perception and understanding, as discussed in the articles by Barner and
Stiller this week where related words “some” and “all” posed a challenge to
young children in inference tests (example: children might not grasp that the
sentence “There are some toys on the table” implies that not all toys are on
the table).
Barner asks, “Could
you a processing limit explain a failure to access alternatives?” This is an interesting
thought, because through readings and discussions in my other class, Minds and
Machines (an introduction to symbolic systems), I have formed this idea that children
are mini miracles of learning, just absorbing everything like a sponge and
being generally limitless. For those trying to replicate human intelligence in
machines, the way children learn is a source of inspiration. Alan Turing
proposed, “Instead of trying to produce a programme to simulate the adult mind,
why not rather try to…simulate the child’s?” A child has the amazing ability to generalize
from a limited sample size, from seeing a couple of dogs to putting the “dog” label
on them to differentiating “dogs” from “cats.” There are researchers that try
to teach machines what children do naturally; one study asked a machine to
identify arches, requiring many, many examples. But Barner’s use of “processing
limit” brings the almighty child to the machine’s level! I wonder: when and how do
children start accessing alternatives?
Another
interesting thought comes from Stiller: “The rarer the feature is, the more
informative it is to note that the object has this feature.” This reminds me of
a past reading where the equilateral triangle was the prototypical triangle. In
order to make someone think “scalene triangle” instead of “triangle” one has to
draw a weird looking triangle with unequal sides. I’d like to learn more about
this phenomenon as well as how it links to Gricean principles.
Interesting post, I especially enjoyed your stance on how we should model "smart" AI after children as per Alan Turing's quote. I think that actually many AI models today are taking inspiration from children's brains, and many professors do make the analogy of a computer vision neural network to a child just learning how to see and collecting data every microsecond of its life.
ReplyDeleteAs for your prototypical triangle comparison, I think that it's possible that the scalene triangles or any other irregularly shaped triangle is rarer than equilateral triangles, but it is also possible that the human mind tends to be more idealistic, imagining things and processing things are more "equal" then they really are in reality.
If either of you are interested in this particular aspect of machines mimicking the ability to learn from few (or even 0) training examples, there are many recent developments in deep learning that are tackling this exact issue. One approach is called 'transfer learning' -- it turns out that if you want to recognize objects in images, similar features are relevant across datasets. That is, if you want to classify pictures of dogs and cats, you'll likely need to start with the same basic properties of the image that you might be concerned about while classifying pictures of human faces: edges, distribution of colors, etc. Thus, you can train a model on pictures of human faces, then with just a few additional training examples, 'transfer' the knowledge from this model to a model for classifying pictures of dogs and cats. You could imagine a similar approach with Natural Language Understanding: there may be common features of sentence structure (e.g. scalar orderings) that are generalizable such that machines would only need a few examples of a new scale set in order to generalize previous knowledge of how implicatures work.
ReplyDeleteAnother, more 'direct' means of learning from few examples is called 'One Shot Learning'. There are many approaches, and the state of the art is far from perfect, but it's still a good thing to keep an eye on. This is obviously a very valuable pursuit within the field, for if we can rid ourselves of the constraint of having massive labeled datasets from training, the types of models we could develop will grow at unprecedented rates.