my question on the debate on whether language models require sensory grounding for understanding

Friday, March 24, 2023

This evening I was at the debate of the Philosophy of Deep Learning conference "Do large language models need sensory grounding for meaning and understanding?" organized by NYU and Columbia, held at the NYU Cantor Film Center in NYC. The recording of the meeting will be soon online.

The debate ended before I got to ask my question, so I thought of writing it here, which I hope will also help me to clarify it in my mind. The debate was especially interesting to me because many years ago, back in 2011, I wondered whether someone could think without sensory input. At that time, in my naivety, I didn't know that others were wondering about that way before I was born, nor I was expecting to go in a few years to a debate about that in NYC, organized by NYU and Columbia (:o).

While listening to the debate, a question bothered my mind:

How helpful or impeding is the fact that we are using our experience to answer the question of whether a system, different from ours, has a sense of understanding? Aren't human language models extensions of our minds?

We are using our experience and sense of understanding to look for it in the language model. For example, we use our experience of the color red to feed it into the system. And say, we get to a point where we are convinced that it understands language the way we do - with or without sensory grounding. Does this mean that the model understands our language or does it mean that we think that it does because it behaves in the way we do when we understand our language?

Is it possible to know the difference, and how?

Languages come in many forms. Let's assume that we are extinct and aliens find the same text data that we have used and will use to train our language models to the extent that we are convinced that they understand human language. Could they -who lack the same sensory experience and sense of understanding as us- tell whether the language models understand the language they are trained on?

Or to put it differently.. and more pragmatically, we are not the only species that use language. Other primates, such as marmosets, use language (see this presentation I gave in 2017 on monkey talk for more info). First, it is still unresolved the decoding of their language, and I'd be very interested to see progress on this front. But, can we decode it? Or else, can a large language model trained on marmoset language (auditory) data understand their language? Could we tell whether it has achieved it? Or only a marmoset can tell whether a language model trained on marmoset language data understands marmosets' language?

If we are bound to the way we understand language, how much of the understanding goes to the language model, and how much to us; are they different?

_______________

before closing, let me give a couple of remarks on the debate, in case someone finds them useful for future debates..

- I missed a clear definition of what understanding is and how we can tell whether a system has it. How can we answer the question of whether a large language model requires sensory grounding for understanding if we don't know what we are looking for and how?

- that applies in all talks: f you want to pass along a message, do not fill up your slides with lots of text that you are not reading out loud, or a lot of information that you are not addressing. It is not only pointless to give the talk if people cannot follow what you are saying, but also frustrating.

- it didn't feel like it was a debate, but rather like independent talks on a similar subject. To increase interaction between the speakers and create the sense of a debate it would maybe be helpful to give a guideline to the speakers before they prepare their slides. The guideline could be a list of questions that the speakers need to address so that there is interaction.

- it was nice that there was online streaming of the event, but it would have been even nicer if it were interactive, where the online audience could also ask a question (with priority to those in person).