Language and information

Lecture 3: Information

3.1. How sentences carry information

[Audio recording—Opens a separate, blank window. Return to this window to read the transcript.]

In this session, which deals with information, there will be several topics which we will try to cover. One is how words carry meaning, and how sentences carry information. Then, I want to touch on how an increment can be made in the informational power of this system. I want to give you an example of how an increment is made, in the case of language, to its informational power, to what it can carry. Then there are some general comments on the relation of form to content in language, more exactly in language structure. And finally, something about the structure of information as it appears in language.

Now, meaning and information are ordinarily not defined terms. In fact, neither is necessarily definable. Words generally are assumed to have meaning, sentences perhaps to have information. The meaning of words is considered perhaps to be an association between the word and things in the real world, but this is a particular kind of meaning, because meaning in general is something that is common to the great bulk of human events. We will try to see here something that can be said explicitly about the structure of information on the basis of what we have seen in both sessions of last week about the structure of language.

I first to say something about how sentences carry information. It was proposed in the first session of this series last week that sentences are obtainable from a few constraints on word combinations. Now, we can therefore ask the question about how sentences carry information by asking how these constraints carry information, because there we can say quite explicit things.

First of all, there are certainly some constraints which have no direct relation to information; for example, the phonemic composition of words, the spelling of words. There are some constraints which have a very particular indirect relation to information, namely, that they affect the access of the hearer—either the time, in respect of time, or the ease of access of the hearer to the information. An example of that is the reductions, which change the phonemic shape—the sound of sentences—and perhaps zero words, even though the words are still present, but zero their phonemic shape, and the person then has to reconstruct the presence of the word, because its meaning is still playing a role in the sentence, even after it has been zeroed. So that the time when words are known to the hearer, and the ease with which he can recognize them is affected by the reductions.

There are some constraints which affect the speaker's attitude to the information which he is giving—again, not the information itself. Such is, for instance, the fronting of certain words, the bringing of certain words to the front of the sentence to express the topic, the attitude of the speaker that this is what he is talking about; or the bringing of some sentences as interruptions into the inside of another sentence, which is part of the process of making the second sentence into a modifier of a word of the first sentence. This is also [a case in which] the substantive information is the same, but there is a certain change in the attitude to the information, to say that this second sentence modifies a word in the first.

There are also some constraints which create categories or relations of information without affecting the information itself, and such are the kind of required reductions that create paradigms in grammar, so that something is required and you have to speak of it in a certain way. The information is not different from the information given in whatever sources there are to the suffixes of the paradigm and so on, but the arrangement is different, and a certain amount of categorization of the information is obtained thereby, but not the information.

There are actually only two of the constraints that we surveyed last week which in a direct way carry the information of the sentence, and those two are the two [constraints] that create the base set, meaning the non-reduced, the unreduced set of sentences in the language. They are the partial ordering of words, which is the relation of operator to argument; and secondly the likelihood inequalities of words within the partial ordering, that is to say, the fact that each argument is more likely or less [likely] to appear under a particular operator than are other arguments of that operator. As we will see in a moment—but in any case it also follows from has been seen last week—these are the only two which create the actual substantive information of the sentence. This is part of the fact that one can show otherwise that the base sentences contain all the information, or virtually all the information—there are various problems—of the sentences of the language as a whole, and they have only these two constraints.

Now, so much to introduce the question of how sentences, or how words in sentences, carry information.