Beate Hampe: "When down is not bad, and up is not good enough" (2005)

This thoughtful paper by Beate Hampe (see also From Perception to Meaning, 2005) is a nice empirical counterweight to some of the wildly speculative claims that are thrown around in cognitive linguistics. In this particular case, the issue is whether there is an inherent and global good/bad valence to the dichotomies up/down, in/out, on/off, front/back, etc.

The Theory
This claim has apparently been made most clearly by the Polish linguistic Tomasz P. Krzeszowski. Krzeszowski himself sees the claim as a specific version of the "Invariance Principle," since it claims that up/down metaphors inherit the good/bad valencies that standing and lying down have in our preconceptual lives.

This is a neat little fairytale about the genesis of meaning, but as usual, the empirical data spoils everything. The first, obvious sign comes from looking at bad things going up:

• Unemployment is up. (bad)
• Employment is up. (good)

Faced with such examples, one would have to say something like this: up and down have an inherent emotional value, but this emotional value is not very strong itself; a strongly laden context can thus pull the words in an "unnatural" direction. However, on average or in neutral contexts, up will have a weak tendency to lean towards positive emotional valence, and down a tendency towards negative.

The Counterevidence
However, this doesn't seem to be the case. Hampe has done a medium-sized corpus study of the constructions finish off, finish up, slow down and (the rare) slow up. For each occurrence, she looked for valence clues in the immediate context and categorized the example according to how positive it was. The result was, surprisingly, that up was much more likely to be used for negative purposes than down or off.

Now, it is of course a little unfortunate that she only looked at two verbs, and that one of her particle pairs (down/off) were not antonyms. A safer strategy would be to pick two antonym verbs and two antonym particles and then combine them in a table like the following:

	in	out
give	28.0 kk	27.3 kk
take	53.5 kk	118.0 kk

The numbers in this table are the number of occurrences (in millions), estimated by Google searches for the exact phrases. This excludes, for instance, split uses like take me out (or in general verb + NP + particle). However, Hampe's study seems to have the same weakness.

Similar tables could be made for the following:

• come/go in/out
• come/leave in/out
• push/pull on/off
• break/make up/down
• break/fix up/down
• etc.

Other Sources of Evidence
Several empirical hypotheses could be tested for such data.

For instance, one test whether there is a statistical tendency for some positive particle (e.g., in) to attach to the positive set of verbs (come, make, stand, keep) compared to its negative counterpart (out). This could be done with respect to grammaticality or with respect to empirical counts. It would essentially amount to collapsing all of the tables into a single 2 × 2 table.

Another question would be whether the positive and negative contexts where distributed evenly across the two columns of any such table. In order to do so, one would have to develop a valence assessment method like Hampe's, preferably an automatic one. After having trained such a model, one could use Fisher's exact test on the contingency table consisting of positive/negative valence × positive/negative column.

The automatic valence assessment might perhaps be achieved through semi-supervised learning. We can imagine starting from a valence function v₀ defined on a small set of good and bad words:

• v₀(w) = 1 for w in some finite set G = {good, pleasant, improve, victory, truth, ...};
• v₀(w) = –1 for any w that is an antonym to a word in G;
•  v₀(w) = 0 for all other words w.

Assuming that good words co-occur with good words, this could be used to train a more fine-grained function, say, v₁₀₀₀.There are a number of problems with this assumption (it ignores rhetorical contrast effects as well as negation), but experiments would show whether it worked or not.

Srinivas Sankara Narayanan: "KARMA," an addition

I forgot to include this quote in the previous post, but it strikes me as illuminating:

[...] our focus on narrow set of expressions, our impoverished target domain and the inability of our system to extend existing metaphors are important shortcomings. (p. 217)

Srinivas Sankara Narayanan: "KARMA" (1997)

Srini Narayanan's PhD dissertation is often cited as an exemplary accomplishment of cognitive metaphor theory. However, when you read the small print, it seems less convincing. It has the distinct feel of the Georgetown-IBM system from 1953.

Since I have now made my way through the 300-page dissertation, I'll now do my best to give a clear and short presentation of its results. I'll do that in reverse order, starting with its evaluation methodology and moving towards a more detailed description of the program.

Evaluation
Narayanan's program was evaluated on a "corpus" consisting of 9 short snippets of text amounting to a total of 116 words. This evaluation set was, as far as I can understand, specifically constructed so as to look as much like the training set as possible in terms of vocabulary and theme.

For each sentence to be evaluated, the methodology consisted in the following steps:

1. For each sentence, a feature structure was constructed; thus, the sentence was parsed and partially interpreted by hand before entering the reasoning module.
2. Any "difficult" aspects of the sentences were smoothed out; e.g. the phrase perception of was removed from one of the examples to create a flatter semantics (p. 246).
3. Any knowledge that the system lacked in order to understand the sentence was fed into it; e.g., in one case it was given the information that NAFTA affects Mexico (p. 248).
4. The subject matter was, as far as I understand, always explicitly stated to be economic policy so that the system did not have to infer whether it should read motion verbs metaphorically or not.
5. A feature structure was produced by the system, and Narayanan read the contents of this structure and determined whether he thought it had fit his intuitions (as he mostly did).

It should be quite clear that this evaluation procedure is quite liberal. The line between system design and system evaluation is effectively invisible, as is the line between subject and experimenter.

Input knowledge
The system was equipped with a number of materials which were designed explicitly to deal with a training "corpus" of 20 sentences (roughly 750 words). These were:

• A background model of the domain of economic policies with some very strong and very simplified assumptions (pp. 173-75).
• A set of motion models or "execution schemas" that describe the temporal progression of actions like walking and recovering from illness in a highly idealized fashion.
• A set of highly specific maps that told the system how to link various objects, attributes, and events from the execution schemas to the economic objects, attributes, and events. These were often very specific and tailored to the corpus. For instance, there were mappings telling the system to an IncompetentDoctor object onto a Government object, and a step event onto an implement_policy event.

This background knowledge would obviously not be very robust if applied more widely.

The X-schema model
The target domain in Narayanan's thesis, economic policies, is represented fundamentally different from the source domains, movement and health. The target domain is represented as a Bayesian net in order to allow probabilistic reasoning. The source domain is represented as a kind of highly idiosyncratic simulation tool.

These simulation tools are what he calls "x-schemas" or "execution schemas." They are abstract representations of the causal structure of processes like walking or grasping something.

The x-schemas are connceted to a feature structure that keeps track of the state of whoever the agent performing the action is. The agent's state may influence the structure of the process and vice versa. The x-schemas are mostly structured like a little story with a beginning, a middle, and an end, and they are run as simulations when the system processes a sentence.

This simulation is a discrete-step process where the value of various parameters at one time (step size, energy level, distance to goal, etc.) combine to produce the parameter settings in the next time step. All parameter values are non-negative integers. The mechanism that produces the state at time t + 1 from the state at time t is designed by hand (and by intuition).

Examples
Narayanan gives a single completely explicit example of the kind of data structures that the system inputs and outputs. His example builds on the newspaper headline Indian Government Stumbling in Implementing Liberalization Plan.

He manually translates this sentences into the following feature structure representation (p. 208):

( (context Economic Policy)

  (type Liberalization)

  (actor IG)

  (event Stumble)
  (aspect Progressive)

  (ut-type Description) )

Fed this data structure, the programs runs a simulation of a walking event and maps the results to the domain of economic policies using maps like ACTOR IS MOVER. The output is the following (p. 215):

( (context Economic Policy)

  (type Liberalization)

  (actor IG)
  (event Stumble)

  (aspect Progressive)

  (status (ongoing 0 1)
  (suspend 1 1))

  (difficulty

    (t 0 .7)

    (t 1 1))

  (outcome (fail 1 .6))
  (goal

    (free-trade 0 1)

    (dereg 0 1)

    (free-trade 1 1)

    (dereg 1 1)))

It is not entirely clear to me what the first argument means in constructions like (dereg 0 1), and the text doesn't seem to help me out.

Narayanan has previously (pp. 55-58) gone through an example simulation of the WALK schema with a set of explicit values on various parameters such as Dist(walked) and Energy. I will not reproduce the tables here, but the schema can roughly be compared to an loop working according to the following rules:

• If walking is enabled, get ready
• If you're ready, check if you can see
• If you can see, take a step
• If you can't see, check your footing
• If your footing is OK, take a step
• If your footing is not OK, change position
• If you've just taken a step and you're not a your destination, get ready

I should be quite obvious that this is a highly idealized model. Whoever, its experiential implausibility does not constitute a problem in Narayanan's dissertation, probably because of the small size of the validation materials.