Srinivas Sankara Narayanan: "KARMA" (1997)

Srini Narayanan's PhD dissertation is often cited as an exemplary accomplishment of cognitive metaphor theory. However, when you read the small print, it seems less convincing. It has the distinct feel of the Georgetown-IBM system from 1953.

Since I have now made my way through the 300-page dissertation, I'll now do my best to give a clear and short presentation of its results. I'll do that in reverse order, starting with its evaluation methodology and moving towards a more detailed description of the program.

Evaluation
Narayanan's program was evaluated on a "corpus" consisting of 9 short snippets of text amounting to a total of 116 words. This evaluation set was, as far as I can understand, specifically constructed so as to look as much like the training set as possible in terms of vocabulary and theme.

For each sentence to be evaluated, the methodology consisted in the following steps:

1. For each sentence, a feature structure was constructed; thus, the sentence was parsed and partially interpreted by hand before entering the reasoning module.
2. Any "difficult" aspects of the sentences were smoothed out; e.g. the phrase perception of was removed from one of the examples to create a flatter semantics (p. 246).
3. Any knowledge that the system lacked in order to understand the sentence was fed into it; e.g., in one case it was given the information that NAFTA affects Mexico (p. 248).
4. The subject matter was, as far as I understand, always explicitly stated to be economic policy so that the system did not have to infer whether it should read motion verbs metaphorically or not.
5. A feature structure was produced by the system, and Narayanan read the contents of this structure and determined whether he thought it had fit his intuitions (as he mostly did).

It should be quite clear that this evaluation procedure is quite liberal. The line between system design and system evaluation is effectively invisible, as is the line between subject and experimenter.

Input knowledge
The system was equipped with a number of materials which were designed explicitly to deal with a training "corpus" of 20 sentences (roughly 750 words). These were:

• A background model of the domain of economic policies with some very strong and very simplified assumptions (pp. 173-75).
• A set of motion models or "execution schemas" that describe the temporal progression of actions like walking and recovering from illness in a highly idealized fashion.
• A set of highly specific maps that told the system how to link various objects, attributes, and events from the execution schemas to the economic objects, attributes, and events. These were often very specific and tailored to the corpus. For instance, there were mappings telling the system to an IncompetentDoctor object onto a Government object, and a step event onto an implement_policy event.

This background knowledge would obviously not be very robust if applied more widely.

The X-schema model
The target domain in Narayanan's thesis, economic policies, is represented fundamentally different from the source domains, movement and health. The target domain is represented as a Bayesian net in order to allow probabilistic reasoning. The source domain is represented as a kind of highly idiosyncratic simulation tool.

These simulation tools are what he calls "x-schemas" or "execution schemas." They are abstract representations of the causal structure of processes like walking or grasping something.

The x-schemas are connceted to a feature structure that keeps track of the state of whoever the agent performing the action is. The agent's state may influence the structure of the process and vice versa. The x-schemas are mostly structured like a little story with a beginning, a middle, and an end, and they are run as simulations when the system processes a sentence.

This simulation is a discrete-step process where the value of various parameters at one time (step size, energy level, distance to goal, etc.) combine to produce the parameter settings in the next time step. All parameter values are non-negative integers. The mechanism that produces the state at time t + 1 from the state at time t is designed by hand (and by intuition).

Examples
Narayanan gives a single completely explicit example of the kind of data structures that the system inputs and outputs. His example builds on the newspaper headline Indian Government Stumbling in Implementing Liberalization Plan.

He manually translates this sentences into the following feature structure representation (p. 208):

( (context Economic Policy)

  (type Liberalization)

  (actor IG)

  (event Stumble)
  (aspect Progressive)

  (ut-type Description) )

Fed this data structure, the programs runs a simulation of a walking event and maps the results to the domain of economic policies using maps like ACTOR IS MOVER. The output is the following (p. 215):

( (context Economic Policy)

  (type Liberalization)

  (actor IG)
  (event Stumble)

  (aspect Progressive)

  (status (ongoing 0 1)
  (suspend 1 1))

  (difficulty

    (t 0 .7)

    (t 1 1))

  (outcome (fail 1 .6))
  (goal

    (free-trade 0 1)

    (dereg 0 1)

    (free-trade 1 1)

    (dereg 1 1)))

It is not entirely clear to me what the first argument means in constructions like (dereg 0 1), and the text doesn't seem to help me out.

Narayanan has previously (pp. 55-58) gone through an example simulation of the WALK schema with a set of explicit values on various parameters such as Dist(walked) and Energy. I will not reproduce the tables here, but the schema can roughly be compared to an loop working according to the following rules:

• If walking is enabled, get ready
• If you're ready, check if you can see
• If you can see, take a step
• If you can't see, check your footing
• If your footing is OK, take a step
• If your footing is not OK, change position
• If you've just taken a step and you're not a your destination, get ready

I should be quite obvious that this is a highly idealized model. Whoever, its experiential implausibility does not constitute a problem in Narayanan's dissertation, probably because of the small size of the validation materials.