An Exercise Using Certainty Factors

A certainty factor expresses how accurate, truthful, or reliable you judge a statement. It is your judgement of the evidence.

It goes without saying that people have been making judgements from the beginning. The key to McAllister's certainty factor formulation is that it enables a computer to calculate a combination of certainty factors. Already, people use computers extensively to analyze probabilities and `what if' situations. I expect even more, as I discuss in Black, White, and Gray.

In McAllister's methodology, people express their judgements with phrases such as `suggestive', `strongly suggestive', or `weakly suggestive' and computers use numbers.

Here is an example of certainty factors that is both topical and controversial. (One reason I picked this exercise that there are differences among people in how they judge one statement or another.)

First, I express the certainty factors for several statements; then, I pull them together in a computer calculated combination. Also, I briefly mention probability analysis, which I think is better for some questions. Finally, I will contend that the notion is especially suited to floor vacuuming robots, and the like, and there is where we will see the most use of the concept.

Several Certainty Factors

In the past, my initial statement was highly controversial but it is now, I think, only mildly controversial.

The US policy of the past 40 years in the Middle East has failed the US.

In years past, the certainty factors that were applied to this statement differed greatly. The claim was highly controversial.

Left-leaning liberals and Democrats considered the statement truthful. They told me that US policy was a failure and that the failure would come to haunt the country. However, others told me that they thought the policy helped the US, such as its support for the Saudi government. Now, I think the current Bush Administration holds the same view as many Liberals and Democrats did in the past.

My judgement of the accuracy and truth of the statement is that it is `strongly suggestive'.

And as for whether current Bush Administration also believes the statement: I think that, too, is `strongly suggestive'.

Here is a controversial statement:

With the best of efforts, the US will need a generation to reduce sharply the likelihood of a few people acting against the US.

One person told me that a single generation is too few, that a change would take many generations. Others have suggested that a change could come in less than a month, most likely from the election of a different President in the US.

In my judgement, the statement is `suggestive'. The vast majority of people around the world may well change their opinions quickly. But too few will act to inform US favoring police to stop those who have dedicated their lives to recreating a Caliphate by, among other actions, defeating the US.

However, I am not sure whether this statement should be analyzed using certainty factors or probabilities. On the one hand, anthropologists talk about and marketing men invest in value changes. On the other hand, it is an assertion about the future. Further down I will talk more about probabilities.

US spying is not very successful against its enemies in or from Moslem countries or inspired by people in or from there

Initially I thought there was little controversy about the preceding statement, but I discovered that some think that US spying is going well.

In my judgement, the validity of the statement is `suggestive'.

Now, a different statement, not about US spying itself, but about what the Bush Administration thinks:

The Bush Administration thinks that US spying is not very successful against its enemies in or from Moslem countries or inspired by people in or from there

In my judgement, the reliability of the claim is `suggestive'.

Americans consider people who attack Americans in the United States or abroad should be stopped.

I think the truth of this statement is `strongly suggestive'. Also, I think the Bush Administration believes this equally.

This is quite different from the statement that

It is not worth trying to stop attacks against the US because the hurt to the US from attacks will be less than the hurt from the US opposing them.

Some people think this is `strongly suggestive'. They combine this with a statement that they think is `suggestive', namely that

Americans can be persuaded that attacks should be ignored.

Since I think there is `suggestive' certainty for the opposite, that Americans cannot be so persuaded, I figure the first statement is not salient, as well as thinking that there is `weakly suggestive' certainty for the opposite.

Probability Analysis

Moreover, as a practical matter, I think this is a question for which probability analysis is a better tool than certainty factor analysis. The judgement is not so much about the accuracy, truth, or reliability of existing evidence, but about the likelihood and magnitude of something happening in the future.

How likely and how important is the chance or probability:

• that hurt comes from an attack comes from the long term consequences of US support of a theocratic regime?

How likely and how important is the chance or probability:

• that hurt comes from the long term consequences of the merging of US domestic databases?

US conservatives tell me they worry about the former: they claim that in November 2003 the US made a deal with Iranian theocrats to gain peace in the short run by favoring certain Shi'ite groups in Iraq. (This claim requires a certainty factor judgement; I think the evidence it occurred is `highly suggestive'. At that time, Shi'ite military groups did not engage in guerilla warfare against the US. In addition, I think there is `suggestive' evidence that the US government began to renege on this agreement in the early part of 2004. Grand Ayatollah Ali al-Sistani asked that the US stage elections before creating institutions to protect Shi'ite enemies from the Shi'ites.)

The conservatives fear that an agreement between the US and Iranian theocrats, if kept by both sides, will mean that the US will not be able to create institutions to protect the losers in a war from the vengeance of the winners, and that as a consequence, there will be no fundamental change and the US will be in danger over the long run.

US liberals and privacy advocates worry about the latter; they fear that merged databases will make it easy for crooks to bribe, blackmail, or bamboozle police or other government officials. They fear that crooks will use such information to rob or otherwise hurt individuals or companies. And they fear that crooked people in government will misuse their positions.

Three More Certainty Factors

Here are three more statements. I do not think any are particularly controversial:

The current US government believes that its covert anti-terrorist actions are incompetent. A few people from Moslem countries wish to weaken the US. Advances in technology have enabled the militarily weak to kill many people at lower direct cost than in the past.

In my opinion, the evidence for the first is `suggestive' and for the second and third is `strongly suggestive'.

(The attacks of 11 September 2001 directly cost Al Qaeda $500,000.00 and the lives 19 operatives. This is less expensive directly to the attacker than the cost of an attack causing the same damage in World War II.)

Combining Certainty Factors

The key to David McAllister's method is that certainty factors can be expressed as numbers and combined by computer.

Here is a table of the relations:

	strongly or highly suggestive	0.8
	suggestive	0.6
	weakly suggestive	0.4
	slight hint	0.2

The rule for adding two positive certainty factors is to add one certainty factor with the other, the other having been reduced by an amount that depends on the size of the first:

    CFcombine (CFa CFb) = CFa + CFb(1 - Cfa)

[ What is the mathematical term for a quantity calculated by subtracting the original from one? I cannot remember. ]

The rule for adding two negative certainties is simple: combine the two factors as if they were positive and negate the result:

    CFcombine (CFc CFd) = -(CFcombine (-CFc -CFd))

The rule for adding positive and negative certainty factors is more complex:

    CFcombine (CFe CFf) =  (CFe + CFf) / (1 - min{|CFe|, |CFn|})

Let us combine statements. I am going to express them according to how I think the Bush Administration thinks:

1. The US Bush Administration think that the US policy of the past 40 years in the Middle East has failed the US.

`strongly suggestive' = 0.8

2. With the best of efforts, the US will need a generation to reduce sharply the likelihood of a few people acting against the US.

`suggestive' = 0.6

    CFcombine (CFa CFb) = CFa + CFb(1 - Cfa)

                       = 0.8 + 0.6(1 - 0.8)

                       = 0.92

The combination tells us that I think it is highly certain that the Bush administration thinks that previous US policy has failed and that it will take a long time to make it a success for the US.

Now let us add four certainty factors. We do this two by two. Note that we can make the calculation in any order; the result comes out the same:

3. The Bush Administration thinks that US spying is not very successful against its enemies in or from Moslem countries or inspired by people in or from there

`suggestive' = 0.6

4. The current US government believes that its covert anti-terrorist actions are incompetent.

`suggestive' = 0.6

5. A few people from Moslem countries wish to weaken the US.

`strongly suggestive' = 0.8

6. Advances in technology have enabled the militarily weak to kill many people at lower direct cost than in the past.

`strongly suggestive' = 0.8

I did the calculations using Emacs Lisp. Here I combine statements 3 and 4, statements 5 and 6, and their results:

(+ 0.6 (* 0.6 (- 1 0.6)))
   which is 0.84

(+ 0.8 (* 0.8 (- 1 0.8)))
   which is 0.96

(+ 0.84 (* 0.96 (- 1 0.84)))
   which is 0.9936

Here I combine statements 3 and 5, statements 4 and 6, and their results:

(+ 0.6 (* 0.8 (- 1 0.6)))
   which is 0.92

(+ 0.6 (* 0.8 (- 1 0.6))) 0.92
   which is 0.92

(+ 0.92 (* 0.92 (- 1 0.92)))
   which is 0.9936

This suggests that the Bush Administration strongly believes that:

The US is incompetent both at spying and at covert operations, that technological advance has made the US vulnerable to a new kind of war, and that a few people wish to attack the US.

You may have come to the same conclusion without needing the calculations. The exercise is simple.

A Floor Vacuuming Robot:
machine-made observations and human evaluation

My point is that over time, more observations will be made by machine. The observations can be classified into statements with associated certainty factors. The statements will not be about politics, at least not soon, but they could well be of the sort:

0.8 certainty, the object on the floor looks elliptical;
0.4 certainty, it looks tarnished like an old silver coin;
0.5 certainty, its largest angular diameter
indicates that for its distance,
the object is the size of a US dime.

Combine the first two certainty factors:
 (+ 0.8 (* 0.4 (- 1 0.8)))
   which is 0.88

Combine that result with the third:
 (+ 0.88 (* 0.5 (- 1 0.88)))
   which is 0.94

A floor vacuuming robot could use sensors that are programmed to translate noisy and poor quality observations into statements such as `looks elliptical with weakly suggestive reliability', and then combine several such statements into results on which it can act. In this case, the programming would lead the robot to decide (with a strong certainty factor of 0.94) that the object it detected is an old silver coin that should be picked up rather than vacuumed up.

Yes, you could program the robot to make the same decisions using fuzzy logic and probabilities. My contention is that for this kind of action, humans will consider certainty factors more understandable than probabilities. It will be easier for humans to figure out what the robot will do.

An expert will not need the `crutch' of certainty factors; the expert will understand fuzzy logic and probabilities. But the expert is not the key here. From the point of view of a company selling a floor vacuuming robot, most sales will be to the majority who purchase robots without bothering to check how many coins they vacuum.

The key is to persuade the latter, the majority of buyers, that the purchase will be safe and do its job. The key persuaders will be a late portion of `early adopters'. These people will not be experts. They will not harm their own cause by causing others to think `it is good for them, but not for me'. Instead, if the task is not too time consuming, and fairly comprehensible, these `early adopters' will investigate how the robot works, either by putting coins and the robot in the same room, or by reading the robot's rule set, or both.

This is how an `early adopter' will convince himself that the robot is not too stupid, and rather than vacuum the floor himself, he will let the robot do it. And his example will persuade others.