One aspect of taking part in the Good Judgment Project that I've found surprising is the difficulty of deciding whether a scenario has occurred even when the facts are known. The GJP uses a variety of methods to extract judgments about the probability of scenarios from participants. The following question - now resolved as 'yes' - is typical:

"Will national military force(s) from one or more countries intervene in Syria before 1 December 2014?"

The GJP provides clarification for certain common terms, in this case including the following definitions:

"An intervention, in this case, will be considered the use of national military force(s) to oppose armed groups in Syria (e.g., airstrikes, the deployment of troops). The provision of support services alone will not be sufficient (e.g., sending military advisers, providing surveillance, or transferring military equipment). For the purposes of this question, the countries represented must acknowledge the intervention, though they do not have to use this term specifically. Whether the intervention is done with the assent of the Syrian government is irrelevant. Inadvertent shelling or border crossing would not count. Outcome will be determined by credible open source media reporting (e.g., Reuters, BBC, AP)."

"National military forces refer to the ground, sea, and/or air components of a country’s official armed forces (e.g., the United States military). This generally excludes members of a country’s intelligence service as well as paramilitary groups, including insurgents, mercenaries, etc. As needed, in resolution criteria, GJP may specify that the definition includes additional entities, such as law enforcement forces."

Even 'before' merits clarification:

"'Before' should be interpreted to mean at or prior to the end (23:59:59 ET) of the previous day. For example, 'before 10 October' means any time up to 23:59:59 ET on 9 Oct. "

Not long after the question was posted, someone noticed an ambiguity: what about CIA drone strikes? A legalistic approach might rule them out, but then this would seem to go against the spirit of the question. Perhaps the clarifications were wrong?

This sort of discussion is very common in the GJP, and exposes a very important feature of intelligence - and indeed policy - work, namely that there is a vast amount of uncertainty and ambiguity about what we mean by concepts such as 'war', 'conflict', 'occupy', 'threat', 'intent', 'objective' to an extent that sometimes bleeds through into our picture of what is actually happening.

The logical positivists of the 1920s and 30s tried to draw a distinction between disagreements about language and definitions (they supposed that this was the role of philosophy), and disagreements about facts (this was the role of science). Unfortunately, for deeper logical reasons, this distinction is itself an ambiguous one, as Quine discusses in his seminal essay 'Two Dogmas of Empiricism'. There is, it seems, an irreducible grey area between disagreements about language, and those about facts.

This is an everyday problem for analysts in the workplace. We can help matters, as the GJP does, by attempting to provide standard definitions for common terms. But we need to learn to live with the edge cases, and consider definitional ambiguity as yet another source of uncertainty.