It may be possible to represent this situation with a
constant error cost by distinguishing sub-classes. For
example, instead of two classes, “sick” and “healthy”,
there could be three classes, “sick-and-young”, “sick-and-
elderly”, and “healthy”. This is an imperfect solution
when the cost varies continuously, rather than discretely.
2.2.2 ERROR COST CONDITIONAL ON TIME OF
CLASSIFICATION
In a time-series application, the cost of a classification
error may depend on the timing. Consider a classifier that
monitors sensors that measure a complex system, such as
a manufacturing process or a medical device. Suppose
that the classifier is intended to signal an alarm if a
problem has occurred or will soon occur. The sensor
readings must be classified as either “alarm” or “no-
alarm”. The cost of the classification depends on whether
the classification is correct and also on the timeliness of
the classification. The alarm is not useful unless there is
sufficient time for an adequate response to the alarm
(Fawcett and Provost, 1996, 1997, 1999).
Again, it may be possible to represent this situation with a
constant error cost by distinguishing sub-classes. Instead
of two classes, “alarm” and “no-alarm”, there could be
“alarm-with-lots-of-time”, “alarm-with-a-little-time”,
“alarm-with-no-time”, and “no-alarm”. Again, this is an
imperfect solution when the cost varies continuously as a
function of the timeliness of the alarm.
2.2.3 ERROR COST CONDITIONAL ON CLASSIFICATION OF
OTHER CASES
In some applications, the cost of making a classification
error with one case may depend on whether errors have
been made with other cases. The familiar precision and
recall measures, widely used in the information retrieval
literature, may be seen as cost measures of this type (van
Rijsbergen, 1979). For example, consider an information
retrieval task, where we are searching for a document on a
certain topic. Suppose that we would be happy if we
could find even one document on this topic. If we are
given a collection of documents to classify as “relevant”
or “not-relevant” for the given topic, then the cost of
mistakenly assigning a relevant document to the not-
relevant class depends on whether there are any other
relevant documents that we have correctly classified.
As another example, in activity monitoring, if you issue
an alarm twice in succession for the same problem, the
benefit of the second alarm is less than the benefit of the
first alarm, assuming both alarms are correct
classifications (Fawcett and Provost, 1999). This is
related to Section 2.2.2.
2.2.4 ERROR COST CONDITIONAL ON FEATURE VALUE
The cost of making a classification error with a particular
case may depend on the value of one or more features of
the case.
3. Cost of Tests
Each test (i.e., attribute, measurement, feature) may have
an associated cost. For example, in medical diagnosis, a
blood test has a cost.
Turney (1995a) points out that we can only rationally
determine whether it is worthwhile to pay the cost of a
test when we know the cost of misclassification errors. If
the cost of misclassification errors is much greater than
the cost of tests, then it is rational to purchase all tests that
seem to have some predictive value. If the cost of
misclassification errors is much less than the cost of tests,
then it is not rational to purchase any tests.
3.1 Constant Test Cost
The cost of performing a certain test may be a constant.
Each test has a different cost, but the cost of a given test is
the same for all cases (Nunez, 1988, 1991; Tan, 1991a,
1991b, 1993).
3.2 Conditional Test Cost
The cost of performing a certain test may be conditional
on the circumstances surrounding the test.
3.2.1 TEST COST CONDITIONAL ON PRIOR TEST
SELECTION
The cost of performing a certain test on a given patient
may be conditional on the previous tests that have been
chosen for the patient. For example, a group of blood tests
ordered together may be cheaper than the sum of the costs
of each test considered by itself, since the tests share
common costs, such as the cost of collecting blood from
the patient (Turney, 1995a).
3.2.2 TEST COST CONDITIONAL ON PRIOR TEST RESULTS
The cost of performing a certain test on a patient may be
conditional on the results of previous tests. For example,
the cost of a blood test is conditional on the patient's age.
Thus a blood test must be preceded by a “patient-age”
test, which determines the cost of the blood test.
3.2.3 TEST COST CONDITIONAL ON TRUE CLASS OF CASE
The cost of performing a certain test on a patient may be
conditional on the correct diagnosis of the patient. For
example, the cost of an exercise stress test on a patient
may be conditional on whether the patient has heart
disease. The stress test could cause heart failure, which
adds to the total cost of the test.
3.2.4 TEST COST CONDITIONAL ON TEST SIDE-EFFECTS
The cost of performing a certain test on a patient may be
conditional on possible side-effects of the test. For
example, some patients are allergic to the dies that are
used in certain radiological procedures. One side-effect of
such a radiological test is an allergic reaction, which may
increase the cost of the test.