stantially with five or ten times more examples, but it is
time consuming and expensive to acquire hand-labeled data.
Another issue with noun-modifier classification is the
choice of classification scheme for the semantic relations.
The 30 classes of Nastase and Szpakowicz [2003] might not
be the best scheme. Other researchers have proposed differ-
ent schemes [Rosario and Hearst, 2001]. It seems likely that
some schemes are easier for machine learning than others.
8 Conclusion
This paper has introduced a new method for calculating
relational similarity, Latent Relational Analysis. The ex-
periments demonstrate that LRA performs better than the
VSM approach, when evaluated with SAT word analogy
questions and with the task of classifying noun-modifier
expressions. The VSM approach represents the relation be-
tween a pair of words with a vector, in which the elements
are based on the frequencies of 64 hand-built patterns in a
large corpus. LRA extends this approach in three ways: (1)
the patterns are generated dynamically from the corpus, (2)
SVD is used to smooth the data, and (3) a thesaurus is used
to explore reformulations of the word pairs.
Just as attributional similarity measures have proven to
have many practical uses, we expect that relational similar-
ity measures will soon become widely used. Relational
similarity plays a fundamental role in the mind and therefore
relational similarity measures could be crucial for artificial
intelligence [Medin et al., 1990]. LRA may be a step to-
wards the black box that we imagined in Section 2, with
many potential applications in text processing.
In future work, we plan to investigate some potential ap-
plications for LRA. It is possible that the error rate of LRA
is still too high for practical applications, but the fact that
LRA matches average human performance on SAT analogy
questions is encouraging.
Acknowledgments
For data and software used in this research, my thanks go to
Michael Littman, Vivi Nastase, Stan Szpakowicz, Egidio
Terra, Charlie Clarke, Dekang Lin, Doug Rohde, and Mi-
chael Berry.
References
[Banerjee and Pedersen, 2003] S. Banerjee and T. Pedersen.
Extended gloss overlaps as a measure of semantic relat-
edness. In Proceedings of the Eighteenth International
Joint Conference on Artificial Intelligence (IJCAI-03),
pages 805-810, Acapulco, Mexico, 2003.
[Barzilay and McKeown, 2001] R. Barzilay and K. McKe-
own. Extracting paraphrases from a parallel corpus. In
Proceedings of the 39th Annual Meeting of the Associa-
tion for Computational Linguistics, pages 50-57, Tou-
louse, France, 2001.
[Berry, 1992] M.W. Berry. Large scale singular value com-
putations. International Journal of Supercomputer Ap-
plications, 6(1): 13-49, 1992.
[Clarke et al., 1998] C.L.A. Clarke, G.V. Cormack, and
C.R. Palmer. An overview of MultiText. ACM SIGIR
Forum, 32(2): 14-15, 1998.
[Dagan and Glickman, 2004] I. Dagan and O. Glickman,
Probabilistic textual entailment: Generic applied model-
ing of language variability. In Learning Methods for Text
Understanding and Mining, Grenoble, France, 2004.
[Golub and Van Loan, 1996] G.H. Golub and C.F. Van
Loan. Matrix Computations. Third edition. Johns Hop-
kins University Press, Baltimore, MD, 1996.
[Landauer and Dumais, 1997] T.K. Landauer and S.T. Du-
mais. A solution to Plato’s problem: The latent semantic
analysis theory of the acquisition, induction, and repre-
sentation of knowledge. Psychological Review, 104:211-
240, 1997.
[Lesk, 1986] M.E. Lesk. Automatic sense disambiguation
using machine readable dictionaries: How to tell a pine
cone from a ice cream cone. In Proceedings of ACM
SIGDOC ’86, pages 24-26, 1986.
[Lewis, 1991] D.D. Lewis. Evaluating text categorization.
In Proceedings of the Speech and Natural Language
Workshop, Asilomar, pages 312-318, 1991.
[Lin, 1998] D. Lin. Automatic retrieval and clustering of
similar words. In Proceedings of the 36th Annual Meet-
ing of the Association for Computational Linguistics and
the 17th International Conference on Computational Lin-
guistics (COLING-ACL ‘98), pages 768-774, Montreal,
Canada, 1998.
[Medin et al., 1990] D.L. Medin, R.L. Goldstone, and D.
Gentner. Similarity involving attributes and relations:
Judgments of similarity and difference are not inverses.
Psychological Science, 1(1): 64-69, 1990.
[Nastase and Szpakowicz, 2003] V. Nastase and S. Szpako-
wicz. Exploring noun-modifier semantic relations. In
Fifth International Workshop on Computational Seman-
tics (IWCS-5), Tilburg, The Netherlands, pages 285-301,
2003.
[Rosario and Hearst, 2001] B. Rosario and M. Hearst. Clas-
sifying the semantic relations in noun-compounds via a
domain-specific lexical hierarchy. In Proceedings of the
2001 Conference on Empirical Methods in Natural Lan-
guage Processing (EMNLP-01), pages 82-90, 2001.
[Salton and McGill, 1983] G. Salton and M.J. McGill. In-
troduction to Modern Information Retrieval. McGraw-
Hill, New York, 1983.
[Turney et al., 2003] P.D. Turney, M.L. Littman, J. Bigham,
and V. Shnayder. Combining independent modules to
solve multiple-choice synonym and analogy problems.
In Proceedings of the International Conference on Re-
cent Advances in Natural Language Processing
(RANLP-03), Borovets, Bulgaria, pages 482-489, 2003.
[Turney and Littman, 2005] P.D. Turney and M.L. Littman.
Corpus-based learning of analogies and semantic rela-
tions. Machine Learning, in press, 2005.