Date of Original Version

9-2012

Type

Article

Journal Title

Computational Linguistics

Volume

38

Issue

3

First Page

479

Last Page

526

Rights Management

© 2012 Association for Computational Linguistics The version of record is available online at http://dx.doi.org/10.1162/COLI_a_00092

Abstract or Description

Probabilistic grammars are generative statistical models that are useful for compositional and sequential structures. They are used ubiquitously in computational linguistics. We present a framework, reminiscent of structural risk minimization, for empirical risk minimization of probabilistic grammars using the log-loss. We derive sample complexity bounds in this framework that apply both to the supervised setting and the unsupervised setting. By making assumptions about the underlying distribution that are appropriate for natural language scenarios, we are able to derive distribution-dependent sample complexity bounds for probabilistic grammars. We also give simple algorithms for carrying out empirical risk minimization using this framework in both the supervised and unsupervised settings. In the unsupervised case, we show that the problem of minimizing empirical risk is NP-hard. We therefore suggest an approximate algorithm, similar to expectation-maximization, to minimize the empirical risk.

DOI

10.1162/COLI_a_00092

Share

COinS
 

Published In

Computational Linguistics, 38, 3, 479-526.