WeSearch_LexicalFiltering

Background

Given a lattice of lexical hypotheses, _aL_bⁱ, and a tagger t^j(_aL_bⁱ), develop a filter function, filter(_aL_bⁱ, t^j(_aLbⁱ)), -> (0,1) that discards unlikely lexical hypotheses.

TNT output for filtering of LE types

One such filter function is to map PTB tags output from the TNT tagger onto LE Types:

• t^j(_aL_bⁱ) : map_t(t^j(_aL_b)) = map_l(_aL_bⁱ) => 1

  • map_t() ∩ map_l() ≠ ∬

Mappings may be derived intuitively from inspection of the TNTLEConfusionMatrix.

An alternative approach is to programatically find mappings based on the desired outcomes of lexical filtering.

Related Work

• Rebecca Driden (2009), [http://www.dridan.com/research/papers/dridan-phdthesis.pdf Using Lexical Statistics to Improve HPSG Parsing], PhD Thesis, Saarland University