Re: [FWP] Natural language processing

[Bennett Todd <bet@rahul.net>]

2000-04-25-20:37:20 Jeff Boes:
> Has there been any natural langauge parser work done, that's
> publicly available?

That one I can't help with. I'd be interested if anyone else has
heard of any, though.

> Is Perl the sort of langauge that you'd use if you had to parse
> 
> "Where can I find out about cheap air fares?"
> vs.
> "What is the cheapest air fare I can get from Chicago to Orlando?"
> vs.
> "How do I get cheap airline tickets?"

That would depend on the nature of the algorithm. I'd tend to guess
(just a guess, mind) that perl would be great for the first step,
namely cleaning and tokenizing, as well as interfacing with the rest
of the system, both input and output (e.g. CGI work:-).

I'd also tend to guess that the actual job of trying to robustly
pick a good parse would involve a fiendishly compute-intensive
template-matching combined with some sort of reduction, trying to
decompose into smaller structures, with some sort of backtracking.
Or maybe parallel trials and some mechanism for picking the best
one. I wouldn't be surprised if lwall didn't know....

In any case, once the text has been tokenized, I'd expect the next
step to be the sort of thing you'd want to write in C, for
performance. Then graft it into perl with XS:-).

You might want to try subscribing to the perl-ai mailing list; I
believe the subscription page should be reachable at
<URL:http://netizen.com.au/mailman/listinfo/perl-ai>.

-Bennett

PGP signature