SpelingKorecksionASurveyOfTechniquesfromPastto.ppt

This is my definition, and is meant to be as general as possible. I don’t say what a “mistake means” Here is an outline of the classical spelling correction algorithm. The first step is to check if the word is a mistake - also called spell checking; (these are the red squiggles in Word) Once we decide a word needs to be corrected, a set of candidates is constructed. By candidates we mean any word that the system is considering internally. The set could Be as big as a whole dictionary, or just a few dozen words, or less. These candidates are then Ranked and sorted, and the top few are presented to the user. We call these suggestions. This is a logical outline, and in practice systems often combine steps together. Before we go any further, I just want to mention the issue of the user interface. Here are screenshots of popular spelling correction systems that we’re probably all familiar with. Microsoft Word automatically spell checks as you type, and you can ask for suggestions by right-clicking. (Actually, for simple errors it automatically corrects as you type. For instance, a couple slides back it didn’t want to let me type the mistake ‘the’ ) In Unix environments, editors like Emacs require you to start a spell checking process that presents a text menu of suggestions for each mistake. I put up this slide to acknowledge how important the user interface is to a final product. For instance, some research indicates that users prefer a system that is not as accurate but is less obtrusive and easier to use. But in terms of this presentation we’re just concerned with the task of coming up with a ranked list of suggestions, and particularly that the first suggestion is right as often as possible. Interestingly, you don’t necessarily need a dictionary to find mistakes. One method is just to look for unusual letter sequences. For example, the mistake “standard” has a lot of strange sequences like “ndr” and “drd”. The idea is that if you have a large piece of text to