.NET中文分词.docVIP

?  HYPERLINK /malimalihun/article/details/6054030 .NET中文分词 分类：? HYPERLINK /malimalihun/article/category/772400 搜索引擎2010-12-03 23:10?148人阅读? HYPERLINK /malimalihun/article/details/6054030 \l comments 评论(0)? HYPERLINK javascript:void(0); \o 收藏 收藏? HYPERLINK /malimalihun/article/details/6054030 \l report \o 举报 举报 ?这两天因为需要就研究了.net环境下的中文分词，发现在目前的最高2.3.1L版本下中文分词效果不好，他自带了一些分词器，均没有什么效果，出来单词切分，连JCK二分法都不提供，更谈不上基于词典的分词了。 ??????? 下面我发俩段代码，前一段是我通过写一个二分法类来实现双字切分，后一段是通过调用肖波的分词器，自己又写了一个.net环境接口来实现基于词典的分词，以供大家学习之用…… 代码段一 [c-sharp]? HYPERLINK /malimalihun/article/details/6054030 \o view plain view plain HYPERLINK /malimalihun/article/details/6054030 \o copy copy using?System;?? using?System.Collections.Generic;?? using?System.Linq;?? using?System.Text;?? using?Lucene.Net.Analysis.Standard;?? using?Lucene.Net.Analysis;?? using?System.IO;?? using?System.Net;?? using?System.Collections;?? ?? namespace?erfenfa?? ?? {?? ?? ???private?class?filter?? ?? ???{?? ?? ???????public???static?void?main(string?[?]arg)?? ????????{?? ?? ???????????string?str=我们都是好孩子，即使你不同意，但又能如何，是不？;?? ????????????TextReader?tr?=?new?StringReader(str?);?? ????????????CJKAnalyzer?wa?=?new?CJKAnalyzer();?? ????????????TokenStream?ts?=?wa.TokenStream(*,tr);?? ????????????Token?t?=?null;?? ????????????while?((t?=?ts.Next())?!=?null)?? ????????????{?? ???????????????????messagebox.show(t.TermText().ToString());//实现循环输出?? ?? ????????????}?? ????????????? ????????????? ????????}?? ?? ???}?? ?? ?public?class?CJKAnalyzer:Analyzer??? ????{?? ?? ???????????public??static?string[]?STOP_WORDS?=?{?? ????????????????????????????????????????????????????a,?and,?are,?as,?at,?be,?? ????????????????????????????????????????????????????but,?by,?for,?if,?in,?? ????????????????????????????????????????????????????into,?is,?it,?no,?not,?? ?????????????????????????????????????????????????????of,?on,?or,?s,?such,?t,?? ???????????????????????????????????????????????????that,?the,?their,?then,?? ??????????????????????????????