scws icon indicating copy to clipboard operation
scws copied to clipboard

Published 20 hours ago • verified publisher icon hightman
→

Metadata

开源免费的简易中文分词系统，PHP分词的上乘之选！

README of SCWS

$Id$

SCWS 绠€浠?/h2>

SCWS 鏄?Simple Chinese Word Segmentation 鐨勯瀛楁瘝缂╁啓锛堝嵆锛氱畝鏄撲腑鏂囧垎璇嶇郴缁燂級銆?杩欐槸涓€濂楀熀浜庤瘝棰戣瘝鍏哥殑鏈烘寮忎腑鏂囧垎璇嶅紩鎿庯紝瀹冭兘灏嗕竴鏁存鐨勪腑鏂囨枃鏈熀鏈纭湴鍒囧垎鎴愯瘝銆傝瘝鏄?涓枃鐨勬渶灏忚绱犲崟浣嶏紝浣嗗湪涔﹀啓鏃跺苟涓嶅儚鑻辫浼氬湪璇嶄箣闂寸敤绌烘牸鍒嗗紑锛屾墍浠ュ浣曞噯纭苟蹇€熷垎璇嶄竴鐩?鏄腑鏂囧垎璇嶇殑鏀诲叧闅剧偣銆?/p>

SCWS 閲囩敤绾?C 璇█寮€鍙戯紝涓嶄緷璧栦换浣曞閮ㄥ簱鍑芥暟锛屽彲鐩存帴浣跨敤鍔ㄦ€侀摼鎺ュ簱宓屽叆搴旂敤绋嬪簭锛屾敮鎸佺殑 涓枃缂栫爜鍖呮嫭 GBK銆?code>UTF-8 绛夈€傛澶栬繕鎻愪緵浜?PHP 鎵╁睍妯″潡锛屽彲鍦?PHP 涓揩閫?鑰屾柟渚垮湴浣跨敤鍒嗚瘝鍔熻兘銆?/p>

鍒嗚瘝绠楁硶涓婂苟鏃犲お澶氬垱鏂版垚鍒嗭紝閲囩敤鐨勬槸鑷繁閲囬泦鐨勮瘝棰戣瘝鍏革紝骞惰緟浠ヤ竴瀹氱殑涓撴湁鍚嶇О锛屼汉鍚嶏紝鍦板悕锛?鏁板瓧骞翠唬绛夎鍒欒瘑鍒潵杈惧埌鍩烘湰鍒嗚瘝锛岀粡灏忚寖鍥存祴璇曞噯纭巼鍦?90% ~ 95% 涔嬮棿锛屽熀鏈笂鑳芥弧瓒充竴浜?灏忓瀷鎼滅储寮曟搸銆佸叧閿瓧鎻愬彇绛夊満鍚堣繍鐢ㄣ€傞娆￠洀褰㈢増鏈彂甯冧簬 2005 骞村簳銆?/p>

SCWS 鐢?hightman 寮€鍙戯紝骞朵互 BSD 璁稿彲鍗忚寮€婧愬彂甯?锛屽弬瑙?COPYING銆?/p>

PHP 鍒嗚瘝婕旂ず

瀹夎

鐩墠 scws 鍙彁渚涙簮鐮佸寘锛屽拰澶у鏁?GNU 杞欢鐨勭紪璇戝畨瑁呮楠や竴鏍枫€備笅闈㈡垜浠互 Linux 绯荤粺涓轰緥锛?瑕佹眰瑁呮湁 gcc 缂栬瘧鍣ㄣ€?code>Windows 涓婂垯寤鸿鐢?cygwin 鎴?mingw 鐜缂栬瘧銆?/p>

  1. 鍙栧緱 scws 婧愮爜骞惰В鍘嬪埌褰撳墠鐩綍

    wget -q -O - http://www.xunsearch.com/scws/down/scws-1.2.1.tar.bz2 | tar xjf -

  2. 杩涘叆婧愮爜鐩綍閰嶇疆鍜岀紪璇?/p> 

    cd scws-1.2.1 ; ./configure ; make install

    娉ㄦ剰锛氳繖鏄互榛樿鏂瑰紡瀹夎鍒?/usr/local 鐩綍銆傝缁嗛厤缃弬鏁版墽琛?./configure --help 鏌ョ湅銆?/p>

PHP 鎵╁睍

鍦?UNIX 绫荤殑绯荤粺涓婏紝璇风洿鎺ヤ粠婧愮爜鏂瑰紡缂栬瘧瀹夎 PHP 鎵╁睍锛涘浜?windows 绯荤粺锛岃鐩存帴涓嬭浇鎴戜滑缂栬瘧濂界殑 php_scws.dll 鍗冲彲銆?/p>

鍋囧畾鎮ㄥ凡缁忓皢 scws 瀹夎鍒?$prefix 鐩綍锛岃繕瑕佹眰绯荤粺鐜瑁呮湁 autoconf automake 鍙?phpize 宸ュ叿銆?/p>

  1. 杩涘叆婧愮爜鐩綍鐨?phpext/ 骞舵墽琛?phpize (浣嶄簬 php 瀹夎鐩綍涓嬬殑 bin 鐩綍閲?

  2. 鎵ц configure 杩涜閰嶇疆鍜屽熀纭€妫€娴嬶紝鍋囧畾鎮ㄧ殑 php 瀹夎鍦?$php_prefix 鐩綍

    ./configure --with-scws=/usr/local --with-php-config=$php_prefix/bin/php-config

  3. 鍏堢紪璇戠劧鍚庝互 root 韬唤瀹夎

    make
sudo make install

  4. 鍦?php.ini 涓姞鍏ヤ互涓嬪唴瀹?/p> 

    [scws]
; 娉ㄦ剰璇锋鏌?php.ini 涓殑 extension_dir 鐨勮瀹氬€兼槸鍚︽纭? 鍚﹀垯璇峰皢 extension_dir 璁句负绌猴紝
; 鍐嶆妸 extension = scws.so 鎴?php_scws.dll 鎸囧畾缁濆璺緞銆?extension = scws.so
scws.default.charset = gbk
scws.default.fpath = $prefix/etc

  5. 鍛戒护琛屼笅鎵ц php -m 灏辫兘鐪嬪埌 scws 浜嗭紝phpinfo() 涔熶細鍖呭惈鍏充簬 scws 鐨勪俊鎭€?/p>

  6. windows 涓嬬殑 php 鎵╁睍鏄噰鐢?dll 搴撴枃浠躲€傛垜浠凡棰勭紪璇戣嫢骞蹭釜鐗堟湰锛岄噰鐢?zip 鍘嬬缉浣嶄簬 phpext/win32/php_scws-1.2.1-win32.zip锛岃鐩存帴瑙ｅ帇鍚庨€夋嫨鐩稿簲鐗堟湰鐨?php_scws.dll銆?灏嗗畠澶嶅埗鍒?PHP 鐨勬墿灞曠洰褰曚腑锛堥€氬父鏄?extensions 鎴?ext 鐩綍锛夛紝鐒跺悗鍙傝€冧笂闈㈢殑鏂瑰紡淇敼 php.ini銆?/p>

  7. 鍏充簬 PHP 鎵╁睍鐨勭ず渚嬬敤娉曞強寮€鍙戝弬鑰冿紝璇峰弬瑙?phpext/README.md銆?/p>

閰嶅宸ュ叿鐢ㄦ硶

  1. $prefix/bin/scws 杩欐槸鍒嗚瘝鐨勫懡浠よ宸ュ叿锛屾墽琛?scws -h 鍙互鐪嬪埌璇︾粏甯姪璇存槑銆?/p> 

    Usage: scws [options] [[-i] input] [[-o] output]
    • -i string|file 瑕佸垏鍒嗙殑瀛楃涓叉垨鏂囦欢锛屽涓嶆寚瀹氬垯绋嬪簭鑷姩璇诲彇鏍囧噯杈撳叆锛屾瘡杈撳叆涓€琛屾墽琛屼竴娆″垎璇?/li>
    • -o file 鍒囧垎缁撴灉杈撳嚭淇濆瓨鐨勬枃浠惰矾寰勶紝鑻ヤ笉鎸囧畾鐩存帴杈撳嚭鍒板睆骞?/li>
    • -c charset 鎸囧畾鍒嗚瘝鐨勫瓧绗﹂泦锛岄粯璁ゆ槸 gbk锛屽彲閫?utf8
    • -r file 鎸囧畾瑙勫垯闆嗘枃浠讹紙瑙勫垯闆嗙敤浜庢暟璇嶃€佹暟瀛椼€佷笓鏈夊悕瀛椼€佷汉鍚嶇殑璇嗗埆锛?/li>
    • -d file[:file2[:...]] 鎸囧畾璇嶅吀鏂囦欢璺緞锛圶DB鏍煎紡锛岃鍦?-c 涔嬪悗浣跨敤锛?
      鑷?1.1.0 璧凤紝鏀寔澶氳瘝鍏稿悓鏃惰浇鍏ワ紝涔熸敮鎸佺函鏂囨湰璇嶅吀锛堝繀椤绘槸.txt缁撳熬锛夛紝澶氳瘝鍏歌矾寰勪箣闂寸敤鍐掑彿(:)闅斿紑锛?鎺掑湪瓒婂悗闈㈢殑璇嶅吀浼樺厛绾ц秺楂樸€?
鏂囨湰璇嶅吀鐨勬暟鎹牸寮忓弬瑙?scws-gen-dict 鎵€鐢ㄧ殑鏍煎紡锛屼絾鏇村鏉句竴浜涳紝鍏佽鐢ㄤ笉瀹氶噺鐨勭┖鏍煎垎寮€锛屽彧鏈?lt;璇?gt;鏄繀澶囬」鐩紝
鍏跺畠鏁版嵁鍙湁鍙棤锛屽綋璇嶆€ф爣娉ㄤ负鈥?鈥濓紙鍙瑰彿锛夋椂琛ㄧず璇ヨ瘝浣滃簾锛屽嵆浣垮湪杈冧綆浼樺厛绾х殑璇嶅簱涓瓨鍦ㄨ璇嶄篃灏嗕綔搴熴€?
    • -M level 澶嶅悎鍒嗚瘝鐨勭骇鍒細1~15锛屾寜浣嶅紓鎴栫殑 1|2|4|8 渚濇琛ㄧず 鐭瘝|浜屽厓|涓昏瀛梶鍏ㄩ儴瀛楋紝缂虹渷涓嶅鍚堝垎璇嶃€?/li>
    • -I 杈撳嚭缁撴灉蹇界暐璺宠繃鎵€鏈夌殑鏍囩偣绗﹀彿
    • -A 鏄剧ず璇嶆€?/li>
    • -E 灏?xdb 璇嶅吀璇诲叆鍐呭瓨 xtree 缁撴瀯 (濡傛灉鍒囧垎鐨勬枃浠跺緢澶ф墠闇€瑕?
    • -N 涓嶆樉绀哄垏鍒嗘椂闂村拰鎻愮ず
    • -D debug 妯″紡 (寰堝皯鐢紝闇€瑕佺紪璇戞椂鎵撳紑 --enable-debug)
    • -U 灏嗛棽鏁ｅ崟瀛楄嚜鍔ㄨ皟鐢ㄤ簩鍒嗘硶缁撳悎
    • -t num 鍙栧緱鍓?num 涓珮棰戣瘝
    • -a [~]attr1[,attr2[,...]] 鍙樉绀烘煇浜涜瘝鎬х殑璇嶏紝鍔爚琛ㄧず杩囨护璇ヨ瘝鎬х殑璇嶏紝澶氫釜璇嶆€т箣闂寸敤閫楀彿鍒嗛殧
    • -v 鏌ョ湅鐗堟湰

  2. $prefix/bin/scws-gen-dict 璇嶅吀杞崲宸ュ叿

    Usage: scws-gen-dict [options] [-i] dict.txt [-o] dict.xdb
    • -c charset 鎸囧畾瀛楃闆嗭紝榛樿涓?gbk锛屽彲閫?utf8
    • -i file 鏂囨湰鏂囦欢(txt)锛岄粯璁や负 dict.txt
    • -o file 杈撳嚭 xdb 鏂囦欢鐨勮矾寰勶紝榛樿涓?dict.xdb
    • -p num 鎸囧畾 XDB 缁撴瀯 HASH 璐ㄦ暟锛堥€氬父涓嶉渶瑕侊級
    • -U 鍙嶅悜瑙ｅ帇锛屽皢杈撳叆鐨?xdb 鏂囦欢杞崲涓?txt 鏍煎紡杈撳嚭 锛圱ODO锛?/li>

    鏂囨湰璇嶅吀鏍煎紡涓烘瘡琛屼竴涓瘝锛屽悇琛岀敱 4 涓瓧娈电粍鎴愶紝瀛楁涔嬮棿鐢ㄨ嫢骞蹭釜绌烘牸鎴栧埗琛ㄧ(\t)鍒嗛殧銆?鍚箟锛堝叾涓彧鏈?<璇?gt; 鏄繀椤绘彁渚涚殑锛夛紝# 寮€澶寸殑琛岃涓烘敞閲婂拷鐣ヤ笉璁★細

    #<璇?gt;  <璇嶉(TF)>  <璇嶉噸(IDF)>  <璇嶆€?鍖楀ぇ鏍囨敞)>
鏂拌瘝鏉?12.0        2.2          n

libscws API

杩欐槸鏁村悎 scws 鍒板叾瀹冨簲鍜岀▼搴忕殑鎺ュ彛璇存槑锛岃瑙?API銆?/p>

rules.ini 瑙勫垯闆?/h2>

锛堟殏缂猴級

鍏充簬 XDB 璇嶅吀

鎴戜滑鐨勮瘝鍏镐娇鐢ㄧ殑鏄嚜琛屽紑鍙戠殑涓撶敤 XDB 鏍煎紡锛屽厤璐规彁渚涚殑璇嶅吀鏄€氱敤鐨勪簰鑱旂綉淇℃伅璇嶆眹闆嗭紝 鏀跺綍浜嗗ぇ绾?28 涓囦釜璇嶃€?/p>

濡傛灉鎮ㄩ渶瑕佸畾鍒惰瘝鍏镐互浣滅壒娈婄敤閫旓紝璇蜂笌鎴戜滑鑱旂郴锛屽彲鑳戒細瑙嗘儏鍐佃繘琛屾敹璐广€?/p>

鎬ц兘鎸囨爣

鍦?FreeBSD 6.2 绯荤粺锛屽崟鏍稿崟 CPU 鑷冲己 3.0G 鐨勬湇鍔″櫒涓婏紝娴嬭瘯闀垮害涓?80,535 鐨勬枃鏈€?鐢ㄩ檮甯︾殑鍛戒护琛屽伐鍏疯€楁椂灏嗙害 0.17 绉掞紝鑻ユ敼鐢?php 鎵╁睍鏂瑰紡璋冪敤锛屽垯鑰楁椂绾︿负 0.65 绉掋€?/p>

鍒嗚瘝绮惧害 95.60%锛屽彫鍥炵巼 90.51% (F-1: 0.93)

鍏跺畠

璇ユ枃妗ｇ敱 hightman 浜?2007/06/08 棣栨缂栧啓锛屽悓鏃跺湪涓嶆柇淇涓紒

椤圭洰涓婚〉锛?a href="http://www.xunsearch.com/scws">http://www.xunsearch.com/scws 鎴戠殑閭锛歨[email protected]

← Metadata

1.6k
Stars
354
Forks
Watchers

Owner

verified publisher icon hightman

Metadata

开源免费的简易中文分词系统，PHP分词的上乘之选！

Back