MeCab English Documentation

This is based on the MeCab documentation.

What is MeCab?

MeCab is an open source morphological analysis engine developed as a joint research project between Kyoto University Information Research department and the Nippon Telegraph and Telecommunications Communication Science Laboratories. It is built with the goal of general purpose analysis and does not depend on any particular language corpus/dictionary.

MeCab uses Conditional Random Fields (CRF) parameter estimation, improving upon the Hidden Markov Models as used by ChaSen. MeCab also typically performs faster than ChaSen, Juman, and KAKASI. Incidentally, mekabu is the author's favourite dish.

(Translator's note: MeCab in Japanese is pronounced 'mekabu' - which is the thick part of wakame seaweed just above the root)

Features

• Generic design that does not depend on a dictionary corpus
• Based on Conditional Random Fields (CRF) for high precision analysis
• Faster than ChaSen and KAKASI libraries
• Dictionary search algorithm uses high speed TRIE structure double-array
• Library is re-entrant
• Bindings for various scripting languages (Perl, Ruby, Python, Java, C#)

Comparison

(TODO)

Mailing List

Changelog

(TODO)

Downloads

See project page for up to date downloads

Installation

Unix

Requirements

• C++ compiler (compiles with g++ 3.4.3 and VC7)
• iconv (libiconv): used for dictionary format conversion

MeCab Installation

Installation is the same as typical free software.

% tar zxfv mecab-X.X.tar.gz
% cd mecab-X.X
% ./configure
% make
% make check
% su
# make install

Dictionary installation

% tar zxfv mecab-ipadic-2.7.0-XXXX.tar.gz
% mecab-ipadic-2.7.0-XXXX
% ./configure
% make
% su
# make install

Windows

For binary installation, please use the self-extracting installer. Do the same for dictionary installation.

Usage

Getting Started

See below MeCab accepting input from stdin upon startup. MeCab assumes one line per sentence.

% mecab
縺吶ｂ繧ゅｂ繧ゅｂ繧ゅｂ繧ゅ?縺?■
縺吶ｂ繧? 蜷崎ｩ?荳?闊ｬ,*,*,*,*,縺吶ｂ繧?繧ｹ繝｢繝｢,繧ｹ繝｢繝｢
繧?     蜉ｩ隧?菫ょ勧隧?*,*,*,*,繧?繝｢,繝｢
繧ゅｂ    蜷崎ｩ?荳?闊ｬ,*,*,*,*,繧ゅｂ,繝｢繝｢,繝｢繝｢
繧?     蜉ｩ隧?菫ょ勧隧?*,*,*,*,繧?繝｢,繝｢
繧ゅｂ    蜷崎ｩ?荳?闊ｬ,*,*,*,*,繧ゅｂ,繝｢繝｢,繝｢繝｢
縺ｮ      蜉ｩ隧?騾｣菴灘喧,*,*,*,*,縺ｮ,繝?繝?縺?■    蜷崎ｩ?髱櫁?遶?蜑ｯ隧槫庄閭ｽ,*,*,*,縺?■,繧ｦ繝?繧ｦ繝?EOS

The output format significantly differs from ChaSen, becoming:

陦ｨ螻､蠖｢\t蜩∬ｩ?蜩∬ｩ樒ｴｰ蛻?｡?,蜩∬ｩ樒ｴｰ蛻?｡?,蜩∬ｩ樒ｴｰ蛻?｡?,豢ｻ逕ｨ蠖｢,豢ｻ逕ｨ蝙?蜴溷ｽ｢,隱ｭ縺ｿ,逋ｺ髻ｳ

or in English:

Original Form\t
Part of Speech,
Part of Speech section 1,
Part of Speech section 2,
Part of Speech section 3,
Conjugated form,
Inflection,
Reading,
Pronounciation

If a file is passed in as the argument, it becomes the analysis target. It is also possible to direct output to a file using the -o option.

% mecab INPUT -o OUTPUT

Division

Use the -O option as below

% mecab -O wakati
螟ｪ驛弱?縺薙?譛ｬ繧剃ｺ碁ヮ繧定ｦ九◆螂ｳ諤ｧ縺ｫ貂｡縺励◆縲?螟ｪ驛?縺ｯ 縺薙? 譛ｬ 繧?莠碁ヮ 繧?隕?縺?螂ｳ諤ｧ 縺ｫ 貂｡縺?縺?縲?

Change output format

Use the -O option as below

% mecab -Oyomi (Assign readings)
% mecab -Ochasen (ChaSen compatible)
% mecab -Odump (Full information dump)

These output formats are stored in /usr/local/lib/mecab/ipadic/dicrc. The user can also create custom definitions. Please take a look at the Output Formats documentation (TODO).

Advanced Usage

Changing the character code

(TODO)

% tar zxfv mecab-ipadic-2.7.0-xxxx
% cd mecab-ipadic-2.7.0-xxxx
% ./configure --with-charset=sjis
% make

% tar zxfv mecab-ipadic-2.7.0-xxxx
% ./configure --with-charset=utf8
% make

(TODO)

% cd mecab-ipadic-2.7.0-xxxx
% /usr/local/libexec/mecab/mecab-dict-index -f euc-jp -t utf-8
# make install

UTF-8 only mode

(TODO)

Unknown Word Estimation

(TODO)

繝帙Μ繧ｨ繝｢繝ｳ蟶?繝帙Μ繧ｨ繝｢繝ｳ      蜷崎ｩ?蝗ｺ譛牙錐隧?蝨ｰ蝓?荳?闊ｬ,*,*,*
蟶?     蜷崎ｩ?謗･蟆ｾ,蝨ｰ蝓?*,*,*,蟶?繧ｷ,繧ｷ
EOS
繝帙Μ繧ｨ繝｢繝ｳ縺輔ｓ
繝帙Μ繧ｨ繝｢繝ｳ      蜷崎ｩ?蝗ｺ譛牙錐隧?莠ｺ蜷?荳?闊ｬ,*,*,*
縺輔ｓ    蜷崎ｩ?謗･蟆ｾ,莠ｺ蜷?*,*,*,縺輔ｓ,繧ｵ繝ｳ,繧ｵ繝ｳ

(TODO)

%mecab --unk-feature "譛ｪ遏･隱?quot; 
繝帙Μ繧ｨ繝｢繝ｳ縺輔ｓ
繝帙Μ繧ｨ繝｢繝ｳ      譛ｪ遏･隱?縺輔ｓ    蜷崎ｩ?謗･蟆ｾ,莠ｺ蜷?*,*,*,縺輔ｓ,繧ｵ繝ｳ,繧ｵ繝ｳ

N-Best Solution Output

(TODO)

% mecab -N2
莉頑律繧ゅ＠縺ｪ縺?→縺ｭ縲?莉頑律    蜷崎ｩ?蜑ｯ隧槫庄閭ｽ,*,*,*,*,莉頑律,繧ｭ繝ｧ繧ｦ,繧ｭ繝ｧ繝ｼ
繧?     蜉ｩ隧?菫ょ勧隧?*,*,*,*,繧?繝｢,繝｢
縺?     蜍戊ｩ?閾ｪ遶?*,*,繧ｵ螟峨?繧ｹ繝ｫ,譛ｪ辟ｶ蠖｢,縺吶ｋ,繧ｷ,繧ｷ
縺ｪ縺?   蜉ｩ蜍戊ｩ?*,*,*,迚ｹ谿翫?繝翫う,蝓ｺ譛ｬ蠖｢,縺ｪ縺?繝翫う,繝翫う
縺ｨ      蜉ｩ隧?謗･邯壼勧隧?*,*,*,*,縺ｨ,繝?繝?縺ｭ      蜉ｩ隧?邨ょ勧隧?*,*,*,*,縺ｭ,繝?繝?縲?     險伜捷,蜿･轤ｹ,*,*,*,*,縲?縲?縲?EOS
莉頑律    蜷崎ｩ?蜑ｯ隧槫庄閭ｽ,*,*,*,*,莉頑律,繧ｭ繝ｧ繧ｦ,繧ｭ繝ｧ繝ｼ
繧ゅ＠    蜑ｯ隧?荳?闊ｬ,*,*,*,*,繧ゅ＠,繝｢繧ｷ,繝｢繧ｷ
縺ｪ縺?   蠖｢螳ｹ隧?閾ｪ遶?*,*,蠖｢螳ｹ隧槭?繧｢繧ｦ繧ｪ谿ｵ,蝓ｺ譛ｬ蠖｢,縺ｪ縺?繝翫う,繝翫う
縺ｨ      蜉ｩ隧?謗･邯壼勧隧?*,*,*,*,縺ｨ,繝?繝?縺ｭ      蜉ｩ隧?邨ょ勧隧?*,*,*,*,縺ｭ,繝?繝?縲?     險伜捷,蜿･轤ｹ,*,*,*,*,縲?縲?縲?EOS

Acknowledgements

Jorge Nocedal for making the FORTRAN implementation of L-BFGS open to the public.

http://www.ece.northwestern.edu/~nocedal/lbfgs.html

J. Nocedal. Updating Quasi-Newton Matrices with Limited Storage (1980), Mathematics of Computation 35, pp. 773-782. D.C. Liu and J. Nocedal. On the Limited Memory Method for Large Scale Optimization (1989), Mathematical Programming B, 45, 3, pp. 503-528.

繧√°縺ｶ