TwitterKoreanProcessorCS icon indicating copy to clipboard operation
TwitterKoreanProcessorCS copied to clipboard

Published 20 hours ago • verified publisher icon modamoda

Update to twitter-korean-text 4.4

Open raon0211 opened this issue 9 years ago • 5 comments

I have tried to make the dll from twitter-korean-text 4.4, but I had no idea how to do it. Can you share me the way I can make it to a C# library?

It is said that version 4.4 is the end of adding features, So I think upgrading the version to 4.4 will be a nice thing to do. (Actually, custom dictionary feature is definitely needed)

Thanks very much.

raon0211 avatar Feb 11 '16 17:02 raon0211

I did it like below:

$ mono ~/Downloads/ikvm-7.2.4630.5/bin/ikvmc.exe -target:library korean-text.jar
IKVM.NET Compiler version 7.2.4630.5
Copyright (C) 2002-2012 Jeroen Frijters
http://www.ikvm.net/

note IKVMC0002: Output file is "korean-text.dll"
$ ls -al
-rwxr-xr-x  1 Joon  staff  9091072 Feb 12 15:46 korean-text.dll

Thanks for notifying me update. I'll follow up.

modamoda avatar Feb 12 '16 23:02 modamoda

Actuailly I tried ikvmc in Windows but some NoClassDefFoundError gets thrown.

raon0211 avatar Feb 13 '16 00:02 raon0211

Since I'm not familiar with Java, Can you check if my procedure is right?

  1. mvn compile & mvn package and generate jar
  2. ikvmc -target:library korean-twitter-text.jar
  3. And there comes a bunch of NoClassDefFoundError.

raon0211 avatar Feb 13 '16 00:02 raon0211

A piece of error is below:

warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.Hangul$.composeHangul(CCC)C"
    ("scala.Predef$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.Hangul$.composeHangul(Lcom.twitter.penguin.korean.util.Hangul$HangulChar;)C"
    ("com.twitter.penguin.korean.util.Hangul$HangulChar")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.Hangul$.<init>()V"
    ("scala.collection.immutable.List$")
warning IKVMC0100: Class "com.twitter.penguin.korean.util.Hangul$HangulChar$" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.Hangul$DoubleCoda$" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanConjugation$$anonfun$com$twitter$penguin$korean$util$KoreanConjugation$$addPreEomi$1" not found
warning IKVMC0100: Class "scala.collection.Set" not found
warning IKVMC0100: Class "scala.runtime.ObjectRef" not found
warning IKVMC0100: Class "scala.runtime.VolatileByteRef" not found
warning IKVMC0100: Class "scala.collection.generic.Subtractable" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanConjugation$$anonfun$expanded$lzycompute$1$1" not found
warning IKVMC0100: Class "scala.runtime.BoxedUnit" not found
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanConjugation$.com$twitter$penguin$korean$util$KoreanConjugation$$addPreEomi(CLscala.collection.Seq;)Lscala.collection.Seq;"
    ("com.twitter.penguin.korean.util.KoreanConjugation$$anonfun$com$twitter$penguin$korean$util$KoreanConjugation$$addPreEomi$1")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanConjugation$.conjugatePredicatesToCharArraySet(Lscala.collection.immutable.Set;Z)Lcom.twitter.penguin.korean.util.CharArraySet;"
    ("scala.collection.JavaConversions$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanConjugation$.conjugatePredicated(Lscala.collection.immutable.Set;Z)Lscala.collection.immutable.Set;"
    ("scala.runtime.ObjectRef")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanConjugation$.expanded$lzycompute$1(Lscala.collection.immutable.Set;ZLscala.runtime.ObjectRef;Lscala.runtime.VolatileByteRef;)Lscala.collection.immutable.Set;"
    ("scala.runtime.VolatileByteRef")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanConjugation$.expanded$1(Lscala.collection.immutable.Set;ZLscala.runtime.ObjectRef;Lscala.runtime.VolatileByteRef;)Lscala.collection.immutable.Set;"
    ("scala.runtime.VolatileByteRef")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanConjugation$.<init>()V"
    ("scala.collection.Seq$")
warning IKVMC0100: Class "scala.Symbol$" not found
warning IKVMC0100: Class "scala.Symbol" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$typoDictionaryByLength$1" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$readStreamByLine$1" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$readStreamByLine$2" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$readStreamByLine$3" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$readWordFreqs$1" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$readWordMap$1" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$readWordMap$2" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$readWordsAsSet$1" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$readWords$1" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$getConjugationMap$1$1" not found
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.koreanEntityFreq$lzycompute()Lscala.collection.mutable.Map;"
    ("scala.runtime.BoxedUnit")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.spamNouns$lzycompute()Lcom.twitter.penguin.korean.util.CharArraySet;"
    ("scala.Predef$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.nameDictionary$lzycompute()Lscala.collection.immutable.Map;"
    ("scala.Predef$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.typoDictionaryByLength$lzycompute()Lscala.collection.immutable.Map;"
    ("com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$typoDictionaryByLength$1")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.predicateStems$lzycompute()Lscala.collection.immutable.Map;"
    ("scala.Predef$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.readStreamByLine(Ljava.io.InputStream;Ljava.lang.String;)Lscala.collection.Iterator;"
    ("scala.Predef$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.readWordFreqs(Ljava.lang.String;)Lscala.collection.mutable.Map;"
    ("scala.collection.JavaConversions$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.readWordMap(Ljava.lang.String;)Lscala.collection.immutable.Map;"
    ("com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$readWordMap$1")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.readWordsAsSeq(Ljava.lang.String;)Lscala.collection.Seq;"
    ("scala.collection.Iterator")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.readWordsAsSet(Lscala.collection.Seq;)Lscala.collection.immutable.Set;"
    ("scala.Predef$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.readWords(Lscala.collection.Seq;)Lcom.twitter.penguin.korean.util.CharArraySet;"
    ("com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$readWords$1")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.addWordsToDictionary(Lscala.Enumeration$Value;Lscala.collection.Seq;)V"
    ("scala.collection.mutable.Map")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.getConjugationMap$1(Lscala.collection.immutable.Set;Z)Lscala.collection.immutable.Map;"
    ("com.twitter.penguin.korean.util.KoreanDictionaryProvider$$anonfun$getConjugationMap$1$1")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.<init>()V"
    ("scala.collection.JavaConversions$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanDictionaryProvider$.<clinit>()V"
    ("scala.Symbol$")
warning IKVMC0100: Class "scala.Enumeration$ValueSet$" not found
warning IKVMC0100: Class "scala.Enumeration$ValueOrdering$" not found
warning IKVMC0100: Class "scala.Enumeration$ValueSet" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanPos$KoreanPosTrie$" not found
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Predicates()Lscala.collection.immutable.Set;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.selfNode()Lcom.twitter.penguin.korean.util.KoreanPos$KoreanPosTrie;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.shortCut()Lscala.collection.immutable.Map;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.OtherPoses()Lscala.collection.immutable.Set;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.ProperNoun()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Others()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Space()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.CashTag()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.URL()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Email()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.ScreenName()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Hashtag()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Punctuation()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Alpha()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.KoreanParticle()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Number()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Foreign()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Korean()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Unknown()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Suffix()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.VerbPrefix()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.NounPrefix()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Conjunction()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.PreEomi()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Eomi()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Josa()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Exclamation()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Determiner()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Adverb()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Adjective()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Verb()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.Noun()Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.ValueSet()Lscala.Enumeration$ValueSet$;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.ValueOrdering()Lscala.Enumeration$ValueOrdering$;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.withName(Ljava.lang.String;)Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.apply(I)Lscala.Enumeration$Value;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.maxId()I"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.values()Lscala.Enumeration$ValueSet;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanPos.toString()Ljava.lang.String;"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanSubstantive$$anonfun$isKoreanNumber$1" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanSubstantive$$anonfun$3" not found
warning IKVMC0100: Class "scala.collection.immutable.IndexedSeq" not found
warning IKVMC0100: Class "scala.collection.immutable.IndexedSeq$" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanSubstantive$$anonfun$4" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanSubstantive$$anonfun$isKoreanNameVariation$1" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanSubstantive$$anonfun$5" not found
warning IKVMC0100: Class "scala.collection.immutable.Seq" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanSubstantive$$anonfun$1" not found
warning IKVMC0100: Class "com.twitter.penguin.korean.util.KoreanSubstantive$$anonfun$2" not found
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanSubstantive$.isJosaAttachable(CC)Z"
    ("scala.runtime.BoxesRunTime")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanSubstantive$.isName(Ljava.lang.CharSequence;)Z"
    ("scala.collection.immutable.Map")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanSubstantive$.isKoreanNumber(Ljava.lang.CharSequence;)Z"
    ("scala.runtime.RichInt$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanSubstantive$.isKoreanNameVariation(Ljava.lang.CharSequence;)Z"
    ("com.twitter.penguin.korean.util.KoreanPos$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanSubstantive$.collapseNouns(Lscala.collection.Seq;)Lscala.collection.Seq;"
    ("scala.Tuple2")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanSubstantive$.<init>()V"
    ("scala.Predef$")
warning IKVMC0111: Emitted java.lang.NoClassDefFoundError in "com.twitter.penguin.korean.util.KoreanSubstantive$.<clinit>()V"
    ("scala.Symbol$")

raon0211 avatar Feb 13 '16 00:02 raon0211

Thanks for reporting, let me spend some time to figure out. I'll try it in another environment.

modamoda avatar Feb 15 '16 03:02 modamoda