lingua-go
lingua-go copied to clipboard
pemistahl
The most accurate natural language detection library for Go, suitable for short text and mixed-language text
Currently, for a given input string, only the most likely language is returned. However, if the input contains contiguous sections of multiple languages, it will be desirable to detect all...
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 3 to 4. Release notes Sourced from codecov/codecov-action's releases. v4.0.0 v4 of the Codecov Action uses the CLI as the underlying upload. The CLI has helped to...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Hi @pemistahl, I'm happy to add a new language to this amazing project. I've carefully followed the SOP for adding a new language, and everything looks good. However, I've noticed...
Bumps google.golang.org/protobuf from 1.31.0 to 1.32.0. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...
Hi, In my scenario, the goal is to detect whether the input text is in English or another language. I'm not sure how to utilize the library to accomplish this...
Currently, the language models are loaded into simple maps at runtime. Even though accessing the maps is pretty fast, they consume a significant amount of memory. The goal is to...
Add more classification metrics such as `Precision`, `Recall`, `Specificity` and `F1`.
**To reproduce:** ``` package main import ( "github.com/pemistahl/lingua-go" "fmt" ) func main() { detector := lingua.NewLanguageDetectorBuilder(). FromAllLanguages(). Build() text := "上海大学是一个好大学. わー!" if language, exists := detector.DetectLanguageOf(text); exists { fmt.Println(language.String())...
Bumps [github.com/shopspring/decimal](https://github.com/shopspring/decimal) from 1.3.1 to 1.4.0. Release notes Sourced from github.com/shopspring/decimal's releases. v1.4.0 Full Changelog can be found in CHANGELOG.md New Contributors @VadimKulagin made their first contribution in shopspring/decimal#278 @CAEL0...
If generated data for languages will be split between per-language, it's possible to strip down bundled language immensely. ### Tag handling Each file can contain `//go:build` directive that controls inclusion...
