spaCy

spaCy copied to clipboard

Dependency sentence segmenter handles newlines inconsistently between languages

3

## How to reproduce the behaviour [Colab notebook demonstrating problem](https://colab.research.google.com/drive/14FFYKqjRVRbN7aAVmHUYEao9CwahY0We?usp=sharing) When parsing a sentence that contains newlines, the Italian parser sometimes assigns the newline to a sentence by itself, for...

freddyheppell

Sentence-terminal periods not tokenized properly in Malayalam text

3

## How to reproduce the behaviour import spacy nlp = spacy.blank('ml') doc = nlp('ഇന്ത്യയിൽ കേരള സംസ്ഥാനത്തിലും കേന്ദ്രഭരണപ്രദേശങ്ങളായ ലക്ഷദ്വീപിലും പോണ്ടിച്ചേരിയുടെ ഭാഗമായ മാഹിയിലും തമിഴ്നാട്ടിലെ കന്യാകുമാരി ജില്ലയിലും നീലഗിരി ജില്ലയിലെ ഗൂഡല്ലൂർ താലൂക്കിലും സംസാരിക്കപ്പെടുന്ന ഭാഷയാണ്...

BLKSerene

Access NEL prediction scores across KB candidates

3

At the time of writing it is not possible to seamlessly access the prediction scores for an Entity Linking model (spaCy 3.4). This impacts the ability to conduct misclassification analyses,...

Luis-R-Flores

Unaligned predicted spans are ignored in `Scorer.score_spans`

5

## How to reproduce the behaviour I originally encountered this issue when attempting to score a simple regex-based span prediction approach as a baseline. My reference documents contain 286 spans...

connorbrinton

NER fails on warning "Token indices sequence length is longer than the specified maximum"

3

## How to reproduce the behaviour code: > text1="My name is robert" + " \r\n " * 1000 + "My name is robert" > text2="My name is robert " +...

schudoku

Russian pos tagging/lemmatization/morphological analysis fails with diacritics

5

It seems that while there is support for tokenization with diacritics in spaCy, the project doesn't lemmatize/morph/pos tag correctly when they are used. ## How to reproduce the behaviour ```...

mtak-

Displacy visualiser only sometimes shows labels

8

In many cases, I do not see the label at the beginning of a span in displacy. This should not be an issue with a particular label since I sometimes...

goonhoon

Installation issue on old macOSes for new Korean tokenizer in v4.0 alpha

1

Hi, I noticed from #12328 that spaCy has switched to `pymecab-ko` for the Korean tokenizer in the upcoming `spaCy` 4.0, but there seems to be some installation/import issues of this...

BLKSerene

### Discussed in https://github.com/explosion/spaCy/discussions/12301 Originally posted by **mbrunecky** February 18, 2023 I am training NER using transformer model. On one of my data sets, during epoch 2, the score reaches...

svlandeg

Not all displaCy templates can be overridden

1

Even though [TPL_ENT](https://github.com/explosion/spaCy/blob/master/spacy/displacy/templates.py#L51) can be overridden using `template` option of a renderer there is no way to override [TPL_ENTS](https://github.com/explosion/spaCy/blob/2d4fb94ba0a23523cc9adb65e0dcf92bbf6177b6/spacy/displacy/templates.py#L46) template which might be useful in some cases. I believe it...

drnextgis