spaCy
                                
                                
                                
                                    spaCy copied to clipboard
                            
                            
                            
                        Support custom attributes for tokens and spans in json conversion
Description
Currently, Doc.to_json() and Doc.from_json() only support custom attributes set on the whole Doc object. This PR adds support for custom attributes in tokens and spans.
Types of change
New Feature
Checklist
- [x] I confirm that I have the right to submit this contribution under the project's MIT license.
 - [x] I ran the tests, and all new and existing tests passed.
 - [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
 
I've adjusted the naming and ensured that added keys in the future wouldn't break the code. I also ensured that the code checks whether the user_data is a tuple + attributes with the same name for Doc, Token, and Span are now supported.
For the new logic, I also added some additional tests.