google
add a class that allows storing locations of errors
Could you please squash all the commits within the PR into a single one, and also run pyink on the code, using the settings from https://github.com/google/ffn/blob/master/ffn/pyproject.toml ?
Thanks! Unfortunately, it looks like it formatted everything with 4 spaces. One problem might have been that our pyproject.toml got accidentally moved to an incorrect location, so maybe that's why the tool didn't pick up the config. This is now fixed. Could you try syncing the repo and running pyink again? (or manually specify the settings that we use in the config file).
sorry for not checking this properly and all the hassle this created (was to happy about seeing pyink printing "all done" and trusted it did work according to toml). I noticed that the docstrings are not corrected to 2 indents by pyink. I did this manually now, but maybe pyink can do this on its own...
thanks for the detailed inspection!
Thanks for your patience with this. We're very close to having this completed now.
thanks for the detailed inspection!
Thanks for your patience with this. We're very close to having this completed now.
I am learning a lot :-) E.g. that conventions are much more "local" than expected :-).
Could you please try running https://github.com/google/pytype on this code? We're getting a bunch of failures caused by the new annotations.
Could you please try running https://github.com/google/pytype on this code? We're getting a bunch of failures caused by the new annotations.
This doesn't install on Windows. Unless there is an alternative to this I will not be able to check this for the next two weeks. Did the checks pass beforehand? Is there any hint to where the problem could be? I cannot view the details to this copybara thing either...
Here are the specific failures that are triggered:
File "third_party/py/ffn/utils/proofreading.py", line 65, in __init__: Built-in function len was called with the wrong arguments [wrong-arg-types]
Expected: (obj: Sized)
Actually passed: (obj: Optional[Iterable[Tuple[int, int, int]]])
Attributes of protocol Sized are not implemented on None: __len__
File "third_party/py/ffn/utils/proofreading.py", line 71, in Base: Invalid type annotation 'list[str, Any]' [invalid-annotation]
list[_T] expected 1 parameter, got 2
File "third_party/py/ffn/utils/proofreading.py", line 115, in update_segments: unsupported operand type(s) for item retrieval: 'aa: str' and 'layer: str' [unsupported-operands]
No attribute '__getitem__' on 'aa: str'
File "third_party/py/ffn/utils/proofreading.py", line 166, in list_segments: bad return type [bad-return-type]
Expected: List[int]
Actually returned: List[str]
File "third_party/py/ffn/utils/proofreading.py", line 316, in mark_bad: No attribute 'add' on list [attribute-error]
File "third_party/py/ffn/utils/proofreading.py", line 316, in mark_bad: No attribute 'add' on list [attribute-error]
In Union[Any, list]
Called from (traceback):
line 281, in <lambda>
File "third_party/py/ffn/utils/proofreading.py", line 318, in mark_bad: No attribute 'add' on list [attribute-error]
File "third_party/py/ffn/utils/proofreading.py", line 318, in mark_bad: No attribute 'add' on list [attribute-error]
In Union[Any, list]
Called from (traceback):
line 281, in <lambda>
File "third_party/py/ffn/utils/proofreading.py", line 330, in mark_removed_bad: unsupported operand type(s) for |: 'list' and 'new_bad: Set[int]' [unsupported-operands]
No attribute '__or__' on 'list' or '__ror__' on 'new_bad: Set[int]'
File "third_party/py/ffn/utils/proofreading.py", line 355, in ObjectReviewStoreLocation: Type annotation for seg_error_coordinates does not match type of assignment [annotation-type-mismatch]
Annotation: Optional[List[str]]
Assignment: Dict[nothing, nothing]
File "third_party/py/ffn/utils/proofreading.py", line 359, in ObjectReviewStoreLocation: Invalid type annotation 'list[str, List[List[int]]]' [invalid-annotation]
list[_T] expected 1 parameter, got 2
File "third_party/py/ffn/utils/proofreading.py", line 373, in __init__: No attribute 'items' on List[str] [attribute-error]
In Optional[List[str]]
File "third_party/py/ffn/utils/proofreading.py", line 418, in get_id: No attribute 'keys' on None [attribute-error]
In Optional[List[str]]
File "third_party/py/ffn/utils/proofreading.py", line 418, in get_id: No attribute 'keys' on List[str] [attribute-error]
In Optional[List[str]]
File "third_party/py/ffn/utils/proofreading.py", line 457, in store_error_location: No attribute 'update' on None [attribute-error]
In Optional[List[str]]
File "third_party/py/ffn/utils/proofreading.py", line 457, in store_error_location: No attribute 'update' on List[str] [attribute-error]
In Optional[List[str]]
File "third_party/py/ffn/utils/proofreading.py", line 526, in delete_location_from_annotation: unsupported operand type(s) for item deletion: None [unsupported-operands]
No attribute '__delitem__' on None
File "third_party/py/ffn/utils/proofreading.py", line 526, in delete_location_from_annotation: unsupported operand type(s) for item deletion: List[str] and str [unsupported-operands]
Function __delitem__ on List[str] expects Union[SupportsIndex, slice]
File "third_party/py/ffn/utils/proofreading.py", line 545, in delete_last_location: unsupported operand type(s) for item deletion: None [unsupported-operands]
No attribute '__delitem__' on None
File "third_party/py/ffn/utils/proofreading.py", line 545, in delete_last_location: unsupported operand type(s) for item deletion: List[str] and str [unsupported-operands]
Function __delitem__ on List[str] expects Union[SupportsIndex, slice]
File "third_party/py/ffn/utils/proofreading.py", line 686, in add_ccs: Function GraphUpdater.update_segments was called with the wrong arguments [wrong-arg-types]
Expected: (self, segments: List[int], ...)
Actually passed: (self, segments: set)
These are not new, we just got to this stage where this could be run. Any chance you could try WSL which the page you linked suggests would make it possible to run pytype? If this doesn't work, you could also try running another type checker like https://mypy-lang.org/
Were you able to get type checking to work on your end?
I'm still seeing some failures with the latest change:
File "third_party/py/ffn/utils/proofreading.py", line 115, in update_segments: unsupported operand type(s) for item retrieval: 'aa: str' and 'layer: str' [unsupported-operands]
No attribute '__getitem__' on 'aa: str'
File "third_party/py/ffn/utils/proofreading.py", line 166, in list_segments: bad return type [bad-return-type]
Expected: List[int]
Actually returned: List[str]
File "third_party/py/ffn/utils/proofreading.py", line 418, in get_id: No attribute 'keys' on None [attribute-error]
In Optional[Dict[str, list]]
File "third_party/py/ffn/utils/proofreading.py", line 457, in store_error_location: No attribute 'update' on None [attribute-error]
In Optional[Dict[str, list]]
File "third_party/py/ffn/utils/proofreading.py", line 526, in delete_location_from_annotation: unsupported operand type(s) for item deletion: None [unsupported-operands]
No attribute '__delitem__' on None
File "third_party/py/ffn/utils/proofreading.py", line 544, in delete_last_location: Function reversed.__init__ was called with the wrong arguments [wrong-arg-types]
Expected: (self, sequence: Reversible)
Actually passed: (self, sequence: Optional[Dict[str, list]])
Attributes of protocol Reversible[_T2] are not implemented on Dict[str, list]: __reversed__
File "third_party/py/ffn/utils/proofreading.py", line 545, in delete_last_location: unsupported operand type(s) for item deletion: None [unsupported-operands]
No attribute '__delitem__' on None
File "third_party/py/ffn/utils/proofreading.py", line 686, in add_ccs: Function GraphUpdater.update_segments was called with the wrong arguments [wrong-arg-types]
Expected: (self, segments: List[int], ...)
Actually passed: (self, segments: set)
Were you able to get type checking to work on your end?
I'm still seeing some failures with the latest change:
File "third_party/py/ffn/utils/proofreading.py", line 115, in update_segments: unsupported operand type(s) for item retrieval: 'aa: str' and 'layer: str' [unsupported-operands] No attribute '__getitem__' on 'aa: str' File "third_party/py/ffn/utils/proofreading.py", line 166, in list_segments: bad return type [bad-return-type] Expected: List[int] Actually returned: List[str] File "third_party/py/ffn/utils/proofreading.py", line 418, in get_id: No attribute 'keys' on None [attribute-error] In Optional[Dict[str, list]] File "third_party/py/ffn/utils/proofreading.py", line 457, in store_error_location: No attribute 'update' on None [attribute-error] In Optional[Dict[str, list]] File "third_party/py/ffn/utils/proofreading.py", line 526, in delete_location_from_annotation: unsupported operand type(s) for item deletion: None [unsupported-operands] No attribute '__delitem__' on None File "third_party/py/ffn/utils/proofreading.py", line 544, in delete_last_location: Function reversed.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, sequence: Reversible) Actually passed: (self, sequence: Optional[Dict[str, list]]) Attributes of protocol Reversible[_T2] are not implemented on Dict[str, list]: __reversed__ File "third_party/py/ffn/utils/proofreading.py", line 545, in delete_last_location: unsupported operand type(s) for item deletion: None [unsupported-operands] No attribute '__delitem__' on None File "third_party/py/ffn/utils/proofreading.py", line 686, in add_ccs: Function GraphUpdater.update_segments was called with the wrong arguments [wrong-arg-types] Expected: (self, segments: List[int], ...) Actually passed: (self, segments: set)
I have tried as far as possible without human support to get pytype running - it would not be recognized in WSL. I therefore used mypy. This did not run through error free but the errors that remained are non-overlapping with the ones you indicate:
proofreading.py:24: error: Skipping analyzing "networkx": module is installed, but missing library stubs or py.typed marker [import]
proofreading.py:24: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
proofreading.py:25: error: Skipping analyzing "neuroglancer": module is installed, but missing library stubs or py.typed marker [import]
proofreading.py:55: error: Need type annotation for "todo" (hint: "todo: List[
IIUC this is mypy not understanding some packages, complains about class attributes not being typed and does not understand None is returned in line509. I was hoping your typing tool would not crash there (apparantly the case!).
About the errors you get: I can fix line 418, line 457, line 526, line 544, line 545 - had this fixed before but must have undone this somehow before committing. Sorry! I think the error in line 686 is likely fixed by cenverting line 95 to '''segments: Sequence[int],''' However, the errors in line 115 and line 166 I don't understand. If self.todo is a list of dictionaries mapping lists of integer on layer (str) the typing should be correct. Do you see the error?
The type check you are doing is based on pytype? I can try to get help to get this running again but it may take me some time (human experts are scarce)
Thanks for the last round of changes! The latest version passes all our internal type checks, but I noticed that the annotation for the object list was actually not completely correct. I just pushed a commit that provides the correct annotations for that list -- could you please rebase on top of that and adjust the annotations where needed?