Type annotation with mypy

[edwinjosegeorge]

Improve code quality by adding type annotation. Working on the following

• https://github.com/BerriAI/litellm/issues/360
• https://github.com/BerriAI/litellm/issues/825
• https://github.com/BerriAI/litellm/issues/1540
• https://github.com/BerriAI/litellm/pull/501
• https://github.com/BerriAI/litellm/pull/1537
• https://github.com/BerriAI/litellm/pull/2908
• https://github.com/BerriAI/litellm/pull/3284

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 3, 2024 1:23am

[edwinjosegeorge]

I have added type annotation to tests directory and would love to expand it to the rest of the project. Would love to hear feedbacks.

Does this align with the projects goals? @ishaan-jaff @krrishdholakia

[krrishdholakia]

this would be amazing.

[edwinjosegeorge]

I was trying to update type annotation on some part of the project, and made some of the observations

• Type annotation can be made at lot of file => A PR with lot of changes??
• I am following inline type annotation, which makes the code even longer. Should we build stub files instead?? ie. a seperate file with pyi extension. That would mean that we will have to maintain multiple files for types when code updates.
• Most of the structures passed between are of dict type => dict[str, Any] is not a useful info for devs. Do we have plans to follow some stricter protocol typing? (maybe passing pydantic objects? or TypedDicts ?
• Should the typing be limited to the public interface only? I find type annotating entire codebase would be healthy for longer run to have a type safe. ( eg: In one of the function when type annotation was introduced, mypy warned for a possible NoneType have no attribute at line similar to value: str = my_dict1.get("key1").get("key2", "") )

what would be the better step to make LiteLLM as a typed package? Any view?

@ishaan-jaff @krrishdholakia @Manouchehri

[krrishdholakia]

What approach would you find most helpful, as a user of litellm? @edwinjosegeorge

[edwinjosegeorge]

My biased opinion (from my exp with previous projects) would be to follow with inline type annotation and restricting dict to pydantic. Something like, we could have a abstract class that would model various integrations ( eg: pre_event_log, etc etc). I try to use tools like black, flake8, mypy and pylint to analyse the code.

[krrishdholakia]

Makes sense. it can be easy to introduce bugs with large-scale changes.

Can we pick 1 e2e case, and start there? @edwinjosegeorge

What set of changes would be useful for you?

[lucasgadams]

I would also consider using pyright vs mypy. Quite similar, but i think in recent years pyright has really started to outshine mypy. But just my 2 cents.

[edwinjosegeorge]

I think we can start type annotation to litellm\llms\bedrock.py and its calls? The file currently have 1000+ line with multiple classes.

Or I could initially fix mypy typing errors, so that we can remove 'partial' tag at litellm\py.typed as mentioned in https://github.com/BerriAI/litellm/pull/3327 That would mean we allow loose types like dict and Any.

[edwinjosegeorge]

Closing this PR as there are a lot of changes with main such that they have become difficult to read/review. One possible (but looong way) is to ensure locally that all new contributions have 'strict' mypy compliance by running mypy . --ignore-missing-imports --strict