Additions to the Tru API for working with large databases #848

[priyalorha]

issue https://github.com/truera/trulens/issues/848

[joshreini1]

Thanks @priyalorha !

It looks like the tests are failing here because of some incorrect imports: ModuleNotFoundError: No module named 'trulens_eval.trulens_eval'

Can you adjust these? For example: from trulens_eval.trulens_eval.database.orm import Record, FeedbackResult

can be adjusted to: from trulens_eval.database.orm import Record, FeedbackResult

[piotrm0]

Hi; can you add abstract definitions of the new methods to db.py class DB ?

[joshreini1]

@ellipsis-dev can you add tests and documentation for this change?

[nsbradford]

hi @joshreini1 - apologies for delay in Ellipsis response, issues with our LLM provider. We're finding a workaround. (Automated summaries and reviews should still be operational)

[joshreini1]

hi @joshreini1 - apologies for delay in Ellipsis response, issues with our LLM provider. We're finding a workaround. (Automated summaries and reviews should still be operational)

Thanks for the update @nsbradford !

[piotrm0]

Thanks for the abstract definitions. A few smaller things:

• Can you add docstrings to the abstract methods (you can reference the abstract ones form the non-abstract ones instead of duplicating the docstrings).
• Run "format.sh --eval" in the main folder to unify the style formatting.

[priyalorha]

• format.sh --eval"

Formatting ./trulens_eval format.sh: line 14: isort: command not found format.sh: line 15: yapf: command not found

[piotrm0]

• format.sh --eval"

Formatting ./trulens_eval format.sh: line 14: isort: command not found format.sh: line 15: yapf: command not found

You can install these with pip.

[joshreini1]

@priyalorha were you able to successfully use these new methods? I was only able to successfully run tru.get_record_and_feedback.

Ran into various errors using:

• tru.list_records
• tru.delete_record
• tru.delete_app

To this end, it would be helpful to add usage examples to the docstrings for these methods and additional typing for the method arguments

[priyalorha]

nope, I was unable to set this up. Can you send me the error so that I can fix them

On Tue, 5 Mar 2024 at 2:41 AM, Josh Reini @.***> wrote:

@priyalorha https://github.com/priyalorha were you able to successfully use these new methods? I was only able to successfully run tru.get_record_and_feedback.

Ran into various errors using:

• tru.list_records
• tru.delete_record
• tru.delete_app

— Reply to this email directly, view it on GitHub https://github.com/truera/trulens/pull/939#issuecomment-1977462522, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI6R4KJQVVBWI7XUYZZTDFLYWTPQ5AVCNFSM6AAAAABD3CCDOKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXGQ3DENJSGI . You are receiving this because you were mentioned.Message ID: @.***>

[joshreini1]

nope, I was unable to set this up. Can you send me the error so that I can fix them … On Tue, 5 Mar 2024 at 2:41 AM, Josh Reini @.> wrote: @priyalorha https://github.com/priyalorha were you able to successfully use these new methods? I was only able to successfully run tru.get_record_and_feedback. Ran into various errors using: - tru.list_records - tru.delete_record - tru.delete_app — Reply to this email directly, view it on GitHub <#939 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI6R4KJQVVBWI7XUYZZTDFLYWTPQ5AVCNFSM6AAAAABD3CCDOKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXGQ3DENJSGI . You are receiving this because you were mentioned.Message ID: @.>

Seeing the following errors:

For all three methods, I'm seeing: AttributeError: type object 'Record' has no attribute 'id'

[priyalorha]

nope, I was unable to set this up. Can you send me the error so that I can fix them … On Tue, 5 Mar 2024 at 2:41 AM, Josh Reini @.> wrote: @priyalorha https://github.com/priyalorha were you able to successfully use these new methods? I was only able to successfully run tru.get_record_and_feedback. Ran into various errors using: - tru.list_records - tru.delete_record - tru.delete_app — Reply to this email directly, view it on GitHub <#939 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI6R4KJQVVBWI7XUYZZTDFLYWTPQ5AVCNFSM6AAAAABD3CCDOKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXGQ3DENJSGI . You are receiving this because you were mentioned.Message ID: _@**.**_>

Seeing the following errors:

For all three methods, I'm seeing: AttributeError: type object 'Record' has no attribute 'id'

I have updated the query both fetch should work fine, I have my doubts on delete , can you provide me with instructions on how to build and use this library?

[joshreini1]

nope, I was unable to set this up. Can you send me the error so that I can fix them … On Tue, 5 Mar 2024 at 2:41 AM, Josh Reini @.> wrote: @priyalorha https://github.com/priyalorha were you able to successfully use these new methods? I was only able to successfully run tru.get_record_and_feedback. Ran into various errors using: - tru.list_records - tru.delete_record - tru.delete_app — Reply to this email directly, view it on GitHub <#939 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI6R4KJQVVBWI7XUYZZTDFLYWTPQ5AVCNFSM6AAAAABD3CCDOKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXGQ3DENJSGI . You are receiving this because you were mentioned.Message ID: _@**.**_>

Seeing the following errors: For all three methods, I'm seeing: AttributeError: type object 'Record' has no attribute 'id'

I have updated the query both fetch should work fine, I have my doubts on delete , can you provide me with instructions on how to build and use this library?

You can install from your branch with the below code (also in README):

pip uninstall trulens_eval -y # to remove existing PyPI version
pip install git+https://github.com/truera/trulens@BRANCH#subdirectory=trulens_eval