langtest icon indicating copy to clipboard operation
langtest copied to clipboard

Add tests for De-Identification

Open dcecchini opened this issue 1 year ago • 1 comments

We already have tests for NER models, but we should add a category for De-Identification so that we can test models for this specific capability.

We could extend it to test on clinical relevant de-identification tasks such as being compliant with HIPAA, etc.

dcecchini avatar Jan 04 '24 20:01 dcecchini

De-Identification examples

Category Test Type Original Test Case Expected Actual Pass
DeIdentification Simple Masking Patient John Doe was admitted on 01/01/2024. Mask names and dates PATIENT [MASK] was admitted on [MASK]. PATIENT [MASK] was admitted on [DATE]. True
DeIdentification HIPAA Compliance The patient's address is 123 Main St, Anytown. Mask address according to HIPAA The patient's address is [MASK] [MASK] [MASK]. The patient's address is [MASK] [MASK] CA. False
DeIdentification Redaction vs. Replacement The patient suffered from depression. Redact mental health conditions The patient suffered from [MENTAL_CONDITION]. The patient suffered from [REDACTED]. True

chakravarthik27 avatar Jan 10 '24 08:01 chakravarthik27