sdv-dev
Improve automatic metadata detection of PII sdtypes (such as email)
In single table metadata API, it failed to recognise the sdtpye of email. It put email in categorical sd type instead of email sdtype
Hi @pritishbhakat , nice to meet you!
Currently the automatic detection of the sdtypes is in its early stages where detecting specific pii or other sdtypes is not yet supported. As you can see in our documentation the detect metadata is not guaranteed to be accurate or complete. Therefore I recommend you to use update_column method and adjust by hand the fields that are pii or incorrect.
how long will it take for working properly
As I mentioned before right now is working as intended, in the future we may include more features to the auto detection, but right now this is out of our scope.
As @pvk-developer mentioned, this feature is not currently supported by the SDV. So I'm turning the issue into a feature request for better support of metadata auto detection. I'll re-label and re-title the issue.
@pritishbhakat, we usually prioritize new features based on demand and importance to your projects. It would be helpful if you could speak a little more about your use case. Does your real data contain actual email addresses? And is the goal to anonymize these ... and potentially other types of PII?
