Make IP Address and Email Database Storage PII / GDPR Compliant

[9mido]

trafficstars

IP addresses and email addresses fall into the PII and GDPR categories. The only way to be compliant and lawful is if you encrypt the IP & email in the database.

For the IP, one way to combat this is to create a setting that enables / disables the IP address collection in the database. Otherwise, encrypt it by default.

Email would just need to be encrypted since it is required.

Both should be able to be decrypted somehow as well.

[dokterbob]

Good point! I'll go into some of the considerations below. If and when other users agree, I would happily merge something like my proposal below.

Encryption

Although I'm not too sure on encrypting it at a database-level. That implies that there is a single key somewhere, and if that key leaks (which is likely when the database itself leaks), there is no improved protection of users. The only way to guarantee a higher level of security would be to store the key on a Yubikey or equivalent HSM, or to load the key on every start of the service from some external key storage.

IP Privacy

A better way would be to:

• Mask the last bits in IP addresses (e.g. GA sets the last 8 bits for IPv4 and the last 80 bits for IPv6 to 0), ref: https://support.google.com/analytics/answer/2763052?hl=en
• Not store IP addresses at all
• Store (salted) hashes - note that it is essential to use a salt as the limited amount of IPv4 addresses would make it trivial to use a rainbow table to resolve them back again. Luckily, strong hash functions as well as a salt are available 'for free' in Django. However, after salting, it becomes impossible to derive any information from the IP, other than being able to recognise the same IP again (on a server with the same secret).

Email privacy

Referring also the the encryption section above, I see no reason to encrypt email addresses. Rather, GDPR seems to require service providers to take all possible precautions to safeguard the personal information of users. This means, the server should be kept safe, no outdated dependencies should be used etc. Only after that, and only if we can store the encryption keys in a safer place than the database or the server itself, does it make sense to encrypt anything.

In addition, encrypting email addresses would make it impossible to do a SELECT/get() on email addresses, which is something that our application heavily relies upon.

Ref; https://www.reddit.com/r/gdpr/comments/91u93h/do_i_need_to_encrypt_user_email_addresses_in_my/

Proposal

1. Add a configuration option filter_ip() option and provide a default (privacy by default!) implementation which masks the last bits as per the GA reference above.
2. Not encrypting email addresses, but adding an explicit reference that we are storing personal information and that GDPR compliance requires that common security precautions are taken.

[9mido]

Wow great explanation of everything and glad to see you think it is a good idea.

I looked at django-fernet-fields which uses the SECRET_KEY to do both the encryption and decryption. If you are using postgres, you can use django-pgcrypto-fields which uses a postgres pgcrypto extension. It would be nice if django-newsletter had something like this built in without relying on these other packages and overriding the app.

I am looking forward to hearing other user's ideas!

[thismatters]

It's kind of a quagmire for this package to get involved in security questions. However, this package could follow Django's lead and provide an AbstractSubscription model which any project maintainer could subclass and make their own security choices.