support custom user model without user groups

[gitaarik]

If no user groups exist, don't query it, so that when you use a custom user model without groups, that you don't get an error.

Relevant GitHub Issue: https://github.com/modlinltd/django-advanced-filters/issues/57