Discussion - _drop_remaining_rules() doesn't drop "required" validation

[KentonParton]

Used Cerberus version: 1.3.4

• [x] I have the capacity to improve the docs when my problem is solved.

• [x] I have the capacity to submit a patch when a bug is identified.

• [ ] My question does not concern a practical use-case that I can't figure out to solve.

---

Use-case abstract

The topic I would like to discuss is regarding the _drop_remaining_rules() and __validate_required_fields() methods and how the "required" rule is not dropped when _drop_remaining_rules() is invoked.

Currently, _drop_remaining_rules() removes all rules from the queue that still need to be evaluated for the current processed field. Specific rules can be passed in and only those rules will be removed from the queue. However, this is not the case for the "required" rule.

The "required" rule will not be removed and this is due to the "required" fields being evaluated in the method __validate_required_fields() after all other rules have been run.

My ask is, would we be able to invoke the "required" rule like the other rules so that _drop_remaining_rules() does in-fact remove all remaining rules?

The second option would be to add an argument to _drop_remaining_rules(path_to_required=None) where path_to_required is the path to the current required property and it is removed from self.schema. Therefore, when __validate_required_fields() is invoked, the field will not exist on the schema.

Option 1 would be preferred as it would be consistent with the rest of the validations.

Bug report / Feature request

[KentonParton]

On a related note, there are scenarios where one would want to perform custom validations even if a field doesn't exist.

We have the following validation

class CustomValidator(Validator):

    def __init__(self, *args, **kwargs):
        super(CustomValidator, self).__init__(*args, **kwargs)
        Validator.priority_validations = ('onpass', 'nullable', 'readonly', 'type', 'empty')
        self.allow_unknown = True

    def _drop_remaining_rules(self, *rules, path_to_required=None):
        """
        Drops rules from the queue of the rules that still need to be evaluated for the
        currently processed field. If no arguments are given, the whole queue is
        emptied.
        """
        if rules:
            for rule in rules:
                try:
                    self._remaining_rules.remove(rule)
                except ValueError:
                    pass
        else:
            self._remaining_rules = []

        if path_to_required:
            d = benedict(self.schema)
            try:
                del d[path_to_required[:], 'required']
            except KeyError:
                pass

    def _drop_property_rules(self, field):
        """
        Drops all remaining rules and rules that have
        already been run for the current property.
        """
        path_to_required = list(self.schema_path)
        path_to_required.append(field)
        self._drop_remaining_rules(path_to_required=path_to_required)
        deep_copy_errors = deepcopy(self._errors)
        for error in deep_copy_errors:
            error_doc_path = list(error.document_path)
            doc_path = list(self.document_path)
            doc_path.append(field)
            if error_doc_path == doc_path:
                self._errors.remove(error)

    def _validate_onpass(self, onpass, field, value):
        """{'type': 'dict'}"""
        passed = CustomValidator(schema=onpass).validate(document=self.document)
        if not passed:
            self._drop_property_rules(field)


schema = {
	'packageId': {
		'onpass': {
			'linkType': {
				'type': 'string',
				'required': True,
				'nullable': False,
				'empty': False,
				'allowed': [
					'pubfinder'
				]
			}
		},
		'required': True,
		'nullable': False,
		'empty': False,
		'type': 'integer',
	},
	'linkType': {
		'required': True,
		'type': 'string',
		'empty': False,
		'nullable': False,
		'allowed': [
			'pubfinder',
			'unknown'
		]
	}
}

document = {
	'linkType': 'unknown'
}

v = CustomValidator()
v.validate(schema=schema, document=document)
print(v.errors)

This is a work in progress so please excuse the messy code.

You will notice packageId has validation onpass. onpass takes in a dict with a document property of linkType and validations for that property. When all the validations pass, the remaining validations for packageId will be run.

If the validations inside onpass FAIL, then all remaining validations for packageId and all validations that have already error'ed are removed.

The issue I am running into is that when a property does not exist, in this case packageId, onpass is not executed. I have added onpass to priority_validations but it is still not evaluated.

If you run the above example, you will notice that cerberus expects packageId to be present. This is only because onpass is not being evaluated.

How would one still perform a custom validation on a property if the property is not present?

Thanks for your time, really appreciate it.

[funkyfuture]

i'm closing this and recommend to ask on Stackoverflow if the question is still relevant.