Feature requests: \yii\data\DataFilter accept filter in JSON format

[petr-panek]

Yii REST API filter format is very verbose (and confusing if complex AND/OR logic is required). If you need enter very complex filter, you may exceed HTTP url max length (2048 chars?).

I developped add-on feature that will allow to provide filter also in JSON format. Url params may looks like: ?filter={"productName":{"like":"iPhone"}}

This is not replacement of "standard" filter param system (?filter[productName][like]=iPhone), you can still use it. Both systems can work in parallel.

Implementation is easy, sligtly change / override load() method in \yii\base\Model. I override it in my custom ActiveDataFilter class this way:

public function load($data, $formName = null)
{
	// if filter attribute is set and is not array, we may expect it's in JSON format
	if (isset($data[$this->filterAttributeName]) && !is_array($data[$this->filterAttributeName])) {
		try{
			// try to decode json
			$data[$this->filterAttributeName] = Json::decode($data[$this->filterAttributeName]);
		} catch (\Exception) {
			// not sure how to properly respond to Json::decode error
			// $this->>addError will not help, later it will be override during validation with error "The format of Filter is invalid".
		}
	}

	// now we have filter in required array format, we can simply pass it to parent class load method
	return parent::load($data, $formName);
}

[samdark]

Looks interesting. Do you want to make it optional?

[petr-panek]

It's up to your recommendation. I'm not so skilfull with Yii to see all possible consequences of this change. For example is this feature related to DataFilter class level only or may sense to implement it to "low level" Model class?

Or is there some security risk to implement it as standard behaviour (not optional)? I believe not because validation mechanism should care about it, right?

[samdark]

For example is this feature related to DataFilter class level only or may sense to implement it to "low level" Model class?

That's filter-specific.

Or is there some security risk to implement it as standard behaviour (not optional)? I believe not because validation mechanism should care about it, right?

Should be alright security-wise.

[petr-panek]

Can you please also implement this change, it's related to the topic:

Currently in filter is not possible to use "standard" operators (=, !=, <, ...) as operator, only the alternate ones (eq, neq, lt, ...). This expression ?filter[ammount][=]=10 will cause error "Ammount must be integer." I understand that using standard operators in url can be risky, but for filter in json format is not the issue.

It can be simply achieved by extending DataFilter->filterControls:

Whole code will be:

public $filterControls = [
        '=' => '=',
        '!=' => '!=',
        '<' => '<',
        '>' => '>',
        '<=' => '<=',
        '>=' => '>=',
        'and' => 'AND',
        'and' => 'AND',
        'or' => 'OR',
        'not' => 'NOT',
        'lt' => '<',
        'gt' => '>',
        'lte' => '<=',
        'gte' => '>=',
        'eq' => '=',
        'neq' => '!=',
        'in' => 'IN',
        'nin' => 'NOT IN',
        'like' => 'LIKE',
    ];

[samdark]

I understand that using standard operators in url can be risky, but for filter in json format is not the issue.

It is an issue since at least = in URLs has special meaning.

[samdark]

As for implementing, that's not a high priority so if you need it anytime soon it's better to do it yourself and send us a pull request to review/merge.

[petr-panek]

And (probably) last idea to this topic:

Operator "like" can only search for string in any position (LIKE %string%). I'm missing possibilty to search string on beggining (LIKE string%) or end (LIKE %string).

I achieved this functionality defining new condition builder functions:

protected function buildLikeXCondition($operator, $condition, $attribute)
{
	if (isset($this->queryOperatorMap[$operator])) {
		$operator = $this->queryOperatorMap[$operator];
	}
	return ['LIKE', $attribute, $this->filterAttributeValue($attribute, $condition) . '%', false];
}

protected function buildXLikeCondition($operator, $condition, $attribute)
{
	if (isset($this->queryOperatorMap[$operator])) {
		$operator = $this->queryOperatorMap[$operator];
	}
	return ['LIKE', $attribute, '%' . $this->filterAttributeValue($attribute, $condition), false];
}

and put new deifitions into DataFilter properties: (this code is from __construct() of my custom ActiveDataFIlter class, but )

$this->filterControls['like%'] = 'LIKE%';
$this->filterControls['%like'] = '%LIKE';

$this->operatorTypes['LIKE%'] = [self::TYPE_STRING];
$this->operatorTypes['%LIKE'] = [self::TYPE_STRING];

$this->conditionValidators['LIKE%'] = 'validateOperatorCondition';
$this->conditionValidators['%LIKE'] = 'validateOperatorCondition';

$this->conditionBuilders['LIKE%'] = 'buildLikeXCondition';
$this->conditionBuilders['%LIKE'] = 'buildXLikeCondition';

Now it's possible to use filter ?filter[productName][like%]=Apple