kyverno
kyverno copied to clipboard
kyverno
[Feature] New JMESPath filter `random()`
Problem Statement
There have been several requests in the community for a JMESPath filter capable of generating random sequences of characters. This is also a fairly common function found in many other languages/libraries.
Solution Description
Implement a custom JMESPath random() filter that is capable of generating a randomized set of output the length of which, at minimum, can be configurable as a parameter.
Alternatives
Use another field like UID from a resource to hijack their random string.
Additional Context
No response
Slack discussion
No response
Research
- [X] I have read and followed the documentation AND the troubleshooting guide.
- [X] I have searched other issues in this repository and mine is not recorded.
@chipzoller Yes I'm working on it. I'm getting some errors. In the GetFunctions I have added this.
{
Entry: &gojmespath.FunctionEntry{Name: random,
Arguments: []ArgSpec{
{Types: []JpType{JpNumber}},
},
Handler: jpRandom,
},
ReturnType: []JpType{JpString},
Note: "Generates a random sequence of characters",
},
Now I'm trying to create a func that will take an integer as argument generate the random string of characters.
func jpRandom(a []interface{}) (interface{}, error) {
var characters = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
b := make([]rune, a[0].(int))
for i := range b {
b[i] = characters[rand.Intn(len(characters))]
}
return string(b),nil
}
- When I'm trying to test my function then I'm getting this error.
$ cat random
random(`5`)
$ echo {} | ./kubectl-kyverno jp -e random
panic: interface conversion: interface {} is float64, not int
goroutine 1 [running]:
github.com/kyverno/kyverno/pkg/engine/jmespath.jpRandom({0xc000a0ab60?, 0xc000a0ab60?, 0x1?})
/home/kranurag7/kyverno/pkg/engine/jmespath/functions.go:899 +0x2dc
github.com/jmespath/go-jmespath.(*functionCaller).CallFunction(0x3631b00?, {0xc000a29280, 0x6}, {0xc000a0ab60, 0x1, 0x1}, 0xc0004aed80)
/home/kranurag7/go/pkg/mod/github.com/kyverno/[email protected]/functions.go:401 +0x252
github.com/jmespath/go-jmespath.(*treeInterpreter).Execute(0xc0004aed80, {0x4, {0x3544400, 0xc000a0a480}, {0xc00015bdd0, 0x1, 0x1}}, {0x369a600, 0xc00015bd70})
/home/kranurag7/go/pkg/mod/github.com/kyverno/[email protected]/interpreter.go:93 +0x1fe9
github.com/jmespath/go-jmespath.(*JMESPath).Search(...)
/home/kranurag7/go/pkg/mod/github.com/kyverno/[email protected]/api.go:37
github.com/kyverno/kyverno/cmd/cli/kubectl-kyverno/jp.Command.func1(0xc0001f7180?, {0xc0006c5f60?, 0x2?, 0x2?})
/home/kranurag7/kyverno/cmd/cli/kubectl-kyverno/jp/jp_command.go:82 +0x846
github.com/spf13/cobra.(*Command).execute(0xc0001f7180, {0xc0006c5f40, 0x2, 0x2})
/home/kranurag7/go/pkg/mod/github.com/spf13/[email protected]/command.go:856 +0x67cAN
github.com/spf13/cobra.(*Command).ExecuteC(0xc000581b80)
/home/kranurag7/go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3b4
github.com/spf13/cobra.(*Command).Execute(...)
/home/kranurag7/go/pkg/mod/github.com/spf13/[email protected]/command.go:902
main.main()
/home/kranurag7/kyverno/cmd/cli/kubectl-kyverno/main.go:35 +0xc9
I have tried using n int as function argument but that's not working out.
While random strings would be extraordinarily useful, random numbers would as well. Perhaps a randomStr() and randomNum()? I understand to_number() exists in the JMESPath spec, but it would be helpful nonetheless.
The filter could be written in such a way that one of the inputs is the composition of the output you'd like, examples being letters, numbers, or symbols.
@chipzoller Yes I'm working on it. I'm getting some errors. In the GetFunctions I have added this.
{ Entry: &gojmespath.FunctionEntry{Name: random, Arguments: []ArgSpec{ {Types: []JpType{JpNumber}}, }, Handler: jpRandom, }, ReturnType: []JpType{JpString}, Note: "Generates a random sequence of characters", },Now I'm trying to create a func that will take an integer as argument generate the random string of characters.
func jpRandom(a []interface{}) (interface{}, error) { var characters = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ") b := make([]rune, a[0].(int)) for i := range b { b[i] = characters[rand.Intn(len(characters))] } return string(b),nil }
- When I'm trying to test my function then I'm getting this error.
$ cat random random(`5`) $ echo {} | ./kubectl-kyverno jp -e random panic: interface conversion: interface {} is float64, not int goroutine 1 [running]: github.com/kyverno/kyverno/pkg/engine/jmespath.jpRandom({0xc000a0ab60?, 0xc000a0ab60?, 0x1?}) /home/kranurag7/kyverno/pkg/engine/jmespath/functions.go:899 +0x2dc github.com/jmespath/go-jmespath.(*functionCaller).CallFunction(0x3631b00?, {0xc000a29280, 0x6}, {0xc000a0ab60, 0x1, 0x1}, 0xc0004aed80) /home/kranurag7/go/pkg/mod/github.com/kyverno/[email protected]/functions.go:401 +0x252 github.com/jmespath/go-jmespath.(*treeInterpreter).Execute(0xc0004aed80, {0x4, {0x3544400, 0xc000a0a480}, {0xc00015bdd0, 0x1, 0x1}}, {0x369a600, 0xc00015bd70}) /home/kranurag7/go/pkg/mod/github.com/kyverno/[email protected]/interpreter.go:93 +0x1fe9 github.com/jmespath/go-jmespath.(*JMESPath).Search(...) /home/kranurag7/go/pkg/mod/github.com/kyverno/[email protected]/api.go:37 github.com/kyverno/kyverno/cmd/cli/kubectl-kyverno/jp.Command.func1(0xc0001f7180?, {0xc0006c5f60?, 0x2?, 0x2?}) /home/kranurag7/kyverno/cmd/cli/kubectl-kyverno/jp/jp_command.go:82 +0x846 github.com/spf13/cobra.(*Command).execute(0xc0001f7180, {0xc0006c5f40, 0x2, 0x2}) /home/kranurag7/go/pkg/mod/github.com/spf13/[email protected]/command.go:856 +0x67cAN github.com/spf13/cobra.(*Command).ExecuteC(0xc000581b80) /home/kranurag7/go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3b4 github.com/spf13/cobra.(*Command).Execute(...) /home/kranurag7/go/pkg/mod/github.com/spf13/[email protected]/command.go:902 main.main() /home/kranurag7/kyverno/cmd/cli/kubectl-kyverno/main.go:35 +0xc9I have tried using
n intas function argument but that's not working out.
https://go.dev/play/p/PkUWMm6hR4a
https://go.dev/play/p/YAZ5yU8Ul61
the interface you're passing is indeed being treated as a float64. What are the contents of "random"? I don't understand your command string, though I'm new here
Apologies, I was busy with university exams. Thanks @JimBugwadia for the guidance.
func jpRandom(a []interface{}) (interface{}, error) {
var characters = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
i := int(math.Round(a[0].(float64)))
b := make([]rune, i)
for i := range b {
b[i] = characters[rand.Intn(len(characters))]
}
return string(b), nil
}
This is what I'm testing now.
$ echo {} | ./kubectl-kyverno jp -e random
"XVlBzgbaiCMRAjW"
@chipzoller Can you give me some real world use case which I should try with random function before submitting the PR.
The filter needs to accept two parameters: length and composition. Here are some basic use cases which must be solved for:
- Create a random string 6 characters long composed of numbers
- Create a random string 12 characters long composed of lower-case letters and numbers
- Create a random string 15 characters long composed of lower-case letters, upper-case letters, and numbers
Simultaneously, the filter must gracefully handle and return errors for invalid inputs. For example, if -1 is supplied as a length parameter, unless it has special meaning it must cause an error. Depending on how the composition input parameter is specified, and invalid specification for that input must return an error. The filter must not induce panic for any ill-formatted or invalid input parameters.
Rather than have one function with several options, I recommend having simpler separate functions that users can combine as they wish, as there are many variations which we will not be able to cover.
So, perhaps have separate random_string and random_digits and then let users combine and use existing upper / lower case functions.
Thoughts?
Seems a little more complex to me than may be necessary. I was envisioning a simple way to specify both in the same filter:
random(`6`,'num')
random(`12`,'llet,num')
random(`15`,'llet,ulet,num')
First input is a number, second input is a string. You have keywords which can be comma separated which inform the function of the composition. Anything more complex than that and you can combine functions. For example a GUID where there are dash-delimited sections where some sections are number and some are pure letter.
"{{ random(`6`,'num')-random(`4`,'llet') }}"
Result of the above may be 810673-iyvu
Thanks, @chipzoller, for outlining the structure on what we want to achieve. I agree with @JimBugwadia having two separate functions, one for digits and one for strings, would make more sense. I think random_string is the one that will be used much.
If a user wants an alphanumeric strings, then we can use random for that.
To summarize
random_string - for strings only (uppercase & lowercase both)
random_digits - for numeric values
random - for alphanumeric strings
The complexity with having one for digits is the output type may be int versus having one function and using the built-in JMESPath filters for either to_string() or to_number() to handle that as needed in the field type.
random_string- for strings only (uppercase & lowercase both)
What if I don't want (or can't use) upper-case letters? I would be forced to pipe the output of that function to another filter which is more lengthy. I need to be able to control the composition of that random string and having a second input parameter which accepts comma-separated values allows that flexible composition.
@chipzoller can you please explain what llet,ulet,num mean and how they should be used?
Is this a standard or an implementation you have seen in other languages?
Does it indicate the first letter is a lowercase character, the second is an uppercase character, and the third is a number? Or that these are allowed in any combination? And what about dashes, etc.
A typical requirement is for the random string to start with an lowercase alphabet. How would something like that be controlled?
can you please explain what llet,ulet,num mean and how they should be used?
Yes, as shown here, these are keywords which mean the following:
-
llet== lower-case letter -
ulet== upper-case letter -
num== number
Note that these keywords are only examples. They can be set to anything.
Is this a standard or an implementation you have seen in other languages?
The concept is what I have seen. I am not aware of this as a development standard anywhere, but then again I also don't have significant experience with what is.
Does it indicate the first letter is a lowercase character, the second is an uppercase character, and the third is a number? Or that these are allowed in any combination? And what about dashes, etc?
It indicates the total composition. There is no ordering which this denotes. For things like composing a dash-separated random string, I provided the example in that previous comment:
"{{ random(`6`,'num')-random(`4`,'llet') }}"
A typical requirement is for the random string to start with an lowercase alphabet. How would something like that be controlled?
Yes, I would expect the filter to handle this logic implicitly if the composition includes whatever the keyword is signifying lower-case letters.
Using such flexible input parameters would allow you to solve for a bunch of valuable use cases:
Make your own "Pod hash"
Example: foo-a8u03p
name: {{ @ }}-{{ random(`6`,'llet,num') }}
Make your own GUID
Example: C3BD119C-47CA-48B3-A120-00B9A51BB6CA
somefield: "{{ random(`8`,'ulet,num') }}-{{ random(`4`,'ulet,num') }}-{{ random(`4`,'ulet,num') }}-{{ random(`4`,'ulet,num') }}-{{ random(`12`,'ulet,num') }}"
Make your own auth token
Example: a9Tjo7UiZ0OppN01lR4
somefield: "{{ random(`14`,'llet,ulet,num') }}"
Make your own license key
Example: AOPE-IOBJ-ZZYE-LQIB
somefield: "{{ random(`4`,'ulet,num') }}-{{ random(`4`,'ulet,num') }}-{{ random(`4`,'ulet,num') }}-{{ random(`4`,'ulet,num') }}"
Yes, but what exactly does llet,num mean? Any mix of lower case letters and numbers? Any order? I feel if we want to go in this direction we are better off specifying a more precise pattern.
Here is an example of using a regex pattern to generate a random string:
https://pkg.go.dev/github.com/zach-klippenstein/goregen
"{{ random(`[a-z0-9]{1,64}`) }}"
Yes, but what exactly does
llet,nummean? Any mix of lower case letters and numbers? Any order?
Yes, exactly. Because in random generation you typically don't care about a high degree of control unless you get into patterns, and with this type of flexibility and and what we can already do, as I've shown the patterns are straightforward.
Here is an example of using a regex pattern to generate a random string:
https://pkg.go.dev/github.com/zach-klippenstein/goregen
"{{ random(`[a-z0-9]{1,64}`) }}"
This seems like it would work and is pretty close to what I had suggested, just uses regex instead which provides more power. As long as the functional requirements can be met which solves the use cases, implementation can be flexible.
@kranurag7 would you like to continue with this issue and implement changes as discussed? If not, this will be released to Linux Foundation mentee applicants for the Fall 2022 term.
@chipzoller I was about to discuss this issue in today's meeting, but if you want, you can mark this for issue for mentorship.
Hey @chipzoller, I have looked into the package that was discussed in the meeting. I have decided to go ahead with using the that package, The function is working fine but i have noticed one thing that users needs to write the regex pattern correctly otherwise it will throw an error. Means the user needs to know regex very well.
- Make your own pod hash -
foo-[a-z0-9]{6}// foo-3r1xww - Make your own GUID -
[A-Z0-9]{8}-[A-Z0-9]{4}-[A-Z0-9]{4}-[A-Z0-9]{4}-[A-Z0-9]{12}// LDEGJQZH-TOFV-C8BB-BVPM-7WGZWVO6NCQ4 - Make your own auth token -
[a-z0-9A-Z]{14}//FTGHVQXxw4JEDC - Make your own license key -
[A-Z]{4}-[A-Z]{4}-[A-Z]{4}-[A-Z]{4}// MNYM-JRCF-NWKN-RMQF
To summarize, I'm taking only one parameter in the function, which is the regex pattern. Regex itself can be used to limit the number of characters we want. If we want 6 characters from a-z then we can do [a-z]{6}
I have tested the following cases and I want some feedback on this.
$ echo '{"foo": "foo-[a-z0-9]{6}"}' | kubectl-kyverno jp 'random(foo)'
"foo-91t6fm"
The above example is working fine but when I tested the same with the policy add_labels then it's appending the labels to the Kubernetes resource. Here is the manifest which I'm trying
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: add-labels
annotations:
policies.kyverno.io/title: Add Labels
policies.kyverno.io/category: Sample
policies.kyverno.io/severity: medium
policies.kyverno.io/subject: Label
policies.kyverno.io/description: >-
Labels are used as an important source of metadata describing objects in various ways
or triggering other functionality. Labels are also a very basic concept and should be
used throughout Kubernetes. This policy performs a simple mutation which adds a label
`foo=bar` to Pods, Services, ConfigMaps, and Secrets.
spec:
validationFailureAction: enforce
rules:
- name: add-labels
match:
resources:
kinds:
- Namespace
- Pod
- Service
- ConfigMap
- Secret
mutate:
patchStrategicMerge:
metadata:
labels:
random: "{{ random('[a-z0-9]{10}') }}"
After applying this & then creating namespace, I'm checking the labels of namespaces using kubectl get ns -o yaml and no labels are getting added. Can you please tell me if I'm doing something wrong here w.r.t manifest above.