run scala codes

[cometta]

does lighter allow run scala through spark magic 'spark' kernel?

[r65535]

Is it worth looking at replacing the nifi.properties logic we have here, to make use of the ENV scripts in apache/nifi's docker image instead? My worry with using templates is that we have a single, local copy of nifi.properties covering multiple minor/major versions of NiFi. I've skimmed through the difference between this PR's template and 2.0.0-M1 and there are dozens of differences (properties being removed, defaults being different etc). If we take nifi.properties values in the CRD and convert them into pod ENVs instead, we don't need a local template - and this should work across multiple versions?

[mh013370]

Is it worth looking at replacing the nifi.properties logic we have here, to make use of the ENV scripts in apache/nifi's docker image instead? My worry with using templates is that we have a single, local copy of nifi.properties covering multiple minor/major versions of NiFi. I've skimmed through the difference between this PR's template and 2.0.0-M1 and there are dozens of differences (properties being removed, defaults being different etc). If we take nifi.properties values in the CRD and convert them into pod ENVs instead, we don't need a local template - and this should work across multiple versions?

Hmm i think we ought to consider doing this as well. And NiFi 2.0.0 would be a good time to introduce breaking changes in nifikop as well, if we find that it's needed.

One minor hurdle may be that NiFi doesn't currently support being entirely configured through environment variables. I think the majority of properties are covered, but it's not a general solution as it's currently written. I saw a PR last week to add support for more properties. For that reason, we may need to keep both options around.

[juldrixx]

Is it worth looking at replacing the nifi.properties logic we have here, to make use of the ENV scripts in apache/nifi's docker image instead? My worry with using templates is that we have a single, local copy of nifi.properties covering multiple minor/major versions of NiFi. I've skimmed through the difference between this PR's template and 2.0.0-M1 and there are dozens of differences (properties being removed, defaults being different etc). If we take nifi.properties values in the CRD and convert them into pod ENVs instead, we don't need a local template - and this should work across multiple versions?

It's why I didn't removed them, just added the new ones. To stay compatible with previous version.

[juldrixx]

Is it worth looking at replacing the nifi.properties logic we have here, to make use of the ENV scripts in apache/nifi's docker image instead? My worry with using templates is that we have a single, local copy of nifi.properties covering multiple minor/major versions of NiFi. I've skimmed through the difference between this PR's template and 2.0.0-M1 and there are dozens of differences (properties being removed, defaults being different etc). If we take nifi.properties values in the CRD and convert them into pod ENVs instead, we don't need a local template - and this should work across multiple versions?

Hmm i think we ought to consider doing this as well. And NiFi 2.0.0 would be a good time to introduce breaking changes in nifikop as well, if we find that it's needed.

One minor hurdle may be that NiFi doesn't currently support being entirely configured through environment variables. I think the majority of properties are covered, but it's not a general solution as it's currently written. I saw a PR last week to add support for more properties. For that reason, we may need to keep both options around.

And I'm not sure, it will work with custom image.