SC-SAST: Provide ability to pass custom targs/sargs when starting scan

[kadraman]

We have many customers that pass custom targs/sargs when starting ScanCentral scans remotely, including:

• filter files (-filter)
• custom rules (-rules)
• scan precision (-scan-precision)/ scan policy (-scan-policy)

We need to update sc-sast start scan command with additional for sargs and targs. Although the scancentral package command accepts a targs options it doesn't appear to accept an sargs option and in case, from the documentation:

Deprecated: Fortify recommends that you use the ‑sargs option directly with the Start Command.

So we should add these options to our start command.

[rsenden]

Duplicate of #198, but keeping this one as it contains slightly more information.

[rsenden]

There is already PR #487, however:

1. I'm not fully convinced yet on how to best pass scan arguments to fcli; do we have a --sargs option taking a string, possibly repeatable --sargs option, or have explicit fcli options for each supported scan argument (also see #2 and #3)?
2. PR doesn't cater for local (rule/filterproject template) files to be included in the zip-file submitted to ScanCentral SAST Controller.
3. PR doesn't explicitly check for supported scan arguments before submitting the request to the controller. I think ScanCentral Client performs such a check, however not sure how we could properly do this in fcli as supported arguments may vary across ScanCentral SAST/Fortify SCA versions, and fcli needs to work with all currently supported product versions. We should discuss with @young-s-park.
4. For some reason, the PR contains a lot of unrelated updates to functional tests.

In general, these are the challenges that we face from an fcli perspective:

1. Fcli should work properly with any recent/supported ScanCentral SAST version
2. Ideally, we don’t want to include any explicit logic into fcli as to what scan arguments are supported, as the list of supported arguments can potentially vary across different SC SAST versions
3. Ideally, we don’t want to have any explicit command-line options like -rules (or alternatively look at -sargs contents to see whether it contains -rules) to identify what local files should be included in the scan payload; although unlikely, potentially a future Fortify SCA / SC SAST version could add support for some new scan argument that requires a local file to be included in the scan payload (or drop support for one of the existing scan arguments like -project-template), in which case we’d need to update options/logic in fcli.

Actually, some of these challenges may now also apply to ScanCentral Client itself; previously client and sensor version were tightly coupled, but this is no longer the case given the new sensor_version_for_all_jobs configuration property.

So far, the following options to add support for scan arguments in fcli (including identifying which local files need to be included in the payload) have crossed our minds. Note that based on conventions, all fcli option names (apart from single-letter options) need to have a double dash --.

1. fcli sc-sast scan start … --autoheap --filter filter.txt --quick
   Explicit options for every supported scan arguments; ideal for users (list of supported scan arguments in help output, fcli auto-completion, …) but virtually impossible due to challenge #2 above
2. fcli sc-sast scan start … --filter filter.txt --sargs “-autoheap -quick”
   Explicit options only for scan arguments that require a local file to be included in the scan payload, all other scan arguments passed through --sargs.
   I think this is confusing, even with ScanCentral Client itself it's unclear what happens if you use -sargs "-filter filter.txt" instead of using the explicit -filter option. Especially given that Supported -sargs options lists -filter, I'd expect to be able to pass this in -sargs but why is there then also an explicit -filter option? Also, this approach isn't ideal due to challenge #3.
3. fcli sc-sast scan start ... --sargs "-filter filter.txt -autoheap -quick"
   Only --sargs option; fcli would parse scan arguments to look for -filter, -rules, ... and then include referenced files in the scan payload.
   More user-friendly than option #2 above, however, again this approach isn't ideal due to challenge #3.
4. fcli sc-sast scan start ... --include-file filter.txt --sargs "-filter filter.txt -autoheap -quick"
   Separate option to explicitly include files into the scan payload.
   Not very user-friendly, as users need to enter the file name(s) twice. From an fcli perspective though, we don't need to have any understanding about what scan arguments take a (local) file as input.
5. fcli sc-sast scan start ... --sargs "-filter file:filter.txt -autoheap -quick"
   Special prefix within --sargs to indicate that an option value references a local file.
   Much more user-friendly than option #3 above, but with same advantages; fcli doesn't need to know which scan arguments take a file as input, we just look for any file: prefix in --sargs, add the referenced file to the scan payload, and replace file:filter.txt with the file name that we added to the payload. Main drawback is that users need to remember to add such a prefix for any files to be included in the scan payload.

With regards to the last option, of course we could also consider a different prefix, for example -filter !filter.txt, -filter @filter.txt, or -filter +filter.txt. We just need to make sure that such a prefix is unlikely to conflict with any actual scan argument that users may want to pass, and that it doesn't conflict with common shell syntax or existing fcli behavior (for example, I think @ may also be used to read fcli options from a file).