[Improvement] [log] If the password is passed as a parameter in the task log, the password will still be in clear text

[Capricorn0010]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

发现如果用参数传递密码的话，在节点的work-log 密码还是会明文显示，希望是密文，该怎么解决呢

What you expected to happen

SensitiveDataConverter.java 上处理敏感密码日志没有处理到，正则表达式没有覆盖到。

How to reproduce

使用sql组件来查下表里存储到密码，参数往下传递。

Anything else

No response

Version

3.1.x

Are you willing to submit PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[github-actions[bot]]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

I found that if I use parameters to pass the password, the password will still be displayed in plain text in the work-log of the node. I hope it is cipher text. How to solve this problem?

What you expected to happen

The processing of sensitive password logs in SensitiveDataConverter.java is not processed, and the regular expression is not covered.

How to reproduce

Use the sql component to check the password stored in the table, and pass the parameters downward.

Anything else

No response

Version

3.1.x

Are you willing to submit PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[delei]

Hi, @Capricorn0010 .I encountered the same problem,please refer to this and see if it is helpful to you.

Version

3.2.0

Solution

I achieved this by customizing the source code.

• add a new pattern to SensitiveDataConverter

private static final Pattern VAR_POOL_PATTERN =
            Pattern.compile("(?s)(\\{[^}]*(\"|\\\\\")prop(\"|\\\\\")\\s*:\\s*(\"|\\\\\")[^(\"|\\\\\")]*(?i:password)[^(\"|\\\\\")]*(\"|\\\\\")[^}]*(\"|\\\\\")value(\"|\\\\\")\\s*:\\s*(\"|\\\\\"))([^(\"|\\\\\")]*)((\"|\\\\\")[^}]*)");

• modify the logic of method maskSensitiveData(),use this new pattern for matching and filtering.replace with ******

public static String maskSensitiveData(final String logMsg) {
    // ... original code
    return replaceVarPoolSensitiveData(sb.toString());
}

public static String replaceVarPoolSensitiveData(String jsonString) {
    Matcher matcher = VAR_POOL_PATTERN.matcher(jsonString);
    return matcher.replaceAll("$1******$3");
}

Result

The content in the log:

prepareParamsMap

"prepareParamsMap" : {
  "DB_PASSWORD" : {
    "prop" : "DB_PASSWORD",
    "direct" : "IN",
    "type" : "VARCHAR",
    "value" : "******"
   },
  "password" : {
    "prop" : "password",
    "direct" : "IN",
    "type" : "VARCHAR",
    "value" : "******"
   },
  "system.project.code" : {
    "prop" : "system.project.code",
    "direct" : "IN",
    "type" : "VARCHAR",
    "value" : "11455319998912"
  }
}

varPool

"varPool" : "[{\"prop\":\"DB_PASSWORD\",\"direct\":\"IN\",\"type\":\"VARCHAR\",\"value\":\"******\"},{\"prop\":\"password\",\"direct\":\"IN\",\"type\":\"VARCHAR\",\"value\":\"******\"},{\"prop\":\"DB_USER\",\"direct\":\"IN\",\"type\":\"VARCHAR\",\"value\":\"test_user\"}]",

Set taskVarPool

[INFO] 2025-07-02 15:44:46.152 +0800 - Set taskVarPool: [{"prop":"DB_PASSWORD","direct":"IN","type":"VARCHAR","value":"******"},{"prop":"password","direct":"IN","type":"VARCHAR","value":"******"},{"prop":"DB_USER","direct":"IN","type":"VARCHAR","value":"test_user"}] successfully

Notes

• This may not be the best solution.
• Does not affect the default pattern.
• I only tested two task types: shell and datax.
• It is recommended to adjust the expression to better comply with parameter naming conventions.