Archery
Archery copied to clipboard
脱敏查询虽然支持union/union all ,但不支持count
重现步骤
select count() as phone_cnt from t1 like '7%' UNION ALL select count() as phone_cnt from t2 like '7%' UNION ALL select count() as phone_cnt from t3 where phone_no like '7%' UNION ALL select count() as phone_cnt from t4 where phone_no like '7%';
预期外的结果
Error
数据脱敏异常:'field'
日志文本
No response
版本
1.8.4
部署方式
手工部署
是否还有其他可以辅助定位问题的信息?比如数据库版本等
No response
您好 这个问题我复现了,这个报错是
[2022-07-20 11:28:00,171][Thread-1:139749714147072][task_id:default][data_masking.py:51][WARNING]- 数据脱敏异常,错误信息:Traceback (most recent call last):
File "/u01/bianque/sql/utils/data_masking.py", line 31, in data_masking
select_list = del_repeat(select_list, keywords_count) if keywords_count else select_list
File "/u01/bianque/sql/utils/data_masking.py", line 71, in del_repeat
result_index = df.groupby(['field']).filter(lambda g: len(g) > 1).to_dict('records')
File "/u01/venv4archery/lib/python3.9/site-packages/pandas/core/frame.py", line 6515, in groupby
return DataFrameGroupBy(
File "/u01/venv4archery/lib/python3.9/site-packages/pandas/core/groupby/groupby.py", line 525, in __init__
grouper, exclusions, obj = get_grouper(
File "/u01/venv4archery/lib/python3.9/site-packages/pandas/core/groupby/grouper.py", line 786, in get_grouper
raise KeyError(gpr)
KeyError: 'field'
[2022-07-20 11:28:00,171][Thread-1:139749714147072][task_id:default][query.py:125][WARNING]- 数据脱敏异常,按照配置放行,查询语句:select count(*) as phone_cnt from users t1 where phone like '%1%'
UNION ALL
select count(*) as phone_cnt from users t2 where phone like '%2%'
UNION ALL
select count(*) as phone_cnt from users t3 where phone like '%3%'
UNION ALL
select count(*) as phone_cnt from users t4 where phone like '%4%' limit 100;,错误信息:'field'
这个问题是因为goInception返回结果是null
(venv4archery) [root@mysql01 goInception-v1.3.0-1]# python test1.py
((1, 'select count(*) from users', 0, '[]', None),)
+----+-----------------------------+----------+------------+--------+
| id | statement | errlevel | query_tree | errmsg |
+----+-----------------------------+----------+------------+--------+
| 1 | select count(*) from users | 0 | [] | None |
相关issue https://github.com/hanchuanchuan/goInception/issues/457
并且前端展示是正常的
所以报错问题不大,
当查询字段包含脱敏列,前端正常脱敏并且后端无报错

下一步:
1 优化 goInception count的返回值
2 优化程序对于count返回值处理
2