fastjson2 icon indicating copy to clipboard operation
fastjson2 copied to clipboard

[FEATURE] JSON 比对

Open yoyo-520 opened this issue 1 year ago • 10 comments

请描述您的需求或者改进建议

之前看有人提过JSON比对的方法,但是后续没啥进度了,想咨询下这个超级实用的方法还会考虑增加吗? 这个链接

yoyo-520 avatar Oct 23 '23 03:10 yoyo-520

有考虑的,但最近比较忙,争取下个月考虑这个需求

wenshao avatar Oct 23 '23 14:10 wenshao

有考虑的,但最近比较忙,争取下个月考虑这个需求

好嘞,辛苦大佬~

yoyo-520 avatar Oct 25 '23 14:10 yoyo-520

我实现了一个json对比的工具,效果如图,看看是否满足需求

21021708853869_ pic

yanchangyou avatar Feb 25 '24 09:02 yanchangyou

这个需求主要用于监控数据变化~ 原始诉求如下:初始JSON、修改后的JSON 希望输出:哪些字段发生了变化,并且oldValue与newValue是什么~

[ { "field": "字段名", "newValue": "新的值", "originalValue": "老的值" } ........... ]

yoyo-520 avatar Feb 26 '24 04:02 yoyo-520

实现思路: 1,把json按照json path平铺展开 2,对平铺展开的json进行对比 3,返回对比接口

几个主要接口: compare: 返回所有结果 diff: 返回差异结果 sum: 对差异统计汇总 支持返回json、array

比较两个json

{
    "field":123456,
    "double":123.456789,
    "text": "ab",
    "string": "abc",
    "boolean": true,
    "object":{
        "field":123456,
        "number":123,
        "double":123.456,
        "string": "abc",
        "boolean": true,
        "newField":123
    },
    "array":[
        {
            "object":{
                "number":123,
                "double":123.456,
                "string": "abc",
                "boolean": true
            }
        }
    ]
}
{
    "field": "abc",
    "text": "abc",
    "number":123,
    "string": "abc",
    "object":{
        "field": "abc",
        "number":123,
        "string": "abc",
        "boolean": true
    },
    "array":[
        {
            "object":{
                "number":123,
                "double":123.456,
                "string": "abc"
            }
        }
    ]
}

比较结果:

{
    "equal": false,
    "total":16,
    "valueEqualCount":7,
    "typeEqualCount":1,
    "diffCount":9,
    "addCount":1,
    "removeCount":5,
    "modifyCount":3,
    "result":[
        {
            "path": "field",
            "valueEqual": false,
            "typeEqual": false,
            "diffType": "MODIFY",
            "value1":123456,
            "value2": "abc"
        },
        {
            "path": "double",
            "valueEqual": false,
            "diffType": "REMOVE",
            "value1":123.456789
        },
        {
            "path": "text",
            "valueEqual": false,
            "typeEqual": true,
            "diffType": "MODIFY",
            "value1": "ab",
            "value2": "abc"
        },
        {
            "path": "string",
            "valueEqual": true
        },
        {
            "path": "boolean",
            "valueEqual": false,
            "diffType": "REMOVE",
            "value1": true
        },
        {
            "path": "object.field",
            "valueEqual": false,
            "typeEqual": false,
            "diffType": "MODIFY",
            "value1":123456,
            "value2": "abc"
        },
        {
            "path": "object.number",
            "valueEqual": true
        },
        {
            "path": "object.double",
            "valueEqual": false,
            "diffType": "REMOVE",
            "value1":123.456
        },
        {
            "path": "object.string",
            "valueEqual": true
        },
        {
            "path": "object.boolean",
            "valueEqual": true
        },
        {
            "path": "object.newField",
            "valueEqual": false,
            "diffType": "REMOVE",
            "value1":123
        },
        {
            "path": "array[0].object.number",
            "valueEqual": true
        },
        {
            "path": "array[0].object.double",
            "valueEqual": true
        },
        {
            "path": "array[0].object.string",
            "valueEqual": true
        },
        {
            "path": "array[0].object.boolean",
            "valueEqual": false,
            "diffType": "REMOVE",
            "value1": true
        },
        {
            "path": "number",
            "valueEqual": false,
            "diffType": "ADD",
            "value2":123
        }
    ]
}

yanchangyou avatar Feb 26 '24 15:02 yanchangyou

顺带问下大神性能如何?因为这个功能在我们系统是个高频操作~ 任何数据库的修改都需要过这个比对方法。因为我们这边需要监控每张表的每行记录变化 所以频率非常高,比较关心性能~

yoyo-520 avatar Mar 04 '24 02:03 yoyo-520

比较json1,json2

{
	"number1":15,
	"number2":11,
	"number3":12,
	"number4":13,
	"number5":14,
	"string1":"abc",
	"string2":"abc3",
	"string3":"abc2",
	"string4":"abc1",
	"string5":"abc0"
}
{
	"number1":1.23,
	"number2":12,
	"number3":10,
	"number4":1,
	"number5":1.0,
	"string1":"abc",
	"string2":"abc1",
	"string3":"abc2",
	"string4":"abc3",
	"string5":"abc"
}

耗时统计:

次数	耗时(ms)
1024	26
2048	12
4096	18
8192	34
16384	59
32768	118
65536	243
131072	593
262144	965
524288	1929
1048576	3695

yanchangyou avatar Mar 04 '24 14:03 yanchangyou

那速度应该可以,看看能否融入进去~

yoyo-520 avatar Mar 13 '24 09:03 yoyo-520

已经作为独立的util推送了,大佬看看怎么融合 @wenshao

https://github.com/alibaba/fastjson2/pull/2267

yanchangyou avatar Mar 13 '24 14:03 yanchangyou

这个功能很实用哇~ 预计什么时候可以合并呢~

yoyo-520 avatar Jun 01 '24 14:06 yoyo-520