fastjson2 icon indicating copy to clipboard operation
fastjson2 copied to clipboard

当我在使用fastjson2序列化redis的时候,反序列化异常

Open cc-7z opened this issue 2 years ago • 13 comments

问题描述

我在springboot项目中,想通过fastjson2序列化redis的ValueSerializer和HashValueSerializer以下为序列化和反序列化代码 image

我序列化到redis的时候是正常的, 使用JSON.toJSONBytes(T) 值如下 {"attributes":{"java.security.Principal":{"authenticated":true,"authorities":[{"authority":"ROLE_admin"},{"authority":"ROLE_user"}],"details":{"remoteAddress":"127.0.0.1","sessionId":"A945B195F014D0BA4C8F0FA42D228153"},"name":"leon","principal":{"accountNonExpired":true,"accountNonLocked":true,"attributes":{},"authorities":[{"authority":"ROLE_admin"},{"authority":"ROLE_user"}],"credentialsNonExpired":true,"email":"[email protected]","enabled":true,"name":"leon","nickname":"leon","phone":"18137772224","userId":1,"username":"leon"}},"org.springframework.security.oauth2.core.endpoint.OAuth2AuthorizationRequest":{"additionalParameters":{},"attributes":{},"authorizationRequestUri":"http://127.0.0.1:9000/whale/oauth2/authorize?response_type=code&client_id=web&scope=all&redirect_uri=https://www.baidu.com","authorizationUri":"http://127.0.0.1:9000/whale/oauth2/authorize","clientId":"web","grantType":{"value":"authorization_code"},"redirectUri":"https://www.baidu.com","responseType":{"value":"code"},"scopes":["all"]},"state":"-q2sa200GhpO36UKcZ4BXgx9gxAqudwKg4i4tTYjMLo="},"authorizationGrantType":{"value":"authorization_code"},"id":"10ce34de-b5e3-4e2c-a3ab-73fa0a7bb46d","principalName":"leon","registeredClientId":"web"}

但是我当反序列化的时候就会提示不可以转换 异常信息 java.lang.ClassCastException: com.alibaba.fastjson2.JSONObject cannot be cast to org.springframework.security.oauth2.server.authorization.OAuth2Authorization

目标转换的类 image

环境信息

以下是我的坐标版本 <groupId>com.alibaba.fastjson2</groupId> <artifactId>fastjson2</artifactId> 2.0.11

cc-7z avatar Aug 12 '22 01:08 cc-7z

问题描述

我在springboot项目中,想通过fastjson2序列化redis的ValueSerializer和HashValueSerializer以下为序列化和反序列化代码 image

我序列化到redis的时候是正常的, 使用JSON.toJSONBytes(T) 值如下 {"attributes":{"java.security.Principal":{"authenticated":true,"authorities":[{"authority":"ROLE_admin"},{"authority":"ROLE_user"}],"details":{"remoteAddress":"127.0.0.1","sessionId":"A945B195F014D0BA4C8F0FA42D228153"},"name":"leon","principal":{"accountNonExpired":true,"accountNonLocked":true,"attributes":{},"authorities":[{"authority":"ROLE_admin"},{"authority":"ROLE_user"}],"credentialsNonExpired":true,"email":"[email protected]","enabled":true,"name":"leon","nickname":"leon","phone":"18137772224","userId":1,"username":"leon"}},"org.springframework.security.oauth2.core.endpoint.OAuth2AuthorizationRequest":{"additionalParameters":{},"attributes":{},"authorizationRequestUri":"http://127.0.0.1:9000/whale/oauth2/authorize?response_type=code&client_id=web&scope=all&redirect_uri=https://www.baidu.com","authorizationUri":"http://127.0.0.1:9000/whale/oauth2/authorize","clientId":"web","grantType":{"value":"authorization_code"},"redirectUri":"https://www.baidu.com","responseType":{"value":"code"},"scopes":["all"]},"state":"-q2sa200GhpO36UKcZ4BXgx9gxAqudwKg4i4tTYjMLo="},"authorizationGrantType":{"value":"authorization_code"},"id":"10ce34de-b5e3-4e2c-a3ab-73fa0a7bb46d","principalName":"leon","registeredClientId":"web"}

但是我当反序列化的时候就会提示不可以转换 异常信息 java.lang.ClassCastException: com.alibaba.fastjson2.JSONObject cannot be cast to org.springframework.security.oauth2.server.authorization.OAuth2Authorization

目标转换的类 image

环境信息

以下是我的坐标版本 com.alibaba.fastjson2 fastjson2 2.0.11

我用springboot的devtools热部署,也有遇到ClassCastException,但这并不是fastjson之类的bug,其他的库也会有这个错,ehcache也会遇到这个错。如果你也是用了devtools就有异常,不用就没异常,那具体可看spring官网的解决方案:https://docs.spring.io/spring-boot/docs/current/reference/html/using.html#using.devtools.restart.customizing-the-classload

fanyawei avatar Aug 12 '22 03:08 fanyawei

我用springboot的devtools热部署,也有遇到ClassCastException,但这并不是fastjson之类的bug,其他的库也会有这个错,ehcache也会遇到这个错。如果你也是用了devtools就有异常,不用就没异常,那具体可看spring官网的解决方案:https://docs.spring.io/spring-boot/docs/current/reference/html/using.html#using.devtools.restart.customizing-the-classload

目前我没有使用devtools,我只要是从redis读取进行强转,所有类都会提示这个异常 T t = (T)redisTemplate.opsForValue().get("key")

cc-7z avatar Aug 12 '22 05:08 cc-7z

@uhleon 使用GenericFastJsonRedisSerializer试试 see https://github.com/alibaba/fastjson2/blob/main/docs/spring_support_cn.md#41-generic-redis-serializer

VictorZeng avatar Aug 12 '22 05:08 VictorZeng

@uhleon 使用GenericFastJsonRedisSerializer试试 see https://github.com/alibaba/fastjson2/blob/main/docs/spring_support_cn.md#41-generic-redis-serializer

我非常感谢您的回复,我使用了提供的两种方案去序列化redis image

FastJsonRedisSerializer 提示的异常是和上述异常一致的,我复现了很多次,感觉好像是指定Object.class的问题

GenericFastJsonRedisSerializer 提示的异常为下述 image

cc-7z avatar Aug 12 '22 05:08 cc-7z

@uhleon 检查一下是否开启了safeMode see https://github.com/alibaba/fastjson2/blob/main/docs/autotype_cn.md

或者可以尝试配置一下自定义的白名单


        GenericFastJsonRedisSerializer serializer = new GenericFastJsonRedisSerializer(
                new String[] {
                        // 这里可以配置多个前缀白名单
                        "org.springframework.security.oauth2.server.authorization.OAuth2Authorization"
                }
        );

VictorZeng avatar Aug 12 '22 06:08 VictorZeng

@VictorZeng 我已经自定义配置完白名单 image

但是我如果使用 JSON.toJSONBytes(t, JSONWriter.Feature.WriteClassName);
去序列化Json到redis的时候,他已经是一个错误的JSON格式,并不能格式化 image

序列后的数据,您可尝试回显一下 {"@type":"org.springframework.security.oauth2.server.authorization.OAuth2Authorization","attributes":{"@type":"java.util.Collections$UnmodifiableMap","java.security.Principal":{"@type":"org.springframework.security.authentication.UsernamePasswordAuthenticationToken","authenticated":true,"authorities":[{"@type":"org.springframework.security.core.authority.SimpleGrantedAuthority","authority":"ROLE_admin"},{"@type":"org.springframework.security.core.authority.SimpleGrantedAuthority","authority":"ROLE_user"}],"details":{"@type":"org.springframework.security.web.authentication.WebAuthenticationDetails","remoteAddress":"127.0.0.1","sessionId":"29584A209FD143DB24C36708547E7A87"},"name":"leon","principal":{"@type":"com.uhbro.whale.dto.AuthUserDto","accountNonExpired":true,"accountNonLocked":true,"attributes":{"@type":"java.util.HashMap"},"authorities":Set[{"@type":"org.springframework.security.core.authority.SimpleGrantedAuthority","authority":"ROLE_admin"},{"@type":"org.springframework.security.core.authority.SimpleGrantedAuthority","authority":"ROLE_user"}],"credentialsNonExpired":true,"email":"[email protected]","enabled":true,"name":"leon","nickname":"leon","phone":"18137772224","userId":1,"username":"leon"}},"org.springframework.security.oauth2.core.endpoint.OAuth2AuthorizationRequest":{"@type":"org.springframework.security.oauth2.core.endpoint.OAuth2AuthorizationRequest","additionalParameters":{"@type":"java.util.Collections$UnmodifiableMap"},"attributes":{"@type":"java.util.Collections$UnmodifiableMap"},"authorizationRequestUri":"http://127.0.0.1:9000/whale/oauth2/authorize?response_type=code&client_id=web&scope=all&redirect_uri=https://www.baidu.com","authorizationUri":"http://127.0.0.1:9000/whale/oauth2/authorize","clientId":"web","grantType":{"value":"authorization_code"},"redirectUri":"https://www.baidu.com","responseType":{"value":"code"},"scopes":Set["all"]},"state":"ITbg4w8iUSHwaFbRmFTu8es9CfFSAMQg6oZEq3fwH4Y="},"authorizationGrantType":{"value":"authorization_code"},"id":"29546756-a176-463c-8dd6-e4f675e20d8a","principalName":"leon","registeredClientId":"web"}

异常信息 image

cc-7z avatar Aug 12 '22 08:08 cc-7z

@uhleon 可以提供一下详细的异常堆栈信息吗


做了一下testcase 问题出现在UsernamePasswordAuthenticationToken 无法基于setAuthenticated方法反序列化 image

该场景可以基于Field进行反序列化 增加如下配置试试:

JSONWriter.Feature.FieldBased
JSONReader.Feature.FieldBased

VictorZeng avatar Aug 12 '22 08:08 VictorZeng

@VictorZeng
我尝试了您提供的方法,但是还是未能解决问题,现在我有一个疑问,当我配置redis的时候 如下 image 当我指定泛型为Object.class 就不可以反序列化Bean了

复现代码: image image

异常信息: image

是我哪里方法用的有错误吗

cc-7z avatar Aug 13 '22 05:08 cc-7z

@uhleon 如果在不使用AutoType的时候,反序列时类型输入Object的话,可能返回JSONObject / JSONArray,可以试试这样:

        FastJsonRedisSerializer fastJsonRedisSerializer = new FastJsonRedisSerializer(SystemRolePo.class);

VictorZeng avatar Aug 15 '22 00:08 VictorZeng

@VictorZeng 我也遇到了同样的问题,但是我通过升级到2.0.11之后解决了这个问题

zhhnever avatar Aug 15 '22 08:08 zhhnever

https://github.com/alibaba/fastjson2/issues/647 跟我这个问题类似吧,反反复复

prog-ape avatar Aug 16 '22 03:08 prog-ape

https://oss.sonatype.org/content/repositories/snapshots/com/alibaba/fastjson2/fastjson2/2.0.13-SNAPSHOT/ 帮用2.0.13版本快照重新测试下,这个版本补充了更多的错误信息,用于方便诊断

wenshao avatar Aug 24 '22 18:08 wenshao

#647 跟我这个问题类似吧,反反复复

这个问题还在么,打不开描述信息,这边有个新方式解决

wwkk7080 avatar Aug 28 '22 23:08 wwkk7080

https://github.com/alibaba/fastjson2/releases/tag/2.0.15 2.0.15版本已发布,请用新版本帮忙验证

wenshao avatar Oct 05 '22 06:10 wenshao

https://github.com/alibaba/fastjson2/releases/tag/2.0.15 2.0.15版本已发布,请用新版本帮忙验证

我试了,仍然不行。

3517277 avatar Oct 05 '22 14:10 3517277

@3517277 参照https://github.com/alibaba/fastjson2/wiki/fastjson2_autotype_cn 这里的第5点配置autoTypeFilter试试看

wenshao avatar Oct 05 '22 15:10 wenshao

这样确实可以了,但这样做,有点麻烦,如果我每次增加类,都要这样增加一次,这有点糟糕了。

3517277 avatar Oct 07 '22 04:10 3517277

可以显示把AutoType打开,但显示打开AutoType会不安全,配置autoTypeFilter虽然麻烦,但安全,配置autoTypeFilter支持通配符的,可以用包前缀

wenshao avatar Oct 07 '22 04:10 wenshao

可以显示把AutoType打开,但显示打开AutoType会不安全,配置autoTypeFilter虽然麻烦,但安全,配置autoTypeFilter支持通配符的,可以用包前缀

好的,谢谢!

3517277 avatar Oct 07 '22 05:10 3517277

2.0.17也有这个问题

yizhishang avatar May 06 '23 05:05 yizhishang

@yizhishang 2.0.17遇到什么问题了?同时是否可以用2.0.29版本验证下?

wenshao avatar May 06 '23 05:05 wenshao

@yizhishang 2.0.17遇到什么问题了?同时是否可以用2.0.29版本验证下?

使用2.0.29,同样如此报错 java.lang.ClassCastException: com.alibaba.fastjson.JSONObject cannot be cast to xxx

我回滚到1.2.75就好了

yizhishang avatar May 06 '23 06:05 yizhishang

我也是redis读取数据反序列化时报的错

yizhishang avatar May 06 '23 06:05 yizhishang

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.parser.ParserConfig;
import com.alibaba.fastjson.serializer.SerializerFeature;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.type.TypeFactory;
import org.springframework.data.redis.serializer.RedisSerializer;
import org.springframework.data.redis.serializer.SerializationException;
import org.springframework.util.Assert;

import java.nio.charset.StandardCharsets;

/**
 * Redis使用FastJson序列化
 *
 * @author yizhishang
 */
public class FastJson2JsonRedisSerializer<T> implements RedisSerializer<T> {
    @SuppressWarnings("unused")
    private ObjectMapper objectMapper;

    private Class<T> clazz;

    static {
        ParserConfig.getGlobalInstance().setAutoTypeSupport(true);
    }

    public FastJson2JsonRedisSerializer(Class<T> clazz) {
        super();
        this.clazz = clazz;
    }

    @Override
    public byte[] serialize(T t) throws SerializationException {
        if (t == null) {
            return new byte[0];
        }
        return JSON.toJSONString(t, SerializerFeature.WriteClassName).getBytes(StandardCharsets.UTF_8);
    }

    @Override
    public T deserialize(byte[] bytes) throws SerializationException {
        if (bytes == null || bytes.length <= 0) {
            return null;
        }
        String str = new String(bytes, StandardCharsets.UTF_8);

        return JSON.parseObject(str, clazz);
    }

    public void setObjectMapper(ObjectMapper objectMapper) {
        Assert.notNull(objectMapper, "'objectMapper' must not be null");
        this.objectMapper = objectMapper;
    }

    protected JavaType getJavaType(Class<?> clazz) {
        return TypeFactory.defaultInstance().constructType(clazz);
    }
}

yizhishang avatar May 06 '23 06:05 yizhishang