msgspec icon indicating copy to clipboard operation
msgspec copied to clipboard

Convert Decimals to ints

Open JimDabell opened this issue 8 months ago • 1 comments

Description

We’re currently storing integer fields on objects in DynamoDB. DynamoDB/boto3 returns all numbers as Decimals – there is no dedicated integer type in DynamoDB. There doesn’t seem to be a convenient way to define integer fields on our Msgspec model classes and convert the Decimals into ints when converting to our model type.

I know there is lax mode, but we want to be strict for everything except parsing fields defined as int from Decimal.

I know we can define the field as float and use multiple_of=1, but float is not the correct type for us, code that uses the model classes should see ints.

I tried a decoding hook, but these only get called for unknown types, and Decimal is a known type.

Is there anything I am missing?

JimDabell avatar Mar 27 '25 15:03 JimDabell

Try this. I have a working solution for BigInteger which is represented as str when send in response and as int in python. Converted my solution to fit your need. CAUTION: Not tested

from __future__ import annotations
from decimal import Decimal
from typing import Any, Self

class Dint:
    """Decimal Int.
    
    iv = int value
    dv = decimal value
    
    """
    __slots__ = ('iv', 'dv')

    def __new__(cls, v: int | Decimal | Dint) -> Self:
        self = object.__new__(cls)
        if v is None:
            self.iv, self.dv = None, None
        elif isinstance(v, Dint):
            self.iv = v.iv
            self.dv = v.dv
        elif isinstance(v, int):
            self.iv = v
            self.dv = Decimal(str(v))
        elif isinstance(v, Decimal):
            is_int = v.normalize().as_tuple().exponent >= 0
            if not is_int:
                raise ValueError("Invalid value for type int.")
            self.iv = int(v)
            self.dv = v
        else:
            raise TypeError(f"Expected int/Decimal/Dint. Got {type(v)}")
        return self

    def __bool__(self):
        return self.iv is not None

    def _cmp(self, other):
        if other is None:
            return False
        if isinstance(other, Dint):
            return self.iv == other.iv
        if isinstance(other, Decimal):
            return self.dv == other
        if isinstance(other, int):
            return self.iv == other
        raise TypeError(f"Unsupported other {other}")

    def __eq__(self, other):
        return self._cmp(other)

    def __ne__(self, other):
        return not self._cmp(other)

    def __int__(self):
        return self.iv

    def __repr__(self):
        return f'Dint({self.iv})'

    def __str__(self):
        return f"{self.iv}"


def dec_hook(t: type, o: Any) -> Dint:
    """Decode int represented as Decimal/int to Dint """
    if t is Dint:
        return t(o)
    raise NotImplementedError(f"Objects of type {type(o)} are not supported.")

def enc_int_hook(o: Dint) -> int:
    """Encode Dint to int"""
    if isinstance(o, Dint):
        return o.iv
    raise NotImplementedError(f"Objects of type {type(o)} are not supported.")

def enc_decimal_hook(o: Dint) -> int:
    """Encode Dint to decimal"""
    if isinstance(o, Dint):
        return o.dv
    raise NotImplementedError(f"Objects of type {type(o)} are not supported.")


class MyClass(Struct, kw_only=True):
    field1: Dint

zulqasar avatar Apr 02 '25 08:04 zulqasar