msgspec icon indicating copy to clipboard operation
msgspec copied to clipboard

Allow bypassing built-in type conversion in `msgspec.convert` via dec_hook

Open diprog opened this issue 6 months ago • 1 comments
trafficstars

Related Issues

#671 ("Support adding hooks for supported objects types") - Similar request for overriding native type handling. #766, #669, #671


Description

When using msgspec.convert with structs, certain built-in types (like datetime, date, etc.) are handled automatically and bypass the dec_hook. This forces developers to create artificial subclasses of native types just to get custom decoding behavior.


Problem Example

Current Workaround (Artificial Subclasses Required)

from datetime import date, datetime
from typing import Any, Type
import msgspec

# Required wrapper classes to bypass built-in handling
class CustomDate(date):
    """Workaround to handle date strings like 'DD/MM/YYYY'"""
    pass

class CustomDatetime(datetime):
    """Workaround to handle ISO datetime strings"""
    pass

def dec_hook(type_: Type, obj: Any) -> Any:
    # Custom parsing only works through wrapper classes
    if type_ is CustomDate:
        if isinstance(obj, str):
            day, month, year = map(int, obj.split('/'))
            return CustomDate(year, month, day)
    
    if type_ is CustomDatetime:
        if isinstance(obj, str):
            return CustomDatetime.fromisoformat(obj)
    
    raise NotImplementedError(f"Unsupported type: {type_}")

class Event(msgspec.Struct):
    # Forced to use non-standard types
    start: CustomDatetime
    end: CustomDatetime | None
    registration_deadline: CustomDate

# Usage requires wrapper types
event = msgspec.convert(
    {"start": "2024-01-01T12:00:00", ...},
    Event,
    dec_hook=dec_hook
)

Problems This Creates:

  1. ❌ Artificial type hierarchy pollution
  2. ❌ Inconsistent with standard library types
  3. ❌ Extra boilerplate for common use cases
  4. ❌ Type checker complications

Proposed Solution

New API Usage

msgspec.convert(
    data,
    cls,
    dec_hook=custom_handler,
    disable_builtin_types={datetime, date}  # New parameter
)

Implementation Benefits:

  1. ✅ Uses standard types directly
  2. ✅ Explicit opt-in/opt-out control
  3. ✅ Maintains backward compatibility
  4. ✅ Clean integration with type checkers

Example Improvement:

def dec_hook(type_: Type, obj: Any) -> Any:
    # Direct native type handling
    if type_ is date:
        return parse_custom_date(obj)  # Implement your format
    
    if type_ is datetime:
        return parse_custom_datetime(obj)

class Event(msgspec.Struct):
    # Uses proper standard types
    start: datetime
    end: datetime | None
    deadline: date

Technical Considerations

  • Would work alongside existing builtin_types parameter
  • Should support all currently auto-converted types:
    • datetime, date, time, timedelta
    • UUID, Decimal
    • bytes, bytearray

diprog avatar May 16 '25 21:05 diprog

Or have dec_hook run prior to the builtin conversion, but yes I want this too!

clintval avatar Jun 10 '25 23:06 clintval

#829 is also an issue that would be solved by this.

JimDabell avatar Aug 04 '25 10:08 JimDabell