numberoverzero
Map supports partial updates
Currently, a types.Map only supports full updates. This can be an issue with concurrent additions.
Consider:
# ============= models.py
class Model(engine.model):
id = Column(String, hash_key=True)
document = Column(**{
'bar': Integer,
'baz': Integer
})
engine.bind()
# ============= Thread 1
from models import engine, Model
model = Model(id='foo')
engine.load(model)
model.document['bar'] = 1
engine.save(model)
# ============= Thread 2
from models import engine, Model
model = Model(id='foo')
engine.load(model)
model.document['baz'] = 1
engine.save(model)
If we now load the model, it will have either bar or baz populated (and not the other), whichever thread wrote last.
There are plenty of ways to monitor which keys have been modified for a dict subclass, but when the value is directly a dict it's not possible to replace references to that dictionary with a custom one.
model.document['foo'] can be intercepted if document is already a dict subclass.
This cannot: model.document = {'foo': 'bar'}.
Should take a look at http://docs.sqlalchemy.org/en/latest/orm/extensions/mutable.html and see how mature products already do this. At a glance, any __set__ on the field is pushed through a coerce to a tracking type, which weakrefs the parent object it's attached to.
Sample model:
class Model(engine.Model):
document = Column(Map(**{
'foo': Integer,
'bar': String
}))
Setting an instance's document field to a value would push that value through a coerce function that creates a new dict-like object which tracks __setitem__ and __getitem__ and has a weakref to the instance:
instance = Model()
some_document = {'foo': 10, 'bar': "Hello"}
instance.document = some_document
assert instance.document is not some_document
# does instance.document even need a weakref to instance?
It might be possible to have the coerced value be a very transparent proxy around the original object, but that would still allow the user to mutate the values (through the original some_document ref) without tripping the proxy's __setitem__ and __getitem__.
I'll play around with SQLAlchemy's ext.mutable.Mutable mixin and see what I can find. I doubt I can replace an existing dict's set/get item, which is what I really want.
Set should be an easier version of Map, and can just push the INSERT <value> changes instead of REPLACE SET <...>
Ugh. Implementing this in any reasonably performant way is killed by https://forums.aws.amazon.com/thread.jspa?threadID=162907, which indicates that paths can't overlap in expressions.
In other words, the following is not allowed:
SET data := if_not_exists(data, {})
SET data.inner := 3
Of course the thread is tagged as if there's anything useful in there: This question is answered. Helpful answers available: 1. Correct answers available: 1. but then that's just their shitty forums and not the DynamoDB team.
petemill-finesse has the correct summary of options here: https://forums.aws.amazon.com/message.jspa?messageID=711992#711992
Removing from the 2.0 milestone because that's within sight and a fix for this is not.