mongo-rust-driver icon indicating copy to clipboard operation
mongo-rust-driver copied to clipboard

RUST-1488 Populate inserted_ids when an insert_many fails

Open kmahar opened this issue 2 years ago • 1 comments

fixes #748

The fix ended up being a little more complicated than I initially expected in order to get it work properly for multi-batch bulk writes but was still relatively straightforward; most of the diff here is just adding testing. @isabelatkinson we should consider adding automated tests around this type of thing as part of the new bulk spec

kmahar avatar Oct 06 '22 22:10 kmahar

the evg failures don't seem related. also, I suppose we should probably backport this to the 2.3 branch and tag 2.3.2

kmahar avatar Oct 07 '22 22:10 kmahar

Being as ignorant and disorganized as I am, I have reopened this in https://github.com/mongodb/mongo-rust-driver/issues/1157 and attempted an implementation in https://github.com/mongodb/mongo-rust-driver/pull/1158. I am currently using that from my private fork where BulkWriteFailure.inserted_ids is pub. If anyone would like to look over the code and give it a general go-ahead I'd be happy to write (steal from this PR) some tests.

But if this is a whole process, (I thought it may not be since it's not a breaking change) I believe it would be a lot more ergonomic to start breaking application "errors" out of the general error. Examples would range from Result<InsertManyResult{ids: Vec<Result<Id, Err>>}, mongodb::Error> to even Result<Result<InsertOneResult, InsertOneFailure>, mongodb::Error>

But that sounds like 4.0.

qm3ster avatar Jun 28 '24 21:06 qm3ster

@qm3ster yeah that would be quite a major overhaul of our error API. 3.0 is very fresh so breaking changes will likely not happen again for a while.

Also worth noting that the new bulk write API does report inserted IDs in its error type; BulkWriteError contains a partial result which, if verbose_results was configured on the call to bulk_write, will record an InsertResult for any successful inserts that happened before the error occurred. e.g.:

let write_models = <InsertOneModel for each insert document>;
let bulk_write_result = client.bulk_write(write_models).verbose_results().await;
let insert_results = match bulk_write_result {
    Ok(result) => Some(result.insert_results),
    Err(error) => {
        if let ErrorKind::BulkWrite(bulk_write_error) = *error.kind {
            match bulk_write_error.partial_result {
                Some(PartialBulkWriteResult::Verbose(result)) => Some(result.insert_results),
                _ => None,
            }
        } else {
            None
        }
    }
}; // returns Option<HashMap<usize, InsertOneResult>>

(The major caveat here is that bulk write will not be available for use in production until MongoDB 8.0 is released, but once that happens this will be a way to inspected inserted IDs upon error without requiring changes to the driver.)

isabelatkinson avatar Jun 28 '24 21:06 isabelatkinson