delta icon indicating copy to clipboard operation
delta copied to clipboard

[Feature Request] Make DELETE operations return the number of deleted rows

Open scottsand-db opened this issue 2 years ago • 4 comments

Feature request

Overview

Currently, the DELETE operation returns an empty result. It would be more useful if it returned the number of deleted rows.

Motivation

The number of deleted rows is an obvious metric that users would want from a delete operation.

Further details

Currently, DeleteCommand.scala is explicitly returning an empty DataFrame here. Instead, we could use the metrics member field to return the numDeletedRows.

Make sure to add tests, too :)

scottsand-db avatar Jun 22 '22 00:06 scottsand-db

this has to be done in scala ?

AhmedEHaddad avatar Jun 22 '22 12:06 AhmedEHaddad

Hi @AhmedEHaddad - I believe so. Do you propose another way? Internally, regardless of SQL or DataFrame API, Delta Lake will eventually end up using DeleteCommand.scala to implement the delete operation.

scottsand-db avatar Jun 22 '22 15:06 scottsand-db

@scottsand-db I had a look at the RunnableCommand and I hadn't grasped the meaning of the returned row. If I understood it correctly, this change should be sufficient.

https://github.com/edmondo1984/delta/commit/17998f4ca58f98fe808ce68114a44349ea345966

I wanted to get feedback around the design before adding the tests, I hadn't opened a PR yet for this reason. It's my first contribution, so rather than implementing tests for the wrong thing, I wanted to double check if I understood the API

edmondop avatar Jun 29 '22 01:06 edmondop

@edmondo1984 i left some comments. Make some small changes then make the PR! I can help with ways to add tests, too. thanks for helping!

scottsand-db avatar Jun 29 '22 22:06 scottsand-db