koalas icon indicating copy to clipboard operation
koalas copied to clipboard

Implements Index.putmask

Open beobest2 opened this issue 4 years ago • 11 comments

Implementing Index.putmask

>>> kidx = ks.Index(['a', 'b', 'c', 'd', 'e'])
>>> mask = [True if x < 2 else False for x in range(5)]
>>> value = 100

>>> kidx
Index(['a', 'b', 'c', 'd', 'e'], dtype='object')

>>> kidx.putmask(mask, value).sort_values()
Index(['100', '100', 'c', 'd', 'e'], dtype='object')

beobest2 avatar Jun 02 '20 06:06 beobest2

Could you also delete the put_mask of MultiIndex like the below and implement it?

@@ -58,7 +58,6 @@ class MissingPandasLikeIndex(object):
     is_type_compatible = _unsupported_function("is_type_compatible")
     join = _unsupported_function("join")
     map = _unsupported_function("map")
-    putmask = _unsupported_function("putmask")
     ravel = _unsupported_function("ravel")
     reindex = _unsupported_function("reindex")
     searchsorted = _unsupported_function("searchsorted")
@@ -131,7 +130,6 @@ class MissingPandasLikeMultiIndex(object):
     is_type_compatible = _unsupported_function("is_type_compatible")
     join = _unsupported_function("join")
     map = _unsupported_function("map")
-    putmask = _unsupported_function("putmask")
     ravel = _unsupported_function("ravel")
     reindex = _unsupported_function("reindex")
     remove_unused_levels = _unsupported_function("remove_unused_levels")

itholic avatar Jun 02 '20 07:06 itholic

@beobest2 can you fix the test?

HyukjinKwon avatar Jun 16 '20 10:06 HyukjinKwon

@HyukjinKwon okay I'll fix the test

beobest2 avatar Jun 16 '20 10:06 beobest2

Codecov Report

Merging #1560 into master will decrease coverage by 0.30%. The diff coverage is 97.53%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1560      +/-   ##
==========================================
- Coverage   94.55%   94.25%   -0.31%     
==========================================
  Files          38       38              
  Lines        8767     8715      -52     
==========================================
- Hits         8290     8214      -76     
- Misses        477      501      +24     
Impacted Files Coverage Δ
databricks/koalas/missing/indexes.py 100.00% <ø> (ø)
databricks/koalas/missing/series.py 100.00% <ø> (ø)
databricks/koalas/indexes.py 96.64% <92.59%> (-0.23%) :arrow_down:
databricks/koalas/__init__.py 93.54% <100.00%> (-0.57%) :arrow_down:
databricks/koalas/frame.py 95.94% <100.00%> (-0.89%) :arrow_down:
databricks/koalas/generic.py 96.65% <100.00%> (-0.02%) :arrow_down:
databricks/koalas/groupby.py 90.44% <100.00%> (-0.08%) :arrow_down:
databricks/koalas/series.py 97.61% <100.00%> (-0.01%) :arrow_down:
databricks/koalas/typedef/string_typehints.py 100.00% <100.00%> (ø)
databricks/koalas/typedef/typehints.py 86.00% <100.00%> (-1.37%) :arrow_down:
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update cc27c2a...02cf26c. Read the comment docs.

codecov-commenter avatar Jun 16 '20 15:06 codecov-commenter

@beobest2 could you rebase this when available ?

itholic avatar Aug 26 '20 11:08 itholic

@itholic sure :)

beobest2 avatar Aug 30 '20 10:08 beobest2

Hi @beobest2, since Koalas has been ported to Spark as pandas API on Spark, would you like to migrate this PR to the Spark repository? Here is the ticket https://issues.apache.org/jira/browse/SPARK-36403. Otherwise, I may do that for you next week.

xinrong-meng avatar Aug 03 '21 23:08 xinrong-meng

Hi @beobest2, since Koalas has been ported to Spark as pandas API on Spark, would you like to migrate this PR to the Spark repository? Here is the ticket https://issues.apache.org/jira/browse/SPARK-36403. Otherwise, I may do that for you next week.

Hi @xinrong-databricks I would like to migrate this PR to the Spark repository. I will try to finish it by next week.

beobest2 avatar Aug 04 '21 17:08 beobest2

Please take your time :) Thank you!

xinrong-meng avatar Aug 04 '21 20:08 xinrong-meng

@xinrong-databricks I created a PR at https://github.com/apache/spark/pull/33744 . Please take a look :)

beobest2 avatar Aug 14 '21 17:08 beobest2

Certainly, let's discuss in the new PR then! Thanks for the porting.

xinrong-meng avatar Aug 16 '21 16:08 xinrong-meng