pandas icon indicating copy to clipboard operation
pandas copied to clipboard

DOC: CoW and column addition

Open JB-AM opened this issue 2 years ago • 2 comments

Pandas version checks

  • [X] I have checked that the issue still exists on the latest versions of the docs on main here

Location of the documentation

https://pandas.pydata.org/docs/getting_started/intro_tutorials/05_add_columns.html

Documentation problem

I’m still grasping at the CoW error message and how to replace the usefull df['newcolumn'] = process(df['existing_column']), as the error message sends to a not so usefull help page (See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy, may need a separate ticket, but a simple example using a single index would definitely have helped me. The MultiIndex example clearly hides the message behind unnecessary complexity). But the above mentioned help page (https://pandas.pydata.org/docs/getting_started/intro_tutorials/05_add_columns.html) does not seem to follow the "new good practices" and should probably be updated.

Suggested fix for documentation

I’m not used enough to the new "standard", but the: air_quality["london_mg_per_cubic"] = air_quality["station_london"] * 1.882 should probably resemble something like: air_quality[:, "london_mg_per_cubic"] = air_quality["station_london"] * 1.882 and the same for the second example.

JB-AM avatar Feb 15 '24 06:02 JB-AM

I believe df['newcolumn'] = process(df['existing_column']) should create a new column on df with CoW enabled, can you provide a reproducible example?

rhshadrach avatar Feb 15 '24 22:02 rhshadrach

A likely point of confusion: df[mask]['newcolumn'] = some_value will not update df because df[mask] is a copy of df, and df[mask] will get updated instead (and then immediately garbage collected).

rhshadrach avatar Feb 15 '24 22:02 rhshadrach

I am very confused by this, we need 2 things here:

  • The warning message that Gets raised (the complete message)
  • A complete reproducer, the thing you wrote up should be completely valid, that's why we need something that we can run

phofl avatar Feb 18 '24 13:02 phofl

Closing for now. @JB-AM - if you can provide a reproducible example, happy to reopen! Just comment here.

rhshadrach avatar Feb 21 '24 22:02 rhshadrach