ydata-profiling icon indicating copy to clipboard operation
ydata-profiling copied to clipboard

Error if index column's name is identical to another column's name.

Open kovar-ursa opened this issue 4 years ago • 3 comments

Describe the bug

If you use set_index with drop=False then your dataframe will have an index and a column with the same name. This produces the following error when opening the profile widget. Dropping the column with the same name as the index clears the error.

/opt/anaconda3/envs/a50-dev/lib/python3.9/site-packages/pandas/core/frame.py in insert(self, loc, column, value, allow_duplicates)
   4412         if not allow_duplicates and column in self.columns:
   4413             # Should this be a different kind of error??
-> 4414             raise ValueError(f"cannot insert {column}, already exists")
   4415         if not isinstance(loc, int):
   4416             raise TypeError("loc must be int")

ValueError: cannot insert TimeUTC, already exists

To Reproduce

Create a dataframe with a column named 'TimeUTC' and then use:

df.set_index('TimeUTC',inplace=True, drop=False)

and then:

from pandas_profiling import ProfileReport
as_df = as_df.drop('TimeUTC', axis=1)
profile = ProfileReport(as_df, title="Pandas Profiling Report", explorative=True)

Version information:

Click to expand Version information

python 3.9.7 ``` altair==4.1.0 anyio @ file:///Users/runner/miniforge3/conda-bld/anyio_1632754437423/work/dist appnope @ file:///Users/runner/miniforge3/conda-bld/appnope_1610094673755/work argon2-cffi @ file:///Users/runner/miniforge3/conda-bld/argon2-cffi_1625821234041/work astor==0.8.1 async-generator==1.10 attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1620387926260/work Babel @ file:///home/conda/feedstock_root/build_artifacts/babel_1619719576210/work backcall @ file:///home/conda/feedstock_root/build_artifacts/backcall_1592338393461/work backports.functools-lru-cache @ file:///home/conda/feedstock_root/build_artifacts/backports.functools_lru_cache_1618230623929/work base58==2.1.0 bleach @ file:///home/conda/feedstock_root/build_artifacts/bleach_1629908509068/work blinker==1.4 boto3 @ file:///home/conda/feedstock_root/build_artifacts/boto3_1631951251863/work botocore @ file:///home/conda/feedstock_root/build_artifacts/botocore_1632782434050/work brotlipy==0.7.0 cachetools==4.2.2 certifi==2021.5.30 cffi @ file:///Users/runner/miniforge3/conda-bld/cffi_1625835329396/work chardet @ file:///Users/runner/miniforge3/conda-bld/chardet_1610093613037/work charset-normalizer @ file:///home/conda/feedstock_root/build_artifacts/charset-normalizer_1626371162869/work click==7.1.2 click-plugins==1.1.1 cligj @ file:///home/conda/feedstock_root/build_artifacts/cligj_1622246659049/work cryptography @ file:///Users/runner/miniforge3/conda-bld/cryptography_1616851523607/work cycler==0.10.0 debugpy @ file:///Users/runner/miniforge3/conda-bld/debugpy_1627075104865/work decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1631346842025/work defusedxml @ file:///home/conda/feedstock_root/build_artifacts/defusedxml_1615232257335/work entrypoints @ file:///home/conda/feedstock_root/build_artifacts/entrypoints_1605121927639/work/dist/entrypoints-0.3-py2.py3-none-any.whl Fiona==1.8.20 fsspec @ file:///home/conda/feedstock_root/build_artifacts/fsspec_1632442862447/work GDAL==3.3.2 geographiclib @ file:///home/conda/feedstock_root/build_artifacts/geographiclib_1624386157733/work geopandas @ file:///home/conda/feedstock_root/build_artifacts/geopandas_1623249625470/work geopy @ file:///home/conda/feedstock_root/build_artifacts/geopy_1626114677070/work gitdb==4.0.7 GitPython==3.1.24 greenlet @ file:///Users/runner/miniforge3/conda-bld/greenlet_1628272262836/work htmlmin==0.1.12 idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1609836280497/work ImageHash==4.2.1 importlib-metadata @ file:///Users/runner/miniforge3/conda-bld/importlib-metadata_1630267507666/work ipykernel @ file:///Users/runner/miniforge3/conda-bld/ipykernel_1631291142144/work/dist/ipykernel-6.4.1-py3-none-any.whl ipython @ file:///Users/runner/miniforge3/conda-bld/ipython_1632763987529/work ipython-genutils==0.2.0 ipywidgets @ file:///home/conda/feedstock_root/build_artifacts/ipywidgets_1631590360471/work jedi @ file:///Users/runner/miniforge3/conda-bld/jedi_1610146808808/work Jinja2 @ file:///home/conda/feedstock_root/build_artifacts/jinja2_1621419064915/work jmespath @ file:///home/conda/feedstock_root/build_artifacts/jmespath_1589369830981/work joblib @ file:///home/conda/feedstock_root/build_artifacts/joblib_1612898609989/work json5 @ file:///home/conda/feedstock_root/build_artifacts/json5_1600692310011/work jsonschema @ file:///home/conda/feedstock_root/build_artifacts/jsonschema_1614815863336/work jupyter @ file:///Users/runner/miniforge3/conda-bld/jupyter_1611871926923/work jupyter-client @ file:///home/conda/feedstock_root/build_artifacts/jupyter_client_1632841627383/work jupyter-console @ file:///home/conda/feedstock_root/build_artifacts/jupyter_console_1616560109969/work jupyter-core==4.8.1 jupyter-server @ file:///home/conda/feedstock_root/build_artifacts/jupyter_server_1631218027728/work jupyterlab @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_1631631911084/work jupyterlab-pygments==0.1.2 jupyterlab-server @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_server_1632590716858/work jupyterlab-widgets @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_widgets_1631590465624/work kiwisolver @ file:///Users/runner/miniforge3/conda-bld/kiwisolver_1630173773255/work mapclassify @ file:///home/conda/feedstock_root/build_artifacts/mapclassify_1627363180059/work MarkupSafe @ file:///Users/runner/miniforge3/conda-bld/markupsafe_1621455730541/work matplotlib @ file:///Users/runner/miniforge3/conda-bld/matplotlib-suite_1632416605047/work matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1631080358261/work missingno==0.5.0 mistune @ file:///Users/runner/miniforge3/conda-bld/mistune_1624941339085/work multimethod==1.6 munch==2.5.0 nbclassic @ file:///home/conda/feedstock_root/build_artifacts/nbclassic_1631880505492/work nbclient @ file:///home/conda/feedstock_root/build_artifacts/nbclient_1629120697898/work nbconvert @ file:///Users/runner/miniforge3/conda-bld/nbconvert_1632536029521/work nbformat==5.1.3 nest-asyncio @ file:///home/conda/feedstock_root/build_artifacts/nest-asyncio_1617163391303/work networkx @ file:///home/conda/feedstock_root/build_artifacts/networkx_1598210780226/work notebook==6.4.4 numpy @ file:///Users/runner/miniforge3/conda-bld/numpy_1629092076711/work olefile @ file:///home/conda/feedstock_root/build_artifacts/olefile_1602866521163/work packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1625323647219/work pandas==1.3.3 pandas-profiling==3.1.0 pandocfilters @ file:///home/conda/feedstock_root/build_artifacts/pandocfilters_1631603243851/work parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1617148930513/work pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1602535608087/work phik==0.12.0 pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work Pillow @ file:///Users/runner/miniforge3/conda-bld/pillow_1630696651849/work prometheus-client @ file:///home/conda/feedstock_root/build_artifacts/prometheus_client_1622586138406/work prompt-toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1629903925368/work protobuf==3.18.0 ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl pyarrow==5.0.0 pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1593275161868/work pydantic==1.8.2 pydeck @ file:///home/conda/feedstock_root/build_artifacts/pydeck_1630093567483/work Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1629119114968/work pyOpenSSL @ file:///home/conda/feedstock_root/build_artifacts/pyopenssl_1608055815057/work pyparsing==2.4.7 pyproj @ file:///Users/runner/miniforge3/conda-bld/pyproj_1632526447121/work PyQt5==5.12.3 PyQt5_sip==4.19.18 PyQtChart==5.12 PyQtWebEngine==5.12.1 pyrsistent @ file:///Users/runner/miniforge3/conda-bld/pyrsistent_1624984762901/work PySocks @ file:///Users/runner/miniforge3/conda-bld/pysocks_1610291470465/work python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1626286286081/work pytz @ file:///home/conda/feedstock_root/build_artifacts/pytz_1612179539967/work PyWavelets==1.1.1 PyYAML==5.4.1 pyzmq @ file:///Users/runner/miniforge3/conda-bld/pyzmq_1631793531263/work qtconsole @ file:///home/conda/feedstock_root/build_artifacts/qtconsole_1625023841584/work QtPy @ file:///home/conda/feedstock_root/build_artifacts/qtpy_1632440819550/work requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1626393743643/work requests-unixsocket==0.2.0 Rtree @ file:///Users/runner/miniforge3/conda-bld/rtree_1626987867021/work s3fs==0.4.2 s3transfer @ file:///home/conda/feedstock_root/build_artifacts/s3transfer_1626384238958/work scikit-learn @ file:///Users/runner/miniforge3/conda-bld/scikit-learn_1632611470401/work scipy @ file:///Users/runner/miniforge3/conda-bld/scipy_1628206569514/work seaborn==0.11.2 Send2Trash @ file:///home/conda/feedstock_root/build_artifacts/send2trash_1628511208346/work Shapely @ file:///Users/runner/miniforge3/conda-bld/shapely_1622660863030/work six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work smmap==4.0.0 sniffio @ file:///Users/runner/miniforge3/conda-bld/sniffio_1610318427209/work SQLAlchemy @ file:///Users/runner/miniforge3/conda-bld/sqlalchemy_1629315197500/work streamlit==0.89.0 tabulate @ file:///home/conda/feedstock_root/build_artifacts/tabulate_1614001031686/work tangled-up-in-unicode==0.1.0 terminado @ file:///Users/runner/miniforge3/conda-bld/terminado_1631192978302/work testpath @ file:///home/conda/feedstock_root/build_artifacts/testpath_1621261527237/work threadpoolctl @ file:///home/conda/feedstock_root/build_artifacts/threadpoolctl_1626092076920/work toml==0.10.2 toolz @ file:///home/conda/feedstock_root/build_artifacts/toolz_1600973991856/work tornado @ file:///Users/runner/miniforge3/conda-bld/tornado_1625488906146/work tqdm==4.62.3 traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1630423529112/work typing-extensions==3.10.0.2 tzlocal==3.0 urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1632350318291/work validators==0.18.2 vega-datasets @ file:///home/conda/feedstock_root/build_artifacts/vega_datasets_1606414058616/work visions==0.7.4 wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1600965781394/work webencodings==0.5.1 websocket-client @ file:///Users/runner/miniforge3/conda-bld/websocket-client_1610127894508/work widgetsnbextension @ file:///Users/runner/miniforge3/conda-bld/widgetsnbextension_1605475534277/work zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1625284368454/work


</p>
</details>


**Additional context**

<!--
Add any other context about the problem here.
-->

kovar-ursa avatar Sep 28 '21 20:09 kovar-ursa

A PR with a solution is very much welcome (e.g. add a suffix to the added column, or not adding the column in this case).

sbrugman avatar Oct 02 '21 17:10 sbrugman

Unfortunately, after encountering two fatal errors when trying to use the package and thus getting no useful information out of it, I am disinclined to invest more time in reproducing either error.

kovar-ursa avatar Oct 02 '21 19:10 kovar-ursa

Sure. The issue is open for anyone from the community that's interested in taking it on.

sbrugman avatar Oct 02 '21 20:10 sbrugman