isds2020 icon indicating copy to clipboard operation
isds2020 copied to clipboard

Problems with pd.read.csv

Open SofusBang opened this issue 5 years ago • 3 comments
trafficstars

Hi everyone I am working on the weather data assignment, with importing weather data from 1864. I succeed in importing the data, and unzipping it using compression='gzip'

I think there is two problems:

  1. The log command returns an error, thus suggesting that my data is not in the right format. I can use the log command on other data frames, so it is not the command itself

  2. The observations I present using the iloc command does not correspond with the prober result, thus suggesting that there is something wrong with the ordering of the data

I post my syntax below

hederlist=['A', 'B','C','D','E','F','G','H',]
df_weather=pd.read_csv('ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/by_year/1864.csv.gz',names= hederlist,compression='gzip')
df_weather.head(n=10)
print(len(df_weather))
print(df_weather.iloc[-1:,0].values[0])

len=31921 df_weather.iloc[-1:,0].values[0]=USP00CA0003

the error with the iloc command ends with: TypeError: cannot do label indexing on with these indexers [0] of

Does anyone know how to solve this problem? Thank you in advance

SofusBang avatar Jul 23 '20 11:07 SofusBang

Hi @SofusBang, when did you clone the repository? Have you seen this issue #1?

jsr-p avatar Jul 23 '20 20:07 jsr-p

Hi @jsr-p

I tried to clone it yesterday this solved the problem of ‘iloc’ nor working. I still get different results when I run the assertion commands. Maybe it is the ordering of the data, because its seems like my data is ordered by date, and then observation alphabetically This is my syntax import pandas as pd import numpy as np

my_url='ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/by_year/1864.csv.gz'
df_weather=pd.read_csv(my_url, compression='gzip', header=None)
print(df_weather.head(n=70))
print(len(df_weather))

print(df_weather.loc[0,0]) print(df_weather.iloc[-1:,0].values[0])

And this is my output billede

Thank you in advance!

SofusBang avatar Jul 24 '20 07:07 SofusBang

hi @SofusBang , I am not sure if I understand the problem. I get the same output when running your code and none of the assert statements raise an error. What error is it specifically that you get from the assert statements? as_

jsr-p avatar Jul 24 '20 09:07 jsr-p