Dataset problem

[emigmo]

Hi, Tianhe There are some problems with the dataset from http://rail.eecs.berkeley.edu/datasets/mil_data.zip I use the following code to check the dataset of './data/sim_push'

import glob
import pickle
file_dir = './data/sim_push'
file_list = glob.glob(file_dir + "/*.pkl")
bad_file = []
for i in range(len(file_list)):
    try:
        with open(file_list[i], 'rb') as f:
        data = pickle.load(f)
    except:
        bad_file.append(file_list[i])
print("open(file_list): bad_file: ", len(bad_file))
print(bad_file)

There are 78 files which can't be loaded using pickle. If I change the open(file_list[i], 'rb') as open(file_list[i]), there would be 753 files which can't be loaded (all the files can't be loaded normally).

I use python 2.7.6 and python 3.6.3 to load your pickle file, but get the same problem For python 2.7.6: I get the print(pickle.format_version) --> 2.0 For python 3.6.3: I get the print(pickle.format_version) --> 4.0

I guess that the problem may be the version of the pickle package. Could you tell me which version of pickle you used when you dump the pickle file?

[tianheyu927]

What's the error message when loading the pickle files failed?

[emigmo]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa8 in position 269495: invalid start byte

[emigmo]

Any idea here? or redump and upload your mil_data.zip and tell me your pickle's version.

[raozhongyu]

Do you solve this problem?

[igiorgi]

try this

results = pickle.load(f, encoding="latin1")