sato icon indicating copy to clipboard operation
sato copied to clipboard

Directory Error

Open Spruces opened this issue 4 years ago • 7 comments

Hello, I tried to extract the features with extract_features.py

image and the result is just like this.

I don't get where the open_data_type_78_header_valid.csv's location is.

  • Can I just use this model as easier as Sato Web Demo, just upload and get return value?

Too complicate to use, and not much instructions

Spruces avatar Oct 09 '20 01:10 Spruces

Same error occurs to me. I think the solution is to understand the usage of Viznet, because there is similar code in it. https://github.com/mitmedialab/viznet

jason022085 avatar Mar 17 '21 09:03 jason022085

Same error occurs to me. I think the solution is to understand the usage of Viznet, because there is similar code in it. https://github.com/mitmedialab/viznet

Thanks bro, it's weird. To execute it, you need certain .csv files(like open_data_type_78_header_valid.csv) that doesn't provided by sato.

and I had examined the codes, you need certain type of NLTKs to generate open_data_type_78_header_valid.csv.

Also, after I found certain types of NLTKs, I couldn't figure out where exactly the NLTK should be located in certain directory.

Poor guidelines.

Spruces avatar Mar 17 '21 10:03 Spruces

And how they can detect semantic columns with NLTKs?

Like my ID, and yours. They don't have any semantic things as they are just nouns.

I think that for detecting 'ID' columns based on our id(Spruces,jason022085) they need something else like the lengths of inputs or how many blanks they had, and the number of separator comma(address).

cus there's no meaning in our names of addresses as themselves.

Spruces avatar Mar 17 '21 10:03 Spruces

p.s. Remember to download VizNet data first(produced by retrieve_corpora.sh) and set RAW_DIR to the path of raw data form Viznet In my case, it is os.environ['RAW_DIR'] = r"D:\viznet-master\raw"

Since other dataset are too large, I am working on "manyeyes" dataset. image

And I found there is a extract_header.py. It may help image

I got it ! You have to extract header first, and then extract features image

jason022085 avatar Mar 19 '21 02:03 jason022085

Hello, from what I understand you can download open_data_type_78_header_valid.csv and the sherlocks features they used using their script ./download_data.sh

RichaMax avatar Mar 24 '21 10:03 RichaMax

Sorry about the confusion. If you wish to extract the features, you'll first need to run extract_header.py to get the valid headers (in the 78 types after canonicalization) from the dataset.

horseno avatar Mar 24 '21 22:03 horseno

I have empty header files generated when I try extract_headers.py for manyeyes. Can you please help? What it might be. Please let me know

iganand avatar Sep 09 '21 16:09 iganand