libraries.io icon indicating copy to clipboard operation
libraries.io copied to clipboard

Documentation about importing libraries.io Open Data

Open besendorf opened this issue 3 years ago • 0 comments

I was trying to import the libraries.io Open Data into a database of a libraries.io instance and ran into a few issues. I think documenting how to import the data would be helpful. For example I had this issue:

--command " "\\copy public.projects (id, name, platform, created_at, updated_at, description, keywords, homepage, licenses, repository_url, repository_id, normalized_licenses, versions_count, rank, latest_release_published_at, latest_release_number, pm_id, keywords_array, dependents_count, language, status, last_synced_at, dependent_repos_count, runtime_dependencies_count, score, score_last_calculated, latest_stable_release_number, latest_stable_release_published_at, license_set_by_admin, license_normalized, deprecation_reason) FROM 'X:/libraries-1.6.0-2020-01-12.tar/libraries-1.6.0-2020-01-12/libraries-1.6.0-2020-01-12/projects-1.6.0-2020-01-12.csv' DELIMITER ',' CSV HEADER QUOTE '\"' ESCAPE '''';""
ERROR: malformed array literal: »5«
DETAIL: Array value must start with \"{\" or dimension information.
CONTEXT:  COPY projects, line 2, columnnormalized_licenses: »5«

Edit: I realized the problem is that the csv is in different order than the SQL Table. Do you know why that is? Also the Names of the columns are different.

besendorf avatar May 25 '21 15:05 besendorf