datapusher-plus icon indicating copy to clipboard operation
datapusher-plus copied to clipboard

Fix freq table

Open avdata99 opened this issue 2 months ago • 7 comments

Error with frquency table

Could not copy frequency data to database:
extra data after last expected column
CONTEXT: COPY 737a3aad-c1c8-4507-8411-2dc6ca176f84-druf-freq,
line 2: "country,Afghanistan,6,2.52101,1"

They are 5 fields and the freq table only have 4

cur_statsfreq.execute(
        sql.SQL(
            """
            DROP TABLE IF EXISTS {};
            CREATE TABLE {} (
                field TEXT,
                value TEXT,
                count INTEGER,
                percentage FLOAT,
                PRIMARY KEY (field, value, count)
            )
        """
        ).format(freq_table, freq_table)

Temporary we add a text field just to continue but probably there is another correct answer

avdata99 avatar Oct 29 '25 14:10 avdata99

hey @avdata99 looking in your case text is ok, but i think we will need rank as FLOAT Would you like to update this PR ?

tino097 avatar Nov 05 '25 20:11 tino097

@tino097 PR updated here https://github.com/dathere/datapusher-plus/pull/255/commits/60d9d7306f22bcba307c9f7a28bb350d7e228553

avdata99 avatar Nov 06 '25 16:11 avdata99

can you please resolve the conflict too ?@avdata99

tino097 avatar Nov 06 '25 16:11 tino097

@tino097 it looks like this PR is no longer valid for main. The base code no longer exist. We are based on the 2.0.0 tag and we create PRs to main from changes in use in our custom branch based on 2.0.0. If you release a new version closed to main, we can move to it and our changes will be close to main

avdata99 avatar Nov 07 '25 12:11 avdata99

Please, feel free to close this if you consider this change is no longer valid @tino097

avdata99 avatar Nov 07 '25 12:11 avdata99

Thanks for this @avdata99. Just wanted to let you know that we'll be making a 3.0.0 release this month as we generalize DP+ and make the DRUF (Data Resource Upload Flow) generally available.

The DRUF is the interactive FAIRification where metadata suggestions are offered to the User based on the qsv-powered analysis of the file when its first uploaded.

jqnatividad avatar Nov 14 '25 11:11 jqnatividad

Thanks @jqnatividad Will DRUF be an optional feature?

avdata99 avatar Nov 19 '25 14:11 avdata99