strfry icon indicating copy to clipboard operation
strfry copied to clipboard

Can't import data in new version

Open psic4t opened this issue 1 year ago • 2 comments

I tried to upgrade the db to v3 and followed the instructions, but I'm unable to import it back again.

I used:

docker run -v "/opt/containers/strfry/strfry.conf:/etc/strfry.conf" -v "/opt/containers/strfry/strfry-db:/app/strfry-db" dockurr/strfry /app/strfry import --fried < dbdump.jsonl

but nothing gets imported. Output from above command is:

date       time         ( uptime  ) [ thread name/id ]   v|
2024-09-14 09:28:48.757 (   0.050s) [main thread     ]INFO| arguments: /app/strfry import
2024-09-14 09:28:48.757 (   0.050s) [main thread     ]INFO| Current dir: /app
2024-09-14 09:28:48.757 (   0.050s) [main thread     ]INFO| stderr verbosity: 0
2024-09-14 09:28:48.757 (   0.050s) [main thread     ]INFO| -----------------------------------
2024-09-14 09:28:48.757 (   0.050s) [main thread     ]INFO| CONFIG: Loading config from file: /etc/strfry.conf
2024-09-14 09:28:48.769 (   0.062s) [main thread     ]INFO| CONFIG: successfully installed
2024-09-14 09:28:48.770 (   0.063s) [main thread     ]INFO| Done. Processed 0 lines. 0 added, 0 rejected, 0 dups
2024-09-14 09:28:48.770 (   0.063s) [main thread     ]INFO| atexit

dbdump.jsonl seems intact and is over 10 GBytes.

Any ideas?

psic4t avatar Sep 14 '24 09:09 psic4t

Hmmm I'm not sure what could be the problem here. Can you paste the first line from dbdump.jsonl (or join our telegram and you can DM it to me).

I'm not too familiar with the docker invocation: Are you sure stdin is getting hooked up correctly? Also what strfry version are you using to import?

hoytech avatar Sep 18 '24 16:09 hoytech

Hello Doug,

one year later I still have the same problem. export with --fried was no problem, but the import does not work.

I tried from inside the container:

app # ./strfry import --fried strfry-new/backup.jsonl
date       time         ( uptime  ) [ thread name/id ]   v|
2025-10-31 09:49:12.673 (   0.038s) [main thread     ]INFO| arguments: ./strfry import --fried strfry-new/backup.jsonl
2025-10-31 09:49:12.673 (   0.038s) [main thread     ]INFO| Current dir: /app
2025-10-31 09:49:12.673 (   0.038s) [main thread     ]INFO| stderr verbosity: 0
2025-10-31 09:49:12.673 (   0.038s) [main thread     ]INFO| -----------------------------------
2025-10-31 09:49:12.673 (   0.038s) [main thread     ]INFO| CONFIG: Loading config from file: /etc/strfry.conf
2025-10-31 09:49:12.687 (   0.052s) [main thread     ]INFO| CONFIG: successfully installed
Unexpected argument: --fried, strfry-new/backup.jsonl

    Usage:
      import [--show-rejected] [--no-verify] [--debounce-millis=<debounce-millis>] [--write-batch=<write-batch>] [--fried]

Unexpected argument? Why?

This is the first line of the JSONL dump:

triton [/mnt/strfry-new]# head -n 1 backup.jsonl
{"content":"No kidding, this is a huge game changer. I have been saying I think we will keep seeing these as platforms get
replaced one by one. I'm really looking forward to NostrTube. Need a \"pay to upload\" video relay that provides high quality
streaming at different
resolutions.","created_at":1676899613,"id":"dc345452506107cf780c66e40d17a36904221587dc82cd38b331562bb4be577f","kind":1,"pubkey":"d
f4cafee85f79769545851db202a8856f82dc917548093c760f3094896e987b2","sig":"575c4ebf1bcf5222b075c5f333770eda1b5ebaaab0881f87a99a1aaf01
6dddb8e5fde8e4c463086003841847d02acbda63cae554b5dea5f68c0bc4f4fe49d81e","tags":[["e","3b346ee4d5a41b15c27edd6acdc72fdf3fc9f5974296
36057b57366db89fca40","","root"],["e","083d137ccd66a3adf8451a4ad7eacb7a29b48c5b6391c633590d8324b6afb07e","","reply"],["p","7fa56f5
d6962ab1e3cd424e758c3002b8665f7b0d8dcee9fe9e288d7751ac194"],["p","e88a691e98d9987c964521dff60025f60700378a4879180dcbbb4a5027850411
"]],"fried":"dc345452506107cf780c66e40d17a36904221587dc82cd38b331562bb4be577fdf4cafee85f79769545851db202a8856f82dc917548093c760f30
94896e987b21d75f363000000000100000000000000000000000000000065203b346ee4d5a41b15c27edd6acdc72fdf3fc9f597429636057b57366db89fca40652
0083d137ccd66a3adf8451a4ad7eacb7a29b48c5b6391c633590d8324b6afb07e70207fa56f5d6962ab1e3cd424e758c3002b8665f7b0d8dcee9fe9e288d7751ac
1947020e88a691e98d9987c964521dff60025f60700378a4879180dcbbb4a5027850411"}

When I try to import it from outside the container:

docker run --log-opt max-size=10m -v /opt/containers/strfry/strfry-db:/app/strfry-db -v
/opt/containers/strfry/strfry.conf:/etc/strfry.conf -v /mnt/strfry-new:/app/strfry-new ghcr.io/hoytech/strfry:latest import
--fried < /mnt/strfry-new/backup.jsonl

it just prints this:

date       time         ( uptime  ) [ thread name/id ]   v|
2025-10-31 09:54:31.253 (   0.036s) [main thread     ]INFO| arguments: /app/strfry import --fried
2025-10-31 09:54:31.253 (   0.036s) [main thread     ]INFO| Current dir: /app
2025-10-31 09:54:31.253 (   0.036s) [main thread     ]INFO| stderr verbosity: 0
2025-10-31 09:54:31.253 (   0.036s) [main thread     ]INFO| -----------------------------------
2025-10-31 09:54:31.253 (   0.036s) [main thread     ]INFO| CONFIG: Loading config from file: /etc/strfry.conf
2025-10-31 09:54:31.264 (   0.047s) [main thread     ]INFO| CONFIG: successfully installed
2025-10-31 09:54:31.265 (   0.049s) [main thread     ]INFO| Done. Processed 0 lines. 0 added, 0 rejected, 0 dups
2025-10-31 09:54:31.266 (   0.049s) [main thread     ]INFO| atexit

Any ideas?

Thanks in advance,

Matthias

psic4t avatar Oct 31 '25 10:10 psic4t