Cloudlog icon indicating copy to clipboard operation
Cloudlog copied to clipboard

file manager: add a file manager layer to handle qsl card storage & etc

Open samhjn opened this issue 3 years ago • 17 comments

Cherry pick works of file manager on 1.0 branch to 2.0

Note: There is a bug in original code that some of qsl model interface privilege checker mixed qsl_id with qso_id. I modified them to let the interface work, but there obviously need another patch to fix it.

samhjn avatar Feb 05 '22 15:02 samhjn

There is a problem that the file name generator may produce same random at a chance of 1/65536 if both back & front are uploaded. A bigger random length may help reduce the chance of collision.

samhjn avatar Feb 05 '22 15:02 samhjn

@samhjn thanks for the PR. I've tested it, and as far as I can tell, it works as it should with no errors. I've only tested it with storing QSLs on the same server (my local machine). The migration script also takes care of the existing QSLs, so they are in place after migration (at least with my test).

AndreasK79 avatar Feb 05 '22 16:02 AndreasK79

There is a problem that the file name generator may produce same random at a chance of 1/65536 if both back & front are uploaded. A bigger random length may help reduce the chance of collision.

I'd suggest we implement this to stop any chance!

magicbug avatar Feb 06 '22 21:02 magicbug

There is a problem that the file name generator may produce same random at a chance of 1/65536 if both back & front are uploaded. A bigger random length may help reduce the chance of collision.

I'd suggest we implement this to stop any chance!

I came up with serveral solution for this:

  1. extend the random str length to 8 or more, which can lower the collision chance to less than 1/4billion, which seems enough in many cases.
  2. add qslfront/qslback to filename, which can avoid collision in normal use case, but it can not prevent bad case created by automatic scripts posting requests too frequent.
  3. check s3 file existance before write, for s3 does not provide atomic check before write apis, it may also fail on some automatic scripts.
  4. modify the filename with $file_id, it will insert mysql first to get a $file_id, a transaction may need here to handle file upload failure.

I'd prefer 1 or 4, which do you prefer?

samhjn avatar Feb 07 '22 13:02 samhjn

Option 1 seems reasonable :)

magicbug avatar Feb 07 '22 18:02 magicbug

There is a problem that the file name generator may produce same random at a chance of 1/65536 if both back & front are uploaded. A bigger random length may help reduce the chance of collision.

I'd suggest we implement this to stop any chance!

now length=10 was set for the random hex

samhjn avatar Feb 08 '22 17:02 samhjn

Okay on not having the views in place, however for most users they will be a much-needed item so hopefully, we can work together and finish those parts.

It could be that it could be stored within Global Settings under "Admin"

Okay, I will work on it later... The weekend time is precious 🥲 , and I may have to take some time to repick my memory of jQuery.ajax and PHP request handling, I may need some time...

Thank you

samhjn avatar Feb 09 '22 17:02 samhjn

Sorry for the delay of the frontend of this feature. Now I finished a version that is able to run end to end on former branch. I cherry-picked the code to recent branch to carry out more test, but I had trouble to run the current branch. It seems the install process failed on the migration procedure. I recieved error message below:

A Database Error Occurred Error Number: 1146 Table 'cloudlog.options' doesn't exist SELECT * FROM options WHERE autoload = 'yes' Filename: D:/sdk/xampp/htdocs/Cloudlog/system/database/DB_driver.php Line Number: 656

Could anyone reproduce this problem?

samhjn avatar Apr 17 '22 17:04 samhjn

@AndreasK79 @magicbug

samhjn avatar Apr 19 '22 16:04 samhjn

Did you run this code against master? I know that it does not work with PHP8. If not, then I'm not sure what is causing this.

I'll see if I can give this PR a test run one of these days.

AndreasK79 avatar Apr 20 '22 20:04 AndreasK79

Did you run this code against master? I know that it does not work with PHP8. If not, then I'm not sure what is causing this.

I'll see if I can give this PR a test run one of these days.

No, I run it with station_logbooks. It fails during installation.

samhjn avatar Apr 22 '22 15:04 samhjn

That is strange. I just tried this branch, and it fails with the same problem here.

AndreasK79 avatar Apr 22 '22 16:04 AndreasK79

I found the problem. It seems to me that you based the code on a broken commit of station_logbooks. You need to edit 073_create_station_logbook_relationship_table.php and rename the class from Migration_create_station_logbook_relationship_table2 to Migration_create_station_logbook_relationship_table.

AndreasK79 avatar Apr 22 '22 17:04 AndreasK79

I found the problem. It seems to me that you based the code on a broken commit of station_logbooks. You need to edit 073_create_station_logbook_relationship_table.php and rename the class from Migration_create_station_logbook_relationship_table2 to Migration_create_station_logbook_relationship_table.

You are right, it seems that I submitted some local fix of station_logbooks branch few months ago. Tests are carrying out now.

samhjn avatar Apr 22 '22 17:04 samhjn

@AndreasK79 @magicbug I think the frontend now is able to use. If there is any other things to do before the pull request being accepted?

samhjn avatar May 01 '22 09:05 samhjn

@samhjn sorry for not getting back sooner. I'll take a look this weekend.

AndreasK79 avatar May 28 '22 08:05 AndreasK79

@magicbug did you check this out? Should we get this in v2 branch?

AndreasK79 avatar Sep 19 '22 17:09 AndreasK79