tus-php icon indicating copy to clipboard operation
tus-php copied to clipboard

Cache file not working on windows

Open gavrochelegnou opened this issue 2 years ago • 9 comments

When using the FileStore Cache on windows, the file can't be written due to the lock.

If i skip the lock or use LOCK_UN on the lock function and then use LOCK_EX on the put() call then it works

Thanks a lot

Notice:  file_put_contents(): write of 549 bytes failed with errno=13 Permission  denied in  C:\laragon\www\filmfest\vendor\ankitpokhrel\tus-php\src\Cache\FileStore.php  on line 191
--
1 | 0.0004 | 412504 | {main}(  ) | ...\index.php:0
2 | 0.0332 | 2608360 | TusPhp\Tus\Server->serve(  ) | ...\index.php:22
3 | 0.0332 | 2608840 | TusPhp\Tus\Server->handlePost(  ) | ...\Server.php:276
4 | 0.0773 | 5572976 | TusPhp\Cache\FileStore->set( $key = 'add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f', $value = ['name'  => 'VID-20220325-WA0010 (1).mp4', 'size' => 3142632, 'offset'  => 0, 'checksum' => '', 'location' =>  'http://filmfest.test/uploads//add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f',  'file_path' =>  'C:\\laragon\\www\\filmfest\\uploads\\files/VID-20220325-WA0010  (1).mp4', 'metadata' => ['name' => 'VID-20220325-WA0010 (1).mp4',  'type' => 'video/mp4', 'filetype' => 'video/mp4', 'filename' =>  'VID-20220325-WA0010 (1).mp4'], 'created_at' => 'Tue, 12 Apr 2022  10:22:20 GMT', 'expires_at' => 'Tue, 12 Apr 2022 11:22:20 GMT',  'upload_type' => 'normal'] ) | ...\Server.php:379
5 | 0.0787 | 5573832 | TusPhp\Cache\FileStore->lock( $path = 'C:\\laragon\\www\\filmfest\\uploads\\files.json', $type = 2, $cb = class  Closure { virtual $closure = "$this->TusPhp\Cache\{closure}", public  $static = ['cacheKey' =>  'tus:server:add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f', 'cacheFile' =>  'C:\\laragon\\www\\filmfest\\uploads\\files.json', 'value' => [...]];  public $this = class TusPhp\Cache\FileStore { protected $cacheDir =  'C:\\laragon\\www\\filmfest\\uploads\\'; protected $cacheFile =  'files.json'; protected $ttl = 3600; protected $prefix = 'tus:server:'  }; public $parameter = ['$handle' => '<required>'] } ) | ...\FileStore.php:217
6 | 0.0789 | 5574208 | TusPhp\Cache\FileStore->TusPhp\Cache\{closure:C:\laragon\www\filmfest\vendor\ankitpokhrel\tus-php\src\Cache\FileStore.php:206-217}(  $handle = resource(143) of type (stream) ) | ...\FileStore.php:150
7 | 0.0792 | 5586920 | TusPhp\Cache\FileStore->put( $path = 'C:\\laragon\\www\\filmfest\\uploads\\files.json', $contents = '{"tus:server:add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f":{"name":"VID-20220325-WA0010   (1).mp4","size":3142632,"offset":0,"checksum":"","location":"http:\\/\\/filmfest.test\\/uploads\\/\\/add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f","file_path":"C:\\\\laragon\\\\www\\\\filmfest\\\\uploads\\\\files\\/VID-20220325-WA0010  (1).mp4","metadata":{"name":"VID-20220325-WA0010  (1).mp4","type":"video\\/mp4","filetype":"video\\/mp4","filename":"VID-20220325-WA0010  (1).mp4"},"created_at":"Tue, 12 Apr 2022 10:22:20 GMT","expires_at":"', $lock = 0 ) | ...\FileStore.php:216
8 | 0.0792 | 5586920 | file_put_contents( $filename = 'C:\\laragon\\www\\filmfest\\uploads\\files.json', $data = '{"tus:server:add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f":{"name":"VID-20220325-WA0010   (1).mp4","size":3142632,"offset":0,"checksum":"","location":"http:\\/\\/filmfest.test\\/uploads\\/\\/add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f","file_path":"C:\\\\laragon\\\\www\\\\filmfest\\\\uploads\\\\files\\/VID-20220325-WA0010  (1).mp4","metadata":{"name":"VID-20220325-WA0010  (1).mp4","type":"video\\/mp4","filetype":"video\\/mp4","filename":"VID-20220325-WA0010  (1).mp4"},"created_at":"Tue, 12 Apr 2022 10:22:20 GMT","expires_at":"', $flags = 0 ) | ...\FileStore.php:191

Warning:  file_put_contents(): Only -1 of 549 bytes written, possibly out of free  disk space in  C:\laragon\www\filmfest\vendor\ankitpokhrel\tus-php\src\Cache\FileStore.php  on line 191
--
1 | 0.0004 | 412504 | {main}(  ) | ...\index.php:0
2 | 0.0332 | 2608360 | TusPhp\Tus\Server->serve(  ) | ...\index.php:22
3 | 0.0332 | 2608840 | TusPhp\Tus\Server->handlePost(  ) | ...\Server.php:276
4 | 0.0773 | 5572976 | TusPhp\Cache\FileStore->set( $key = 'add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f', $value = ['name'  => 'VID-20220325-WA0010 (1).mp4', 'size' => 3142632, 'offset'  => 0, 'checksum' => '', 'location' =>  'http://filmfest.test/uploads//add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f',  'file_path' =>  'C:\\laragon\\www\\filmfest\\uploads\\files/VID-20220325-WA0010  (1).mp4', 'metadata' => ['name' => 'VID-20220325-WA0010 (1).mp4',  'type' => 'video/mp4', 'filetype' => 'video/mp4', 'filename' =>  'VID-20220325-WA0010 (1).mp4'], 'created_at' => 'Tue, 12 Apr 2022  10:22:20 GMT', 'expires_at' => 'Tue, 12 Apr 2022 11:22:20 GMT',  'upload_type' => 'normal'] ) | ...\Server.php:379
5 | 0.0787 | 5573832 | TusPhp\Cache\FileStore->lock( $path = 'C:\\laragon\\www\\filmfest\\uploads\\files.json', $type = 2, $cb = class  Closure { virtual $closure = "$this->TusPhp\Cache\{closure}", public  $static = ['cacheKey' =>  'tus:server:add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f', 'cacheFile' =>  'C:\\laragon\\www\\filmfest\\uploads\\files.json', 'value' => [...]];  public $this = class TusPhp\Cache\FileStore { protected $cacheDir =  'C:\\laragon\\www\\filmfest\\uploads\\'; protected $cacheFile =  'files.json'; protected $ttl = 3600; protected $prefix = 'tus:server:'  }; public $parameter = ['$handle' => '<required>'] } ) | ...\FileStore.php:217
6 | 0.0789 | 5574208 | TusPhp\Cache\FileStore->TusPhp\Cache\{closure:C:\laragon\www\filmfest\vendor\ankitpokhrel\tus-php\src\Cache\FileStore.php:206-217}(  $handle = resource(143) of type (stream) ) | ...\FileStore.php:150
7 | 0.0792 | 5586920 | TusPhp\Cache\FileStore->put( $path = 'C:\\laragon\\www\\filmfest\\uploads\\files.json', $contents = '{"tus:server:add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f":{"name":"VID-20220325-WA0010   (1).mp4","size":3142632,"offset":0,"checksum":"","location":"http:\\/\\/filmfest.test\\/uploads\\/\\/add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f","file_path":"C:\\\\laragon\\\\www\\\\filmfest\\\\uploads\\\\files\\/VID-20220325-WA0010  (1).mp4","metadata":{"name":"VID-20220325-WA0010  (1).mp4","type":"video\\/mp4","filetype":"video\\/mp4","filename":"VID-20220325-WA0010  (1).mp4"},"created_at":"Tue, 12 Apr 2022 10:22:20 GMT","expires_at":"', $lock = 0 ) | ...\FileStore.php:216
8 | 0.0792 | 5586920 | file_put_contents( $filename = 'C:\\laragon\\www\\filmfest\\uploads\\files.json', $data = '{"tus:server:add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f":{"name":"VID-20220325-WA0010   (1).mp4","size":3142632,"offset":0,"checksum":"","location":"http:\\/\\/filmfest.test\\/uploads\\/\\/add7ddc4-12a0-4b60-b8ae-2efa5fc32d5f","file_path":"C:\\\\laragon\\\\www\\\\filmfest\\\\uploads\\\\files\\/VID-20220325-WA0010  (1).mp4","metadata":{"name":"VID-20220325-WA0010  (1).mp4","type":"video\\/mp4","filetype":"video\\/mp4","filename":"VID-20220325-WA0010  (1).mp4"},"created_at":"Tue, 12 Apr 2022 10:22:20 GMT","expires_at":"', $flags = 0 ) | ...\FileStore.php:191

gavrochelegnou avatar Apr 12 '22 11:04 gavrochelegnou

I think the problem is that the file is locked and file_put_contents doesn't use the same $handle so it can't write the file.

The solutions could be :

  • using fwrite instead of file_put_contents and passing the $handle from the lock to the put() function
  • Or unlocking the file before the file_put_contents and locking it again via LOCK_EX in the file_put_contents

gavrochelegnou avatar Apr 12 '22 12:04 gavrochelegnou

@gavrochelegnou maybe you can use RedisStore instead? I also faced some problems with the FileStore.

jmglsn avatar Apr 13 '22 07:04 jmglsn

@gavrochelegnou Not completely sure if this is the same issue but could you please check if https://github.com/ankitpokhrel/tus-php/pull/383 fixes your issue? Also as @jmglsn mentioned, please use RedisStore in production. FileStore was initially made for dev purposes and is not completely reliable.

ankitpokhrel avatar Apr 21 '22 06:04 ankitpokhrel

Hello @ankitpokhrel ,

I confirm that uploads are workling fine on windows with this fork thide11:fix-windows-lock-files

The problem with redistore is that many shared hosting do not provide a Redis server.

Would an Sqlite version solve some of the problems related to FileStore ?

Thanks !

gavrochelegnou avatar Apr 21 '22 07:04 gavrochelegnou

@gavrochelegnou yes SQLite should be fine. You will have to implement a cache adapter for that tho. Contributions are welcome!

ankitpokhrel avatar Apr 21 '22 18:04 ankitpokhrel

@gavrochelegnou could you please check if you still get Permission denied issue in patch/issue-387 branch (#390). I don't have windows machine to test but I think the issue is because of the stat cache.

ankitpokhrel avatar Apr 21 '22 19:04 ankitpokhrel

Hi,

patch/issue-387 does not work on windows and triggers the same errors as before.

I think the problem really is that file_put_contents is not using the same $handle used in the flock so the file is locked and can't be written.

I'll try to work on an implementation of a DB Cache via PDO to support SQLite, MySQL, ...

For the FileStore maybe we would have much less concurency problems if there were one cache file per key instead of a single file ? The cache structure could be something like cache/{Prefix}/{key}.json ?

gavrochelegnou avatar Apr 22 '22 07:04 gavrochelegnou

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Jun 22 '22 00:06 github-actions[bot]

@gavrochelegnou Thanks for the fix you provided here. It solves the described error on my local Windows machine.

spiess-demos avatar Jun 27 '22 10:06 spiess-demos

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Aug 27 '22 00:08 github-actions[bot]

Warning: file_put_contents(): Only -1 of 549 bytes written, possibly out of free disk space in C:\laragon\www\filmfest\vendor\ankitpokhrel\tus-php\src\Cache\FileStore.php on line 191

jessice-ai avatar Aug 30 '22 06:08 jessice-ai

#383 is merged and is available in the new release v2.3.0

ankitpokhrel avatar Sep 18 '22 09:09 ankitpokhrel