PixivUtil2 icon indicating copy to clipboard operation
PixivUtil2 copied to clipboard

Filename sanitation didn't work for \

Open Ikkoru opened this issue 11 months ago • 5 comments

Creating directory: Z:\Visual Media\[Pics]\[Artists]\Pixiv\39318908\R-18\Manga\_161553 18 【睡眠】 起きない子
Error at download_image(): Cannot save https://i.pximg.net/img-original/img/2025/04/23/00/00/52/129597353_p0.jpg to Z:\Visual Media\[Pics]\[Artists]\Pixiv\39318908\R-18\Manga\_161553 18 【睡眠】 起きない子 \ OKINAI KO 129597353 起きない子【女教師】 _ OKINAIKO[teacher] 1.jpg: (<class 'FileNotFoundError'>, FileNotFoundError(2, 'No such file or directory'), <traceback object at 0x000001F5E8E9EB80>)

Didn't sanitize the \. It also placed a _ in the beginning of the filename for some reason?

Latest PixivUtil2 version. Windows 11. NTFS.

More info below.

Ikkoru avatar May 08 '25 17:05 Ikkoru

When I added \ to the customBadChars setting, I got the following error:

Creating directory: Z:\Visual Media\[Pics]\[Artists]\Pixiv\_39318908_R-18_Manga_161553 18 【睡眠】 起きない子
Error at download_image(): Cannot save https://i.pximg.net/img-original/img/2025/04/23/00/00/52/129597353_p0.jpg to Z:\Visual Media\[Pics]\[Artists]\Pixiv\_39318908_R-18_Manga_161553 18 【睡眠】 起きない子 \ OKINAI KO 129597353 起きない子【女教師】 _ OKINAIKO[teacher] 1.jpg: (<class 'FileNotFoundError'>, FileNotFoundError(2, 'No such file or directory'), <traceback object at 0x000001A8A195BF80>)

It seems to have replaced ALL \ in my filenameMangaFormat, which is:

%member_id%\%R-18%\Manga\%manga_series_id% %manga_series_order% %manga_series_title% %image_id% %title% %page_number%

Even more info further below.

Ikkoru avatar May 09 '25 20:05 Ikkoru

I also encountered the issue with _ in the filename, but no error was shown in the terminal.

Image

Image

yjlin0224 avatar May 21 '25 17:05 yjlin0224

Testing various filenames (part 1).

filenameMangaFormat = %member_id%\%R-18%\Manga\%manga_series_id% %manga_series_order% %manga_series_title% %image_id% %title% %page_number%. customBadChars = (default).

Series name: /<>:"|*? Filename: \/

Getting Manga Series: 295867 from page: 1
 - Fetching artist details 1514277
Member Url    : https://www.pixiv.net/ajax/user/1514277/profile/all
Processing Image Id: 133383142
Title     : \/
Tags      : AI-generated, フランドール・スカーレット, 東方, 東方Project, 女の子, ロリ, フラン, 少女, 百合, レミリア・スカーレット, レミリア
Date      : 2025-08-01 19:27:00+00:00
Mode      : manga
Bookmarks : 1
Pages     : 2
[1/2] Image URL : https://i.pximg.net/img-original/img/2025/08/02/04/27/34/133383142_p0.png
[1/2] Filename  : Z:\Visual Media\[Pics]\[Artists]\Pixiv\1514277\Manga\_295867 2 \_______ 133383142 __ 1.png
Creating directory: Z:\Visual Media\[Pics]\[Artists]\Pixiv\1514277\Manga\_295867 2
Traceback (most recent call last):
  File "C:\Portable Programs\[Downloaders]\PixivUtil2\PixivImageHandler.py", line 286, in process_image
    (result, filename) = PixivDownloadHandler.download_image(caller,
                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
                                                                img,
                                                                ^^^^
    ...<6 lines>...
                                                                page,
                                                                ^^^^^
                                                                notifier)
                                                                ^^^^^^^^^
  File "C:\Portable Programs\[Downloaders]\PixivUtil2\PixivDownloadHandler.py", line 53, in download_image
    test_utf = open(filename_test + '.test', "wb")
FileNotFoundError: [Errno 2] No such file or directory: 'Z:\\Visual Media\\[Pics]\\[Artists]\\Pixiv\\1514277\\Manga\\_295867 2 \\あいうえお.test'
Error at process_image(): 133383142
Exception: (<class 'FileNotFoundError'>, FileNotFoundError(2, 'No such file or directory'), <traceback object at 0x00000222F369A880>)
Dumping html to: Error medium page for image 133383142.html
Traceback (most recent call last):
  File "C:\Portable Programs\[Downloaders]\PixivUtil2\PixivImageHandler.py", line 505, in process_manga_series
    result = process_image(caller,
                           config,
    ...<10 lines>...
                           manga_series_order=order,
                           manga_series_parent=manga_series)
  File "C:\Portable Programs\[Downloaders]\PixivUtil2\PixivImageHandler.py", line 286, in process_image
    (result, filename) = PixivDownloadHandler.download_image(caller,
                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
                                                                img,
                                                                ^^^^
    ...<6 lines>...
                                                                page,
                                                                ^^^^^
                                                                notifier)
                                                                ^^^^^^^^^
  File "C:\Portable Programs\[Downloaders]\PixivUtil2\PixivDownloadHandler.py", line 53, in download_image
    test_utf = open(filename_test + '.test', "wb")
FileNotFoundError: [Errno 2] No such file or directory: 'Z:\\Visual Media\\[Pics]\\[Artists]\\Pixiv\\1514277\\Manga\\_295867 2 \\あいうえお.test'
Error at process_manga_series(): 295867
Exception: (<class 'FileNotFoundError'>, FileNotFoundError(2, 'No such file or directory'), <traceback object at 0x00000222F1F4D600>)
Unknown Error, please check the log file: (<class 'FileNotFoundError'>, FileNotFoundError(2, 'No such file or directory'), <traceback object at 0x00000222F36A0800>)

Ikkoru avatar Aug 01 '25 19:08 Ikkoru

Testing various filenames (part 2).

Same config settings as above.

Series name: /<>:"|*? Filename: /<>:"|*?

Processing Image Id: 133382877
Member Url    : https://www.pixiv.net/ajax/user/1514277/profile/all
Member Name   : Ikkoru Befuze
Member Avatar : https://i.pximg.net/user-profile/img/2010/02/28/02/45/38/1529334_1be078359fd7e785ec52b5dc466853a5.png
Member Token  : bmxarena
Member Backgrd: no_background
Title     : /<>:"|*?
Tags      : AI-generated, 背景, 女の子, 少女, 蜘蛛ですが、なにか?, 白織, 白髮, 赤目
Date      : 2025-08-01 19:11:00+00:00
Mode      : manga
Bookmarks : 0
Pages     : 2
[1/2] Image URL : https://i.pximg.net/img-original/img/2025/08/02/04/11/05/133382877_p0.png
[1/2] Filename  : Z:\Visual Media\[Pics]\[Artists]\Pixiv\1514277\Manga\_295867 1 \_______ 133382877 ________ 1.png
Creating directory: Z:\Visual Media\[Pics]\[Artists]\Pixiv\1514277\Manga\_295867 1
Traceback (most recent call last):
  File "C:\Portable Programs\[Downloaders]\PixivUtil2\PixivImageHandler.py", line 286, in process_image
    (result, filename) = PixivDownloadHandler.download_image(caller,
                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
                                                                img,
                                                                ^^^^
    ...<6 lines>...
                                                                page,
                                                                ^^^^^
                                                                notifier)
                                                                ^^^^^^^^^
  File "C:\Portable Programs\[Downloaders]\PixivUtil2\PixivDownloadHandler.py", line 53, in download_image
    test_utf = open(filename_test + '.test', "wb")
FileNotFoundError: [Errno 2] No such file or directory: 'Z:\\Visual Media\\[Pics]\\[Artists]\\Pixiv\\1514277\\Manga\\_295867 1 \\あいうえお.test'
Error at process_image(): 133382877
Exception: (<class 'FileNotFoundError'>, FileNotFoundError(2, 'No such file or directory'), <traceback object at 0x000001E97C39C680>)
Dumping html to: Error medium page for image 133382877.html
Unknown Error, please check the log file: (<class 'FileNotFoundError'>, FileNotFoundError(2, 'No such file or directory'), <traceback object at 0x000001E97D923580>)

Ikkoru avatar Aug 01 '25 19:08 Ikkoru

Using customBadChars = %pattern<1>(/)%%replace<1>(/)% fixes the two examples above. However, it results in incorrect filenames in both cases: 295867 2 /_______ 133383142 __ 1.png should be 295867 2 /_______ 133383142 _/ 1.png 295867 1 /_______ 133382877 ________ 1.png should be 295867 1 /_______ 133382877 /_______ 1.png

Using customBadChars = %pattern<1>(/)%%replace<1>(/)%%pattern<2>(\)%%replace<2>(\)% breaks the program.

Using customBadChars = %pattern<1>(/)%%replace<1>(/)%%pattern<2>(\\)%%replace<2>(\)% is broken: it replaces all of the \ in filenameMangaFormat = %member_id%\%R-18%\Manga\%manga_series_id% %manga_series_order% %manga_series_title% %image_id% %title% %page_number%. It also doesn't replace the \ in the filename: Z:\Visual Media\[Pics]\[Artists]\Pixiv\1514277\Manga\295867 2 /_______ 133383142 __ 1.png should be Z:\Visual Media\[Pics]\[Artists]\Pixiv\1514277\Manga\295867 2 /_______ 133383142 \/ 1.png

Checking the rest using: customBadChars = %pattern<1>(/)%%replace<1>(/)%%pattern<2>(\\)%%replace<2>(\)%%pattern<3>(<)%%replace<3>(˂)%%pattern<4>(>)%%replace<4>(˃)%%pattern<5>(:)%%replace<5>(꞉)%%pattern<6>(")%%replace<6>(“)%%pattern<7>(|)%%replace<7>(⏐)%%pattern<8>(?)%%replace<8>(?)%%pattern<9>(*)%%replace<9>(∗)% Breaks the program due to the ? and *.

Checking the rest using: customBadChars = %pattern<1>(/)%%replace<1>(/)%%pattern<2>(\\)%%replace<2>(\)%%pattern<3>(<)%%replace<3>(˂)%%pattern<4>(>)%%replace<4>(˃)%%pattern<5>(:)%%replace<5>(꞉)%%pattern<6>(")%%replace<6>(“)%%pattern<7>(|)%%replace<7>(⏐)%%pattern<8>(\?)%%replace<8>(?)%%pattern<9>(\*)%%replace<9>(∗)% Results in utter nonsense due to the |: ⏐1⏐5⏐1⏐4⏐2⏐7⏐7⏐\⏐M⏐a⏐n⏐g⏐a⏐\⏐2⏐9⏐5⏐8⏐6⏐7⏐ ⏐2⏐ ⏐/⏐_⏐_⏐_⏐_⏐_⏐_⏐_⏐ ⏐1⏐3⏐3⏐3⏐8⏐3⏐1⏐4⏐2⏐ ⏐_⏐_⏐ ⏐1⏐.⏐p⏐n⏐g⏐

Checking the rest using: customBadChars = %pattern<1>(/)%%replace<1>(/)%%pattern<2>(\\)%%replace<2>(\)%%pattern<3>(<)%%replace<3>(˂)%%pattern<4>(>)%%replace<4>(˃)%%pattern<5>(:)%%replace<5>(꞉)%%pattern<6>(")%%replace<6>(“)%%pattern<7>(\|)%%replace<7>(⏐)%%pattern<8>(\?)%%replace<8>(?)%%pattern<9>(\*)%%replace<9>(∗)% Results in 295867 1 /_______ 133382877 ________ 1.png should be 295867 1 /˂˃꞉“⏐∗? 133382877 /˂˃꞉“⏐∗? 1.png

You can use https://www.pixiv.net/user/1514277/series/295867 for testing.

Ikkoru avatar Aug 01 '25 21:08 Ikkoru