Data set

[tomspilman]

Hi @mitaki28 .

Would it be possible to share your data set used on the demo website?

We're looking into an alternative to waifu2x for certain types of images and pixcaler generates better results.

[mitaki28]

Thank you for using pixcaler!

You mean the data set used for training the model on the demo website, right?

Sorry, some images in the data set are not allowed to redistribute these copies. So you need to download these images manually.

1. I uploaded redistributable images in the data set. Please download from http://mitaki28.info/pixcaler/dataset/dataset_redistributable-part.tar.gz and extract it.
2. To download fsmvx_characv.zip (character images), please visit https://razor-edge.work/material/fsmchcv/ and click the link "素材の一括ダウンロード" (it means "download all materials")
   • These material's license is https://razor-edge.work/material/
3. Please download and extract fsmvx_characv.zip
4. Please execute below command in pixcaler project root directory to merge character images into dataset directory and make their background color transparent

   python -m pixcaler.tool.transparent-chartip /path/to/dataset_redistributable-part/chartip /path/to/fsmvx_characv/*/*.png
   

5. Finally, the data set contains 5 directories:
   • chartip
   • font
   • obj
   • tile
   • test

I have some additional notices if you are going to train your custom model,

• Please pull the latest master branch, because training with old default parameters generates too many training snapshot and they will be more than 500GB at the end of training.
• I strongly recommend to use cupy (the chainer extension for Nvidia GPU and CUDA) for training models. (Using GPU makes training 〜100x faster than using CPU. So if you train a model without GPU, you may take several week or several month to create the model.)

  pip install cupy
  python -m pixcaler.train -g [your GPU ID] --composite -i /path/to/dataset/directory -b 4
  

• You can customize model by changing variable lam2 at https://github.com/mitaki28/pixcaler/blob/dd569451624952c78b6b3044a3e92ae84094f711/pixcaler/updater.py#L45
  • Larger lam2 make created models to generate clearer, but more unstable images. For my data set, 1/4 seems to be good for generating clearer image.

[tomspilman]

Thanks @mitaki28 ... we're looking into this now.

[SenhorFlibble]

Have you considered using actual 8-bit graphics from the 90s that came in both high and low-resolution versions? There are a few examples of this, like the following (I'm cross-posting this from comments in the waifu2x project):

• in C&C: Red Alert, there's a number of cinematic stills (actually 3D renders) that are shown during the game credits, which are both in 320x200 and 640x400 resolutions
• the early versions of In Pursuit of Greed use lower resolution character sprites (again, of the pre-rendered variety) than the final version, apparently made from the same models; both sprite sheets can be found in the source code release of the game
• the Macintosh release of Wolfenstein 3-D uses high-resolution versions of the textures and decorations (enemy sprites are probably too different from their DOS counterparts to be useful). Some of these were used in Doom II's secret levels
• the data files of The Elder Scrolls: Daggerfall contain a set of higher-resolution duplicates of certain NPC flats that are used in the game, but in some cases there are different details. These are hand-drawn sprites
• also with Daggerfall, back in 1996 some of the NPC animations (more precisely, two dancing females) were published as GIFs in higher resolution in the NUKE review than the same characters used in the actual game. These are pre-rendered
• Rowan Software's flight sims (Dawn Patrol, Air Power etc.) contain high and low resolution cockpit images, the high-res ones being 640x400 matching the 320x200 low-res versions. Also there are some static screens in both resolutions
• Interplay's website had GIFs of some of Descent's weapon and other pickups at their old website, retrievable via the Wayback Machine

More stuff can be probably obtained from other PC titles that supported both high and low resolution back then. Of course these would not be redistributable but perhaps they would allow to achieve greater precision in results?

[diegovazquez]

Hi, http://mitaki28.info/pixcaler/dataset/dataset_redistributable-part.tar.gz is broken (404), can you re-upload ?, thank you

[mitaki28]

Oh, sorry. I wrongly remove dataset. Now I re-uploaded them.

[ghost]

Hi, @mitaki28

I encountered a pip error when installing chainercv using python3.5 venv, the error is No local packages or working download links found for numpy, but I run pip install numpy it can be installed.

The fsmvx_characv.zip download link is broken, do you have a backup of this pack? Due to this problem, I changed some code see below

Remove: self.chartips = list((self.data_dir/'chartip').glob("*.png"))
Remove: if r < 0.5:
Remove:     with Image.open(str(self.chartips[np.random.randint(len(self.chartips))])) as img:
Remove:     front = img_to_chw_array(img)

It can be run after modification, but aborted (core dumped) once without any errors. (It reminded me Installation of cv2 is recommended for faster computation., then I run pip install opencv-contrib-python to install cv2)

I feel confused, then remove your requirements.txt and venv, restore to default shell, and install all python package manually. This is my new requirements.txt

matplotlib==3.4.2
chainer==7.8.0
chainerui==0.11.0
chainercv==0.13.1

But I still failed. I think pixcaler can run, but I am missing some sample files for training the model.

This is my run command: CHAINER_DEBUG=1 python -m pixcaler.train --composite -i image/dataset -b 4

Error Info:

GPU: -1
# Minibatch-size: 4
# epoch: 50

# upscale learning with automatically generated images
10 tiles loaded
3 objs loaded
29 fonts loaded
10 tiles loaded
3 objs loaded
29 fonts loaded

# start training
Exception in main training loop: NaN is detected on backward computation of Convolution2DFunction
Traceback (most recent call last):
  File "/home/sources/.local/lib/python3.9/site-packages/chainer/training/trainer.py", line 343, in run
    update()
  File "/home/sources/.local/lib/python3.9/site-packages/chainer/training/updaters/standard_updater.py", line 240, in update
    self.update_core()
  File "/home/sources/src/pixcaler/pixcaler/updater.py", line 72, in update_core
    self.loss_gen(gen, x_out, t_out, y_fake).backward()
  File "/home/sources/.local/lib/python3.9/site-packages/chainer/variable.py", line 1581, in backward
    _backprop._backprop_to_all(
  File "/home/sources/.local/lib/python3.9/site-packages/chainer/_backprop.py", line 225, in _backprop_to_all
    _backprop_utils.backprop_step(
  File "/home/sources/.local/lib/python3.9/site-packages/chainer/_backprop_utils.py", line 206, in backprop_step
    raise RuntimeError(
Will finalize trainer extensions and updater before reraising the exception.
Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/sources/src/pixcaler/pixcaler/train.py", line 178, in <module>
    main()
  File "/home/sources/src/pixcaler/pixcaler/train.py", line 175, in main
    trainer.run()
  File "/home/sources/.local/lib/python3.9/site-packages/chainer/training/trainer.py", line 376, in run
    six.reraise(*exc_info)
  File "/usr/lib/python3.9/site-packages/six.py", line 719, in reraise
    raise value
  File "/home/sources/.local/lib/python3.9/site-packages/chainer/training/trainer.py", line 343, in run
    update()
  File "/home/sources/.local/lib/python3.9/site-packages/chainer/training/updaters/standard_updater.py", line 240, in update
    self.update_core()
  File "/home/sources/src/pixcaler/pixcaler/updater.py", line 72, in update_core
    self.loss_gen(gen, x_out, t_out, y_fake).backward()
  File "/home/sources/.local/lib/python3.9/site-packages/chainer/variable.py", line 1581, in backward
    _backprop._backprop_to_all(
  File "/home/sources/.local/lib/python3.9/site-packages/chainer/_backprop.py", line 225, in _backprop_to_all
    _backprop_utils.backprop_step(
  File "/home/sources/.local/lib/python3.9/site-packages/chainer/_backprop_utils.py", line 206, in backprop_step
    raise RuntimeError(
RuntimeError: NaN is detected on backward computation of Convolution2DFunction

Dataset:

tree image/dataset
image/dataset
├── font
│   ├── mplus-1c-black.ttf
│   ├── mplus-1c-bold.ttf
│   ├── mplus-1c-heavy.ttf
│   ├── mplus-1c-medium.ttf
│   ├── mplus-1c-regular.ttf
│   ├── mplus-1m-bold.ttf
│   ├── mplus-1m-medium.ttf
│   ├── mplus-1mn-bold.ttf
│   ├── mplus-1mn-medium.ttf
│   ├── mplus-1mn-regular.ttf
│   ├── mplus-1m-regular.ttf
│   ├── mplus-1p-black.ttf
│   ├── mplus-1p-bold.ttf
│   ├── mplus-1p-heavy.ttf
│   ├── mplus-1p-medium.ttf
│   ├── mplus-1p-regular.ttf
│   ├── mplus-2c-black.ttf
│   ├── mplus-2c-bold.ttf
│   ├── mplus-2c-heavy.ttf
│   ├── mplus-2c-medium.ttf
│   ├── mplus-2c-regular.ttf
│   ├── mplus-2m-bold.ttf
│   ├── mplus-2m-medium.ttf
│   ├── mplus-2m-regular.ttf
│   ├── mplus-2p-black.ttf
│   ├── mplus-2p-bold.ttf
│   ├── mplus-2p-heavy.ttf
│   ├── mplus-2p-medium.ttf
│   └── mplus-2p-regular.ttf
├── obj
│   ├── arabian2.png
│   ├── arabia.png
│   └── kasi5.png
├── test
└── tile
    ├── akumu_kabe.png
    ├── akumu_yuka.png
    ├── arabiaInside_A2.png
    ├── ArabianOutside_A2.png
    ├── arabianOutside_A3.png
    ├── ArabianOutside_A4.png
    ├── kabe.png
    ├── kasi12.png
    ├── kasi.png
    └── yuka.png

Environment: Arch Linux Python 3.9.6 Python35 3.5.10 No NVIDIA GPU