chaiNNer
chaiNNer copied to clipboard
joeyballentine
Add xBRZ and ScaleNX pixel art upscalers
As requested by metiNN on discord:
Looking at the drop-down menu options, I miss the best algorithm though: xBRZ. It'd be great if you could add the xBRZ scale variations, up to and including the 6x version. https://sourceforge.net/projects/xbrz/
Many thanks in advance!
And there's another really good pixel upscaler: ScaleNX.
https://www.google.com/search?q=scalenx
For anyone wanting to take a stab at this: all current pixel art upscalers are implemented in Rust here: https://github.com/chaiNNer-org/chaiNNer-rs/tree/main/crates/image-ops/src/pixel_art
xBRZ have nothing with AI / neural networks to do... is a good, though outdated algorithm, which was popular before the widespread use of neural networks in various game emulators. I don't think Chainner's developer would want to add it here for any reason. It's simply irrelevant to the philosophy and goals of this project - namely the use of neural networks and artificial intelligence for convenient image processing. P.S. the same refers to ScaleNX upscaler, this is the other type of real time game emulator upscalers with different goals.
@korvin2000
It's simply irrelevant to the philosophy and goals of this project - namely the use of neural networks and artificial intelligence for convenient image processing.
To quote the second sentence of our project description: "Born as an AI upscaling application, chaiNNer has grown into an extremely flexible and powerful programmatic image processing application."
Just because chainner started out as AI software, doesn't mean that it can't grow beyond that. AI is just one tool to enhance images, so why shouldn't chainner offer more tools for that purpose? At the end of the day, we want to make software that is useful to our users and fun for us to make.
Just because chainner started out as AI software, doesn't mean that it can't grow beyond that. AI is just one tool to enhance images, so why shouldn't chainner offer more tools for that purpose? At the end of the day, we want to make software that is useful to our users and fun for us to make.
For sure, but modern AI Models are a way more powerfull and flexible than xBRZ and ScaleNX alghoritms. You can just train e.G. ESRGAN model to act like a xBRZ or ScaleNX upscalers. Look at this models: https://openmodeldb.info/?q=scalenx https://openmodeldb.info/?q=brz So you have already pretrained ESRGAN models / architectures, which doing fast exact he same upscaling job, and yes, about 10-15 years ago I was also interested in trying such nice real time (pixel art / sprites upscaling) algorithms like xBRZ, (xBRZ was developed in 2012, so 12 years ago... scaleNX - multiаfactor fork/variant of Scale2X/Scale3X developed in 2001, also 23 years old ) but nowadays it's just history... I personally will mark them as "deprecated", phased out, halted further development. They have a narrow field of use/application and I IMHO don't see the point and prospects of integrating them into ChaiNNer. These are fairly simple (compared to neural networks based arch's) algorithms developed for old low res games / game emulators and old low-performance PC's . I am not the developer and not the author of this cool nice "Swiss Army knife" ;-) tool, I have no right to speak on his behalf, these are just my thoughts, let him, I mean project owner and main developer/architect/maintainer ( @joeyballentine ) correct me if I am wrong.
@RunDevelopment and I am by the way surprised and admire your productivity and contribution to this and other related open source projects (I know you are also active participant of openmodeldb project and many others ), so do not take my above / words negative as criticism, even more so like an arrogance, I respect your opinion, but only as an attempt to substantiate spoken (written) point of view.
You can just train e.G. ESRGAN model to act like a xBRZ or ScaleNX upscalers
Just wanted to chime in on this specific bit: why would you do that when you can just use the algorithm itself, which is far faster and more efficient? There's literally no reason other than "wow ai cool" to do that, it's such a waste of resources.
To put it this way: imagine using ChatGPT to do 2+2 instead of just using a calculator
They have a narrow field of use/application and I IMHO don't see the point and prospects of integrating them into ChaiNNer.
IMO, the more features chaiNNer has the better. Should we get rid of nodes that only 5% of our users use? Or should we keep them there because of the versatility? I'd prefer the latter.
Maybe we should get rid of the "resize" node or limit it to only downscales because bicubic is "outdated" and AI is better? Or should we have it just because there's legitimate applications for using something like that?
AI is powerful but normal algorithms still have their place when someone wants to use them, and using AI to inefficiently do something an algorithm can do fast is just a waste of time and energy (literally)
but nowadays it's just history...
I mean, just because it's old doesn't mean that it isn't useful. Yes, these old rule-based upscalers aren't all that good in general, but they can shine for specific uses. In the end, this is a question of "what do people that use chainner want?", and apparently there is a demand for these old pixel art upscalers.
the more features chaiNNer has the better
Scope creep as a feature ;)
There are limits of course.
do not take my above / words negative as criticism, even more so like an arrogance
Don't worry 😄
To put it this way: imagine using ChatGPT to do 2+2 instead of just using a calculator AI is powerful but normal algorithms still have their place when someone wants to use them, and using AI to inefficiently do something an algorithm can do fast is just a waste of time and energy
I understand your arguments and I accept them. anyway, I will try to explain my thoughts - Software implementing algorithms like Bicubic or Lanzos, Hermite, Bell, Mitchell, B-Spine, etc. filters are not rare, your Application became a pleasant discovery for me precisely because of the vacuum of convenient user friendly UI Tools for working with AI models. You could try to replicate Photoshop's functionality and add/reimplement all >100 photoshop/imagemagick/etc. filters, but why? extending the application with additional controversial functionality - it complicates further development process, flexibility, code readability, extensibility, etc. It seemed to me that you're also trying to keep the code readable, well structured, modular and trying not to clutter it up with all kind of stuff. This is my subjective opinion as a person who himself wrote last 20 years various filters and effects such as Lanczos, different types of sharpening, blur, edge filters and the like, maybe that's why it's not so really interesting (many alternatives, large, choice, many editors and utilities implementing all these trivial algorithms.). And AI is a new exciting world and I know it's bad, but nobody cares about a waste of resources.
Maybe we should get rid of the "resize" node or limit it to only downscales because bicubic is "outdated
And you know the answer, it's because image size downscaling is a much more trivial and simple task in terms of algorithmis / logic and and the result that the bicubic produces when reducing the image size is simply impossible and makes no sense to improve with AI... actually the best results for shrinking images your could achieve using Lanczos 3 and even better using ImageMagic RobidouxSharp filter. I also wrote my own filter, which is even visually a bit better than RobidouxSharp, but these are all trivial tasks for which AI makes no sense.
I mean - chaiNNer is an extreme handy tool with user friendly UI and is a kind of software that has something that others don't have and that is its important advantage. his uniqueness features, user-friendly philosophy. It just doesn't have any competitors right now when it comes to testing and using different kinds of AI models and using them for image processing and that why we like it. We don't need another photoshop )
Anyway I was glad to hear the latest news, the appearance of a separate project for importing and connecting models, using it in SD and I'm sure - you are moving in the right direction.
but these are all trivial tasks for which AI makes no sense.
So the same argument as can be said for the pixel art filters 😁
And the reason we replicate that kind of functionality (like implementing things you can do in Photoshop), is because we want chaiNNer to (eventually) be one application that can do all sorts of things, with complex batching and programming-like behavior, all in one easy to use no-code application. We don't want it to just be limited to AI upscaling.
And to be quite honest, without any of that stuff, a node program is kinda overkill for just a simple upscaling task. If all chaiNNer could do was process a folder of images and run an AI model on each, then using nodes kinda doesn't make any sense.
In fact, what made me want to start making chaiNNer was because of that exact reason. I had a complex workflow idea that involved me doing 4 upscale-downscale pairs in a row, and there was simply no application available that would let me do it. And if I wanted to customize what it was doing further, well that is really only possible with nodes.
When we have all this functionality, it lets us get that extensible freedom for all sorts of different things.
I've actually considered many times simply stopping development on chaiNNer because many people start using it, set up a simple chain to upscale with a single model, and then complain about how complex it is for such a simple task. But that completely misses the point. The point is that yes, you can do that simple task and yes it's a tad bit more complex than a dedicated upscaling UI, but you can also do an infinite amount of other things with any combination of as many nodes as you want. And the more features we have, the more freedom people get to do their image processing tasks with just one program.
That being said though, I do also want to not bloat the default node list with stuff like the pixel art upscale node in the future. I want to build out a more complex extension system where nodes can be installed, enabled, and disabled so people can only enable whay they want to use (like vscode).
I hope that answers your questions.
Thank you for detailed answer. my brief remark: Even in the world of AI Upscaling tasks, there are still many opened interesting tasks(TODOs): upscale by example / reference image(s), also diffusion based models instead of only GAN arch'es / ones, using AI models for other kind of stuff: denoising, sharpening, debluring, etc. Ability to use user workflow(s) from the command line or via special API's, even more flexible parameterization of applied node effects / filters, etc.
You've done a lot of things already, batching tasks, combining(merging) models, working with different arch'es / models. It's definitely not just a utility for only upscaling images, it's a powerful well designed multifunctional tool. So please don't stop... I like it very much..