scikit-image
scikit-image copied to clipboard
scikit-image
High Dynamic Range (HDR) imaging
This implements the method for HDR imaging as described in Debevec and Malik, J. (1997). doi:10.1145/258734.258884 with additions to handle the complexity of 16 bit images
Thanks for the contribution @paalge! Is there a license text associated with the original code? Also, we typically try to add a test whenever we add a new function. Perhaps test against a Creative Commons licensed dataset (such as from NASA). The test could be as simple as running it, verifying that it looks as expected, and verifying statistics about the resulting image (a regression test).
@paalge This looks very interesting, thanks for the PR!
After having a brief look, here's a short to-do list that needs to happen before getting merged in:
- As this is a calculation to combine images, rather than a true input/output operation,
skimage.iois probably not the best place for it in the package. Personally I'd place this inskimage.exposure - We will need unit tests covering the code (see
testssubdirectories throughout the package) - We'll need example data (a small HDR series) in
skimage.data. I believe I can help with this from my own work if you have difficulty finding a permissively licensed set of images. - A gallery example in
doc/examplesnamedplot_hdr_debvec.pyor similar illustrating code use for the website - Your code looks to be nicely formatted according to PEP8, but function names and docstrings will need adjustment - probably easiest to see/imitate other functions in the package. We use the NumPyDoc conventions.
That may seem like a lot, but we're here to help!
the doc can also point to wikipedia https://en.wikipedia.org/wiki/High-dynamic-range_imaging that provides a nice introduction. There's also an example.
It's a great addition btw :)
@blink1073 The code is based on the Matlab code in the publication sited, I would think that made it free from licensing, but I don't have experience with licensing issues (being a researcher). @JDWarner Looks like a good list. As for the examples, there looks to be a dataset in the wikipedia article mentioned by @sciunto. I'm starting the process of changing variable names, but I'm a bit confused about the docstring, as I was under the impression that this was how it is done in numpy?
@paalge Nothing in free from licensing ever, basic IP stuff is really something that should be taught is cs 101 these days.
https://dl.acm.org/citation.cfm?doid=258734.258884 is a direct-ish link to the paper. There is no licence on the code at the bottom. Presumably either ACM or the authors hold copyright?
@tacaswell In research I can implement any algorithm I want without the need to license it, as long as it is not for profit, this also holds true for patents. This said, I don't know what is normal with algorithms published in scientific domains, I know that patents have to be applied for before publishing. My code is partly from the appendix in the article and partly my own implementation of the method described in the article
@paalge possibly true for research, but scikit-image is BSD licensed and may be used for profit. When you start distributing code there is a higher standard. Usually "clean room" implementations from papers are OK, however, sometimes patents cloud the matter. If the specific implementation was developed based off of code in the paper or licensed elsewhere, this can also complicate the picture (viral licenses, etc.)
However, like @tcaswell I am not a lawyer.
@JDWarner and @tacaswell I'm a researcher and I've only had to deal with the legal requirements of filing a patent application. That said this algorithm is implemented by opencv for 8 bit image (the reason I've had to do it again is because my images are 16 bit, and rewriting opencv to support 16 bit images is well beyond my skill). Their source code can be found at: opencv github
Before I sent the email, I had a look at the paper itself. That code is really nothing other than a straightforward implementation of the ideas in the paper, so I think it is perfectly safe to reimplement the code in Python without asking for special licensing permission.
@stefanv That was my feeling also, and it seems that opencv has viewed it the same way
I've updated the code, it is still not perfect, as the example lacks a tone mapping algorithm, which I guess needs an implementation in skimage. Also the stability of the camera response function has been improved by selection more random spots.
Just to double check, but are you sure HDR cannot be done simply using numpy operations? I.e., do we absolutely need for-loops?
@stefanv @vighneshbirodkar I tried cython, but I got some strange errors (cython turning 1 into a very large number), so I vectorised the code instead, resulting in a speedup of at least 160 times. @stefanv I though that CC license meant public domain? This legal business is turning out to be so complicated, that I'm spending to much time not coding what actually will help me in my research, and will have to reconsider which code I'll add.
@paalge CC is Creative Commons, not public domain. Any government datasets (e.g., the NASA images mentioned above) are public domain, though.
You are right: it takes significant effort and dedication to develop high quality open source scientific software. But, in the end, I think it is worth it a thousand times over! A "product" that benefits thousands, instead of only ourselves, and a good reference for reproducible research.
Thank you for your contribution.
I never used HDR with my nikon but it's a good occasion. I can try this week-end and upload them under CC-0 (public domain).
If needed I also have a large number of HDR stacks sitting on my hard drive that I can donate (I was using a pre-electronics lens on a modern low-end nikon so I did not have light metering so I took every picture in triplicate with exposure bracketing). Mostly landscapes around Utah.
@stefanv I agree that NASA dataset would be good, though I can't find any... Also the images used by Debevec I can't find any license for. I agree that sharing the code is good, that is why I'm doing this.
If anyone knows of a good, correctly licensed dataset, please point me in the correct direction.
Hi. I need some help.
I'm trying to save the resulting hdr image with to either the hdr or the exr formats, which are hdr formats. These are supposedly supported by freeimage, and by looking at io._plugins.freeimage_plugin.pyit looks like there should be support by it in skimage also, though the resulting file is not readable by any exr/hdr reader, and by my estimates to small. Minimal example code is:
from skimage import io
import numpy as np
x = np.arange(-5, 5, 0.1)
y = np.arange(-5, 5, 0.1)
xx, yy = np.meshgrid(x, y, sparse=True)
A = np.zeros([yy.shape[0], xx.shape[1], 3])
A[:, :, 0] = np.sin(xx**2 + yy**2) / (xx**2 + yy**2) #R
A[:, :, 1] = -np.sin(xx**2 + yy**2) / (xx**2 + yy**2) #G
A[:, :, 2] = 0.5 * np.sin(xx**2 + yy**2) / (xx**2 + yy**2) #B
A = A / np.max(A.flatten())
print(A.shape)
print('Expected size in byte', A.size * 4)
io.imsave(
'test.exr', np.array(A, dtype=np.float32), plugin='freeimage')
This is going to important as my implementation of hdr should be able to save the resulting hdr image to hdr formats.
@tacaswell Thank you, though this is related to saving the images, not example images I've also noted that freeimage doesn't give you a warning if openexr isn't installed when trying to save to .exr
@paalge my comment was meant for @jni regrading the Utah images.
I have never heard of those formats so I doubt I would be much help there.
@tacaswell Were you able to take some images? If not I'll take some, as I would like to finish this while I'm still working with the code
I have them, just need to get them posted someplace. I will get this done tonight.
Sorry, I just spent a while going through my pictures and none of them are as cool as I remember! I have many pairs/triples, but many of them are hand held (so there needs to be some registration) and almost all of the are landscapes that don't really have enough dynamic range to be a cool example of hdr. :disappointed:
If you want demo photos for panoramas I have that covered.
@tacaswell Thank you for looking. I'll see if I can take some, though I have to think about what has enough range to get a good photo. As of now I don't need panoramas :P
If anyone knows about usable photos it would be great