internetarchive
I don't understand this picture
Why would we need so many colors smeared to the bottom right that are not behind the foreground mask?
Those could all be optimized away to facilitate Run Length Encoding.
I agree that in the case of little text this might be overkill. The thought was that it would help compress the images, but it might not have to go down all the way to the bottom.
What would you suggest the foreground looks like to optimise quality and size?
The reason this is done, by the way, is that if we would only fill in the foreground parts and leave the rest black, the compression would (severely) affect the colours in the foreground image. Initially I had something a little different, which would "radiate" the foreground colours in all directions, mixing them in according to other colours in the vicinity (like here https://archive.org/~merlijn/projects/archive-pdf-tools/doc-imgs/alien-mrc-fg.png), but this was quite slow (like, 10-20x slower), and ultimately didn't compress much better, nor give much better quality.
The DjVu-philosophy is to use the cheapest coding behind occlusions. c44 compression is optimized for that. So I think the best picture would depend on the chosen compression. If there exists a PDF-compatible picture compressor that takes into account the occlusion mask that might be helpful.
Outlook voor Android downloadenhttps://aka.ms/ghei36
From: Merlijn Wajer @.> Sent: Tuesday, November 30, 2021 9:34:06 PM To: internetarchive/archive-pdf-tools @.> Cc: rmast @.>; Author @.> Subject: Re: [internetarchive/archive-pdf-tools] I don't understand this picture (Issue #32)
The reason this is done, by the way, is that if we would only fill in the foreground parts and leave the rest black, the compression would (severely) affect the colours in the foreground image. Initially I had something a little different, which would "radiate" the foreground colours in all directions, mixing them in according to other colours in the vicinity (like here https://archive.org/~merlijn/projects/archive-pdf-tools/doc-imgs/alien-mrc-fg.png), but this was quite slow (like, 10-20x slower), and ultimately didn't compress much better, nor give much better quality.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/internetarchive/archive-pdf-tools/issues/32#issuecomment-982997884, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAZPZ5VL6LM3BA43IK3IDVDUOUYL5ANCNFSM5JCRL56Q. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
Right, so the constraints are that it must be encoded in an image format the PDF supports, so no images with custom compression can be embedded. If you can figure out a way to use c44 compression on a mask + input image and create (say) a PNG or TIF file from that, I could look at turning that into a JPEG/JPEG2000 and see if it compresses better or at least provides better quality.
Some JPEG2000 encoders support "region of interest" (ROI) encoding, where they actually take a mask with the region that should be better encoded than other parts, but I tried for days to get that to produce anything sensible, and it never worked well for me.
Then you're probably also further than this publication:
https://www.researchgate.net/publication/3180593_JPEG2000_coded_images_optimization_using_a_content-dependent_approach
Outlook voor Android downloadenhttps://aka.ms/ghei36
From: Merlijn Wajer @.> Sent: Tuesday, November 30, 2021 9:58:49 PM To: internetarchive/archive-pdf-tools @.> Cc: rmast @.>; Author @.> Subject: Re: [internetarchive/archive-pdf-tools] I don't understand this picture (Issue #32)
Right, so the constraints are that it must be encoded in an image format the PDF supports, so no images with custom compression can be embedded. If you can figure out a way to use c44 compression on a mask + input image and create (say) a PNG or TIF file from that, I could look at turning that into a JPEG/JPEG2000 and see if it compresses better or at least provides better quality.
Some JPEG2000 encoders support "region of interest" (ROI) encoding, where they actually take a mask with the region that should be better encoded than other parts, but I tried for days to get that to produce anything sensible, and it never worked well for me.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/internetarchive/archive-pdf-tools/issues/32#issuecomment-983014756, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAZPZ5U4FRCYO3C5IOXX7HDUOU3ITANCNFSM5JCRL56Q. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
I was looking at the ROI in the documentation https://gist.github.com/ahankinson/494572, and close by in the same part of the manual was another construction: "VIS means visual masking;"
https://ieeexplore.ieee.org/document/901044
https://www.researchgate.net/profile/Scott-Daly-3/publication/2389863_Point-Wise_Extended_Visual_Masking_For_JPEG-2000_Image_Compression/links/0c960521b76274e233000000/Point-Wise-Extended-Visual-Masking-For-JPEG-2000-Image-Compression.pdf?origin=publication_detail
I do not have clear whether this 'masking' allows providing a mask, or is some result of locally interpreting of the provided image.
https://www.researchgate.net/profile/Christos-Chrysafis/publication/3970298_JPEG2000-matched_MRC_compression_of_compound_documents/links/54491d820cf244fe9ea21f60/JPEG2000-matched-MRC-compression-of-compound-documents.pdf?origin=publication_detail
One of the writers has 60 patents on his name, however the publication is almost 20 years old. https://theinformaticists.com/portfolio/debargha-mukherjee/
https://www.researchgate.net/profile/Christos-Chrysafis/publication/3970298_JPEG2000-matched_MRC_compression_of_compound_documents/links/54491d820cf244fe9ea21f60/JPEG2000-matched-MRC-compression-of-compound-documents.pdf?origin=publication_detail
"Because the layer coder employed is wavelet-based JPEG2000, smoothness guarantees compactness. Pixels are traversed in raster scan order in each layer, and for each hole pixel encountered, interpolation is performed by a weighted averaging with Gaussian weights, in a P ×P sliding window (where P is typically 5 or 7) centered on it. "
Sounds a bit like what I tried to do initially, and then made a fast approximation for later on (the image in your first post in this issue). Maybe it makes sense to make the background in the foreground image gray, and not black, for more smoothness.