display.drawImage Fails on Some Images

[leoherzog]

I have an Inkplate 10 project that's using the sample Image_Frame code, but to pull a thumbnail image from Google Slides instead of Unsplash. Attached is an example of the generated image:

Example URL that I'm dynamically fetching via Apps Script and using in drawImage() (not sure how long this will remain valid):

~~https://lh5.googleusercontent.com/hdLLrW3Lo5Z6qs8NEZfgf8GvnWElxnPadUUVbWKm3Qs9e7RNqTS2xR98vIFaLqUE8qPeXEzzP-r9C3MRsZpoQIJ5nO5-ndkDlvGBZypsL3fRsqtK_TTifinKQwVkT3zj0uKcP2c6ahqVTO2gd7YsWZoZIy_5i60QOkuR_cg7O5nuemI9puELDlET9TUElSCxZMRLMNfpYCLjyNzp02RJstshrgrnKc1qSk4iolO_jjIURspTPSo=s1200~~

When I call Serial.println(display.drawImage(url, display.PNG, 0, 0));, a 0 is printed, indicating that there was an error. How do I determine what the error is? Are there certain types of PNG files that aren't supported? The url variable is correct when logged.

[leoherzog]

I tried taking this test.png file and hosting it elsewhere, then hard-coding that URL into the drawImage() function. Still no dice.

[leoherzog]

After trying various ways of exporting the file, building a proxy with Cloudflare Workers, and trying Github Pages hosting, I learned that the function works correctly when pulling the image from a simple HTTP server on my local machine. Sure enough, any host with https does not work, but http works fine. Turned off SSL enforcement on my Github Pages hosting, hard-coded http in my URL, and it's working fine.

Now, to figure out why SSL isn't working...

[BornaBiro]

Hello leoherzog!

I'm glad that you found the problem.

The only thing that comes to mind is that drawImage() uses a basic HttpClient for getting the data from the Internet (in this case image) doesn't have any support for HTTPS and SSL. Maybe using WiFiClientSecure in downloadFile() function would fix the problem, but that means rewriting whole downloadFile() function or making a new one only for HTTPS and SSL.

But the weird thing is and tried to display a few random images from the Internet that have https:// in the link on the Inkplate and they work perfectly fine.

[leoherzog]

Revisiting this. Here's code that doesn't work:

// Next 3 lines are a precaution, you can ignore those, and the example would also work without them
#ifndef ARDUINO_INKPLATE10
#error "Wrong board selection for this example, please select Inkplate 10 in the boards menu."
#endif

#include "Inkplate.h"

Inkplate display(INKPLATE_3BIT);

void setup() {
  
    Serial.begin(115200);
    display.begin();

    Serial.println("Battery: " + String(display.readBattery()));

    while (!display.joinAP("myssid", "mypassword")) {
      Serial.println("Connecting to WiFi...");
    }
    Serial.println("Connected!");

    display.clearDisplay();
    Serial.println(display.drawImage("https://placehold.jp/1200x825.jpg", display.JPG, 0, 0));
    
    display.display();

    Serial.println("Going to sleep...");
    delay(100);
    esp_sleep_enable_timer_wakeup(30 * 60 * 1000 * 1000); // 30 minutes
    esp_deep_sleep_start();

}

void loop() {
    // Never here, as deepsleep restarts esp32
}

I have also tried proxying through Cloudflare Workers in order to disable SSL, turn gzip on and off, stripping all headers, etc. I can't for the life of me get this to work :(

My Cloudflare Workers URL that also doesn't work: http://inkplate.herzog.tech/image.jpg

The serial monitor:

rst:0x1 (POWERON_RESET),boot:0x32 (Sets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x32 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:10900
load:0x40080400,len:6360
entry 0x400806b4
Wavefrom load failed! Upload new waveform in EEPROM. Using default waveform.
Battery: 3.26
Connecting to wifi.
Connected!
0
Going to sleep...

I really wish the display.drawImage() function returned a more helpful error than 0.

[leoherzog]

This URL doesn't work when I hard-code it:

https://loremflickr.com/1200/825

But if I proxy it through Cloudflare Workers, remove all headers, and disable SSL, it works?

http://inkplate.herzog.tech/image.jpg

[leoherzog]

Any ideas, @nitko12 or @davidzovko?

[leoherzog]

or @BornaBiro or @gogi640?

[davidzovko]

@ifadiga Can you take a look, please?

[BornaBiro]

@leoherzog The reason why it doesn't work with https://loremflickr.com/1200/825 and it works on http://inkplate.herzog.tech/image.jpg it's because of HTTPS usage. For now, the Inkplate library doesn't support HTTPS, but it will in the near future.

For some deeper investigation, we need to see what is going on under the hood of the ESP32 HTTP library.

[leoherzog]

After more experimentation, I think it has something to do with the HTTP Content-Length: x vs Transfer-Encoding: chunked HTTP headers. Even if I proxy through non-SSL Cloudflare Workers, it doesn't work on the latter. Like I said, I've still had problems on some requests when doing http-only through Cloudflare.

[leoherzog]

Confirmed this issue. Using Cloudflare Workers, I turned the request into a non-chunked request with:

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

// https://developers.cloudflare.com/workers/examples/modify-response/
// strip all headers from the docs.google.com response and just return the body
async function handleRequest(request) {
  let response = await fetch('https://docs.google.com/presentation/d/1lSFR-zJzr_ecwGRiFzzz7cc3xVwgZzzLlYe-VNsnTzz/export/jpeg');
  let blob = await response.blob();
  return new Response(blob, {"status": 200, "content-type": "image/jpeg" });
}

https://community.cloudflare.com/t/can-workers-transform-a-chunked-request-to-a-non-chunked-one/432927/4?u=xd1936

and it works perfectly.

[KarloLeksic]

The issue is closed because there is no more activity

[leoherzog]

This is still an issue.

[rsoric]

Hi @leoherzog, we've looked into this a bit more.

On the /dev branch of the Inkplate Library repo you can now find a function 'setFollowRedirects'. From the .ino file It's called as such:

display.setFollowRedirects(HTTPC_FORCE_FOLLOW_REDIRECTS);

There are several options you can set, as you can see in ESP32's Arduino library: HTTPC_DISABLE_FOLLOW_REDIRECTS , HTTPC_STRICT_FOLLOW_REDIRECTS and HTTPC_FORCE_FOLLOW_REDIRECTS .

After implementing this function and calling it to 'FORCE_FOLLOW_REDIRECTS', your link worked for me with the draw image function. Note that you should call the version of the function which specifies the image type, just as you have in your example:

display.drawImage("https://loremflickr.com/1200/825", display.JPG, 0, 0)

I also managed to display "https://placehold.jp/1200x825.jpg" without problems.

Let us know if it works, again, make sure to use the version of the library on /dev.

[misel228]

Hi @rsoric , both URLs return a specific Content-Length in the response headers

Here's an example where the response header Content-Length is not set. Instead it's using "Transfer-Encoding: chunked"

Can you please try this URL, too? http://home.misel.ws/chunked.php

Note: the image is originally a PNG (hence the three large red letters) but the PHP script converts it to Jpeg and sets the Content-Type accordingly.

[rsoric]

Hi @misel228

I tried the URL you provided and that one seems to be running into issues even with the newly implemented function. After forcing Inkplate to use HTTP1.0 for the GET request, it seems to download and show the first chunk of the image (horizontally the top 20% of the image).

The primary focus of the draw image function is to show a specific image pointed to by a URL, file path or memory location. Unfortunately, we can't do much with changing how files are downloaded because it's related to ESP32's core library which handles it. If you have a specific suggestion for a method to try and resolve this issue, let us know, we'll test it for you.

[misel228]

I was able to solve my problem by simply refactoring my server side code so that the content is not sent chunked. I would assume the other commenters here did the same so the URLs work now and don't show the affected problem.

Also, I understand that actually fixing it can be a lot of trouble. The content-length header is probably used to allocate the memory before the actual download. Not knowing how much you receive makes the code much more complex.

What you could do however - and should in my opinion - is put a big red warning flag in the documentation: "Chunked transfers are not supported" or something similar.

[rsoric]

Good to hear it's working for you now.

We will add it in the docs, as more information is always valuable for users.