Different behavior for fetch call between Node.js & Deno

[MiguelRipoll23]

Version: Deno 1.45.5

Seems like Deno is handling the second fetch call differently and the server (router) thinks the user is unathenticated and ends up in a redirection loop.

Code to reproduce (requires LiveBox router)

await login()
  .then((urn) => getConnectedDevices(urn))
  .catch((error) => {
    console.error("An error occurred:", error);
  });

async function login() {
  const bodyParams = new URLSearchParams({
    GO: "status.htm",
    pws: "md5 password here",
    usr: "admin",
    ui_pws: "password here",
    login: "acceso",
  });

  return fetch("http://livebox.home" + "/login.cgi", {
    method: "POST",
    body: bodyParams.toString(),
  })
    .then((response) => {
      console.log("#1", response);
      return response.text();
    })
    .then((responseText) => {
      return responseText.split("'")[1];
    })
    .catch((error) => {
      console.error("Login failed:", error);
    });
}

async function getConnectedDevices(urn) {
  const headers = {
    Cookie: `urn=${urn}`,
  };

  return fetch("http://livebox.home" + "/cgi/cgi_network_connected.js", {
    method: "GET",
    headers,
  })
    .then((response) => {
      console.log("#2", response);
    })
    .catch((error) => {
      console.error("Failed to get connected devices:", error);
    });
}

Deno First fetch call:

{
  body: ReadableStream { locked: false },
  bodyUsed: false,
  headers: Headers {
    "cache-control": "no-cache,no-store,must-revalidate, post-check=0,pre-check=0",
    connection: "close",
    "content-language": "en",
    "content-type": "text/html",
    date: "Sun, 18 Aug 2024 16:14:12 GMT",
    expires: "0",
    pragma: "no-cache",
    server: "httpd"
  },
  ok: true,
  redirected: true,
  status: 200,
  statusText: "OK",
  url: "http://livebox.home/status.htm"
}

Second fetch call:

Failed to get connected devices: TypeError: Fetch failed: Maximum number of redirects (20) reached at ext:deno_fetch/26_fetch.js:370:25 at eventLoopTick (ext:core/01_core.js:168:7) at async fetch (ext:deno_fetch/26_fetch.js:391:7) at async file:///C:/Users/migue/Documents/GitHub/livebox-reporter/test.mjs:1:1

Node.js

{
  status: 200,
  statusText: 'Ok',
  headers: Headers {
    server: 'httpd',
    date: 'Sun, 18 Aug 2024 16:51:38 GMT',
    pragma: 'no-cache',
    'cache-control': 'no-cache,no-store,must-revalidate, post-check=0,pre-check=0',
    expires: '0',
    'content-type': 'text/html',
    'content-language': 'en',
    connection: 'close'
  },
  body: ReadableStream { locked: false, state: 'readable', supportsBYOB: true },
  bodyUsed: false,
  ok: true,
  redirected: true,
  type: 'basic',
  url: 'http://livebox.home/status.htm'
}
{
  status: 200,
  statusText: 'Ok',
  headers: Headers {
    server: 'httpd',
    date: 'Sun, 18 Aug 2024 16:51:38 GMT',
    pragma: 'no-cache',
    'cache-control': 'no-cache,no-store,must-revalidate, post-check=0,pre-check=0',
    expires: '0',
    'content-type': 'text/javascript',
    cookie: 'urn=2f355162420fe882',
    'set-cookie': 'PATH=/; HttpOnly;',
    'content-language': 'en',
    connection: 'close'
  },
  body: ReadableStream { locked: false, state: 'readable', supportsBYOB: true },
  bodyUsed: false,
  ok: true,
  redirected: false,
  type: 'basic',
  url: 'http://livebox.home/cgi/cgi_network_connected.js'
}

Browser:

Thanks.

[MiguelRipoll23]

Second fetch call with redirect as manual.

Node.js:

{
  status: 200,
  statusText: 'Ok',
  headers: Headers {
    server: 'httpd',
    date: 'Sun, 18 Aug 2024 16:31:22 GMT',
    pragma: 'no-cache',
    'cache-control': 'no-cache,no-store,must-revalidate, post-check=0,pre-check=0',
    expires: '0',
    'content-type': 'text/javascript',
    cookie: 'urn=3af13d094d6719f6',
    'set-cookie': 'PATH=/; HttpOnly;',
    'content-language': 'en',
    connection: 'close'
  },
  body: ReadableStream { locked: false, state: 'readable', supportsBYOB: true },
  bodyUsed: false,
  ok: true,
  redirected: false,
  type: 'basic',
  url: 'http://livebox.home/cgi/cgi_network_connected.js'
}

Deno:

 {
  body: ReadableStream { locked: false },
  bodyUsed: false,
  headers: Headers {
    "cache-control": "no-cache,no-store,must-revalidate, post-check=0,pre-check=0",
    connection: "close",
    "content-language": "en",
    date: "Sun, 18 Aug 2024 16:32:22 GMT",
    expires: "0",
    location: "index.htm",
    pragma: "no-cache",
    server: "httpd"
  },
  ok: false,
  redirected: false,
  status: 302,
  statusText: "Found",
  url: "http://livebox.home/cgi/cgi_network_connected.js"
}

[MiguelRipoll23]

Another user reporting a similar scenerario: https://stackoverflow.com/questions/78285531/deno-fetch-post-not-working-as-expected

[lucacasonato]

Hey - without a more minimal reproduction (including a server we can actually hit), this is unfortunately impossible to fix, because we can not determine what difference there is between Deno and Node.

[kt3k]

@MiguelRipoll23 What was the 'urn' value in the first example in Deno?

[MiguelRipoll23]

It’s a token set by the router’s login page. I tried to obtain more information about what’s happening but unfortunately there’s no additional data I can provide.

Feel free to close the issue, if I see the problem in a way you can reproduce it I will reopen.

[DasKoebi]

I have a similar problem with deno (version: 2.06) and node. I want to download a GitHub action job log from a recent run. I use the GitHub REST API for this: Download Job Logs

Test file

const TOKEN = "your PAT";

const URL =
    "https://api.github.com/repos/{owner}/{repo}/actions/jobs/{job_id}/logs";

const myHeaders = new Headers();

myHeaders.append("Accept", "application/vnd.github.v3+json");
myHeaders.append("Authorization", `Bearer ${TOKEN}`);
myHeaders.append("X-GitHub-Api-Version", "2022-11-28");

const requestOptions = {
    method: "GET",
    headers: myHeaders,
    redirect: "follow",
};

fetch(
    URL,
    requestOptions,
)
    .then((response) => response.text())
    .then((result) => console.log(result))
    .catch((error) => console.error(error));

When I run this code in deno, I get the following error message:

<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidAuthenticationInfo</Code><Message>Authentication information is not given in the correct format. Check the value of Authorization header.
RequestId:<Request-ID>
Time:2024-11-12T08:13:13.6041229Z</Message></Error>

But if I execute the same file with Node, I get the log file.

After some testing, I realized that GitHub redirects to *.blob.core.windows.net. And with the current fetch spec (4.4.13), it looks like the fetch used in deno deletes the Authorization header for cross origin redirects and node does not. Deno is spec conform here, but in my opinion it doesn't make much sense on the server side.

[MiguelRipoll23]

Hey @lucacasonato, is the information provided by @DasKoebi helpful?