[RFC] HTTP Module

[Redfire75369]

I've been working on a HTTP module for a while now. I'm just stuck on a few of the API decisions, specifically on how to use the Response to get the data. If you have any other feedback on it, that'd be nice too.

Here's the current API in http.d.ts. <Removed as it is outdated, See below hidden comment>

[Redfire75369]

Old API

declare module "http" {
	export type Header = string | string[];
	export interface Headers {
		[key: string]: Header,
	}

	type TypedArray = Int8Array | Int16Array | Int32Array
		| Uint8Array | Uint8ClampedArray | Uint16Array | Uint32Array
		| Float32Array | Float64Array;
	export type Body = string | String | ArrayBuffer | TypedArray | DataView;

	interface CommonOptions {
		setHost?: boolean,
		headers?: Headers,
		uniqueHeaders?: Headers,
		body?: Body,
	}

	export type RequestOptions = CommonOptions & {
		auth?: string,
	};

	export type RequestBuilderOptions = CommonOptions & {
		method?: string,
	};

	export function get(url: string, options?: RequestOptions): Promise<Response>;
	export function post(url: string, options?: RequestOptions): Promise<Response>;
	export function put(url: string, options?: RequestOptions): Promise<Response>;
	export function request(resource: string, method: string, options?: RequestOptions): Promise<Response>;
	export function request(resource: Request): Promise<Response>;

	export class Request {
		constructor(url: string, options?: RequestBuilderOptions);
	}

	export class Response {
		constructor();

		get status(): number;

		get_header(name: string): Header;

		get headers(): Headers;

		to_bytes(): Promise<Uint8Array>; // TBD for API
	}

	namespace Http {
		export {
			Header,
			Headers,
			Body,
			RequestOptions,
			RequestBuilderOptions,

			get,
			post,
			put,
			request,

			Request,
			Response,
		};
	}

	export default Http;
}

I've updated a few things to more closely follow the fetch specification.

declare module "http" {
	export function get(url: string, options?: RequestOptions): Promise<Response>;
	export function post(url: string, options?: RequestOptions): Promise<Response>;
	export function put(url: string, options?: RequestOptions): Promise<Response>;
	export function request(resource: string, method: string, options?: RequestOptions): Promise<Response>;
	export function request(resource: Request): Promise<Response>;

	export type Header = string | string[];
	export interface Headers {
		[key: string]: Header,
	}

	type TypedArray = Int8Array | Int16Array | Int32Array
		| Uint8Array | Uint8ClampedArray | Uint16Array | Uint32Array
		| Float32Array | Float64Array;
	export type Body = string | String | ArrayBuffer | TypedArray | DataView;

	interface CommonOptions {
		setHost?: boolean,
		headers?: Headers,
		uniqueHeaders?: Headers,
		body: Body,
	}

	export type RequestOptions = CommonOptions & {
		auth?: string,
	};

	export type RequestBuilderOptions = CommonOptions & {
		method?: string,
	};

	export class Request {
		constructor(url: string, options?: RequestBuilderOptions);
		constructor(request: Request, options?: RequestBuilderOptions);
	}

	export class Response {
		constructor();

		get ok(): boolean;
		get status(): number;
		get statusText(): number;

		get bodyUsed(): boolean;
		get headers(): Headers;

		arrayBuffer(): Promise<ArrayBuffer>;
		text(): Promise<string>;
	}
}

[Redfire75369]

I have just pushed the first implementation to the feature/http branch.

In some sense, it may be smarter to implement this as fetch first, but a lot of the options on fetch are quite useless for spiderfire, and only make sense on a browser.

As usual, the TypeScript bindings and API can be found here: https://github.com/Redfire75369/spiderfire/blob/feature/http/bindings/modules/typescript/http.d.ts. The internal implementation in Rust can be found here: https://github.com/Redfire75369/spiderfire/tree/feature/http/modules/src/http.

[Redfire75369]

Since the main features are complete now, I'll be merging this into master in a few days. The final-ish bindings are as above. The classes are Headers, Request and Response, and I've completed both redirects and timeouts, which are probably the most important subfeatures.

If anyone has any opinions or queries about the API or thinks I've missed out something important, please bring it up here. I'd like to hear about other's opinions on this.

Here's an example of usage with the API, so you can judge ergonomics:

import http, {Headers, Request} from "http";

const headers = new Headers([["Custom-Keep-Alive", "1000"], ["Custom-Cookies", "spider=fire"]]);
const request = new Request("http://www.example.com", { auth: "spider:fire", headers, signal: AbortSignal.timeout(150) });
const response = await http.request(request);
console.log(await response.text());

Here's a simpler example:

import http from "http";

const response = await http.get("http://www.example.com");
console.log(await response.text());

[Redfire75369]

The HTTP Client API for HTTP 0.9/1,0/1.1 has now been merged! Adding HTTP 2.0 support and HTTP Server support will be worked on later.