static file serving

[scottlamb]

As mentioned on reddit: I noticed you have "static file serving" described as a required feature for a 1.0 release.

A plug: https://github.com/scottlamb/http-entity is intended to be useful for this: it handles the conditional GET, byte range serving, and such. The hyper-0.11.x branch is async. You're most welcome to use them if you find them helpful; and if you don't find them helpful, I'd like to fix that.

My crate's young. There's no continuous build yet; the docs haven't gotten much attention; the APIs aren't stable yet; and I'm not in love with the current names http-entity and http-file. On the bright side, that means it's a good time if you have strong opinions on these things. Some possible upcoming API changes:

• the type of a given Entity's body stream and chunk type could become associate types rather than the current generics.
• http_entity::serve could become a trait method rather than its current free function. My habit is pure interfaces from languages such as Go, but it seems a lot of Rust stuff such as futures puts a lot of functionality in trait methods that the implementer doesn't need to override.
• http_file::ChunkedReadFile::new could move into a builder.

[moosingin3space]

That crate looks useful. Could you make a branch to demonstrate how hyproxy might use it?

[moosingin3space]

Also, thoughts on this crate? https://crates.io/crates/tk-sendfile

[scottlamb]

That crate looks useful. Could you make a branch to demonstrate how hyproxy might use it?

Hmm, that's a little more than I was looking to do. I extracted http-entity out of another server I'm working on; I want to share the parts that are useful to others, but I don't want to get too distracted from where I started by plugging http-entity into other servers myself. I'm definitely open to changes to http-entity that would make it more useful to you, though.

Also, thoughts on this crate? https://crates.io/crates/tk-sendfile

Looks neat; one could make a http_entity::Entity implementation that uses sendfile (getting the zero-copy of tk-sendfile and the conditional GET / byte range serving of http-entity) , but it'd require changes to hyper; see this comment.

You can do an mmap implementation today (my server does in fact), and it's faster for large files, but there's a significant caveat: your server will crash with SIGBUS if the file is truncated as you're serving it, so you really only want to do this for files you know aren't going to be touched. Also, to avoid stalling the reactor thread, you want to use mlock, and in some cases that requires modifying /etc/security/limits.conf (adding a * - memlock unlimited line), so there's some additional administrative work required.

[moosingin3space]

Okay, thanks! I shall write a prototype of it soon (probably after I get some testing infrastructure working).

Thanks for the crates!

On Mar 7, 2017 12:48 AM, "Scott Lamb" [email protected] wrote:

That crate looks useful. Could you make a branch to demonstrate how hyproxy might use it?

Hmm, that's a little more than I was looking to do. I extracted http-entity out of another server I'm working on; I want to share the parts that are useful to others, but I don't want to get too distracted from where I started by plugging http-entity into other servers myself. I'm definitely open to changes to http-entity that would make it more useful to you, though.

Also, thoughts on this crate? https://crates.io/crates/tk-sendfile

Looks neat; one could make a http_entity::Entity implementation that uses sendfile (getting the zero-copy of tk-sendfile and the conditional GET / byte range serving of http-entity) , but it'd require changes to hyper; see this comment https://github.com/hyperium/hyper/issues/953#issuecomment-279421899.

You can do an mmap implementation today (my server does in fact), and it's faster for large files, but there's a significant caveat: your server will crash with SIGBUS if the file is truncated as you're serving it, so you really only want to do this for files you know aren't going to be touched. Also, to avoid stalling the reactor thread, you want to use mlock, and in some cases that requires modifying /etc/security/limits.conf (adding a *

• memlock unlimited line), so there's some additional administrative work required.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/moosingin3space/hyproxy/issues/4#issuecomment-284628465, or mute the thread https://github.com/notifications/unsubscribe-auth/AAyqgk236kSwN-mbfWQ6wK7c4pOCcmA0ks5rjO_CgaJpZM4MN6xM .