crow
crow copied to clipboard
ipkn
how to access connection from middleware? / static file support
I'm looking at source code and some parts of it are too complex for me to understand at my current level of C++ knowledge. Is there way/were should I look to get acess to adapter.raw_socket inside my middleware?
the thing is, I want to access raw socket to implement static file delivery via sendfile systemcall. It works if with some tinkering in do_write(), but I don't want to patch the core. Though maybe it's a good idea to make something like do_write_static() when url routes to static directories?
Does the file need to be sent that way? Why not do something like
std::string const get_file_contents(...)
{
std::string file_contents;
file_contents.resize(file_size);
// ... read to &file_contents[0]
return file_contents;
}
CROW_ROUTE(...)([...](...) -> crow::response
{
auto response = crow::response(get_file_contents(...));
response.set_header("Content-Type", "application/octet-stream"); // or set to whatever you need
return response;
});
If you need it to be asynchronous then
#include <memory>
#include <thread>
void get_file_contents_async(..., std::function<void(const std::shared_ptr<std::string>&)> on_ready)
{
std::shared_ptr<std::string> file_contents_ptr(new std::string());
file_contents_ptr->resize(file_size);
// ... call read or ReadFileEx
// use &(*file_contents_ptr)[0] as the buffer
std::thread([file_contents_ptr, on_ready]()
{
while (is_ready != true)
{
// ... poll for completion/errors
}
// completed or there was an error - handle both
on_ready(file_contents_ptr);
});
}
CROW_ROUTE(...)([...](const crow::request&, crow::response& response, ...) -> void
{
get_file_contents_async(..., [&response, ...](const std::shared_ptr<std::string>& file_contents_ptr) -> void
{
if (response.is_alive() == true)
{
response.set_header("Content-Type", "application/octet-stream"); // or set to whatever you need
response.write(*file_contents_ptr);
response.end();
}
}
});
@pierobot Yes, I tried this approach, but anyway, I don't like an idea to read file to memory and then push it through socket. Files could be very big, and even small files with many connections will eat all the memory.
I guess what I need is to modify response object so it has access to raw_socket(). Thus any middleware can sendfile(). no need for nginx/copy+write overheads on linux. p.s. thanks for async example
it was a huge pain in the ass, because i'm stupid ) but it works now, headers/content-length, png/text etc, though it needed some (little, i guess) refactoring in the crow core. I guess it could be useful to steal nginx's mime lib, but in essence - it works. sendfile(/restrcited/by/middleware/url) from a file to socket, without copies the main problem was to unbind request_body/crow's header from middleware/request
I pushed it here https://github.com/digitalist/crow/tree/StaticFileSupport
there's guards for win32 (guess it doesn't support sendfile)
i'll post an example of middleware using this later if someone needs it, but basically
you just setup a restricted path
// app.get_middleware<m_static>().setup("/tmp/");
register your route
// CROW_ROUTE(app, "/tmp/
@digitalist: It seems someone also made some middleware for this (by copying the entire file to memory): https://github.com/Smartype/crow/commit/5a630b20cb91d180a1a4cea98457a5a50d7d2677
@knedlsepp copying entire file into memory - it seems - is a bad design choice, serving statics (at least on linux) should be done by using sendfile (as nginx does it ) - as I did in a patch proposed above. I guess someone (me, if I'll find some time) should try to benchmark it and write a windows compatible solution (that's the primary reasons I didn't do a pull request)
I agree that reading it into memory is not optimal performance-wise. However I wanted to mention the approach since it only requires an additional middleware, which might be preferable to a change in the internals of crow.
auto response = crow::response(get_file_contents(...));
This will block the code isn't it?
Yes.
You call whatever asynchronous functions Linux/Windows provide inside get_file_contents_async and it will not block so long as you poll for completion from aio_read/ReadFileEx in a separate thread.
I've updated the example. Hopefully it makes more sense now?
But as others have already mentioned, this approach isn't good for large files. Crow needs a function that internally calls sendfile/TransmitFile.
Hey @digitalist , I know this was a long time ago but I am currently looking for a solution of problem of mine and this is exactly what I needed - effective serving of static resources. Could you please link me to a complete working example of how to wire everything up so that static files can be served. I would be really grateful!
I will try to collect what I can, but don't count on this promise too much.
On Dec 3, 2017 10:04 PM, "Stanislav Arnaudov" [email protected] wrote:
Hey @digitalist https://github.com/digitalist , I know this was a long time ago but I am currently looking for a solution of problem of mine and this is exactly what I needed - effective serving of static resources. Could you please link me to a complete working example of how to wire everything up so that static files can be served. I would be really grateful!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ipkn/crow/issues/116#issuecomment-348805983, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHmtohmuXNPmoOHU_Hplc4SkN12NvQvks5s8vC3gaJpZM4HV8TP .