libevhtp

libevhtp copied to clipboard

v0.4.3

• Added IPv6 Support
• Fixed some query argument parsing
• Fixed issue with parsing OPTION methods.
• Added thread-safe hooking callback add/delete functionality

v0.4.2

• Moved ./libhtparse to ./htparse

• Increased the default stack size limit in the request parser.

• Tweaked SSL thread initialization code.

• Fixed strdup build warnings and errors on OSX

• Added checks for sys/tree.h and use compat if not found.

• Added proper status code string generation.

  • Prior to this release, any http return status code would return the same status code string "derp". Now all known status codes are mapped to the proper status code string.

• Added chunking hooks

  • Prior to this commit, a chunked message is treated as just a body of data. So when a single chunk header has been read, it would just start using it as the content-length and send the body to the on_read hook.

  • This can be problimatic when it comes to doing things like proyxing a request to another server who is expecting chunked data. So the following hooks have been added to give the user access to various stages of chunk processing:

  • evhtp_hook_on_new_chunk

    • Called at the start of a chunk with the number of bytes which will be parsed.

  • evhtp_hook_on_chunk_complete

    • called when a single chunk has been parsed and processed.

  • evhtp_hook_on_chunks_complete

    • called when all chunks for a request have been parsed and processed

  • Refer to ./test.c for example usage of the chunk hooking functionality.

v0.4.1

• Various CMAKE fixes

• Added BSD Lic.

• Added status code to string matching. This fixes the status string "DERP" issue.

• The default callback (if not set by user) now properly returns a 404

• Connection structure now contains the saddr.

• parse_query bugfixes

• Removed deprecated test2.c

• Added timeout function evhtp_set_timeouts

  • This allows the user to set timeouts on both recv and send.

• SSL Fixes / Additions: * Initialize with RAND_poll() * Disable SSL Compression * Added RELEASE_BUFFERS option if openssl version >= 1.0

• evhtp_bind_socket() now has a fourth argument for setting the backlog

• htparser fixes:

  • Integer overflow fix for str_to_uint64
  • Response parsing fixes

v0.4.0

• API fully documented via Doxygen

• Major re-factoring:

  • evhtp_request_t, evhtp_connection_t evhtp_t are now exposed to the user

  • htparse_hooks are no longer tied to a evhtp_t, but are private globals (shouldn't ever be changed).

  • bufferevent operations now share a single set of callbacks with an evhtp_connection_t argument. This keeps things consistent.

  • evhtp_connection_readcb reads and parses requests evhtp_connection_writecb determines if a response is completed and whether it should keep the connection alive or close it.

  • requests and connections can be paused and resumed. if you pause a request, all further request parsing is stopped so that no further hooks are called until evhtp_request_resume() is called.

  • Callback definitions are now abstracted into a parent evhtp_callback_t structure containing the type, hooks which can be added to a callback

  • The callback-hooks act just like the old per-connection hooks. Callback searches have been cleaned up to facilitate both connection based hooks and request based.

    • Callback searches have now been broken up into 3 different functions _evhtp_callback_find, _evhtp_callback_regex_find, _evhtp_callback_hash_find

      • _evhtp_callback_find is a wrapper around the other functions,
      • _evhtp_callback_regex_find will now set a start and end matched
      • offset (so that a user can see exactly where the regex matched)

  • Better API for defining a RFC compliant URI

    • evhtp_uri_t structure is a wrapper around an entire URI

      • evhtp_request_authority_t Container where username/password/hostname/port information is stored if in the request.

      • evhtp_request_path_t Container where various path information from the URI is stored

        • "full" = the full path including file, query args, and fragments
        • "path" = just the parsed path (doesn't include file / qargs / fragments)
        • "file" = the file found "/path/<file.ext>", file.txt being the file
        • matched_soff = the offset where matched path was found
        • matched_eoff = the offset where matched path was ended
        • fragment = a string containing fragment data in URI "...#stuff"
        • evhtp_request_query_t A key/val queue of parsed URI query arguments
        • htp_scheme - If a scheme was present, this is the enum value of the scheme

  • Added evhtp_request_parse_query which will take the query portion of the request URI and does a no-copy parse into an evhtp_request_query_t structure (?h=b;c=d&b=4 would become a list of key-values stored in the query_t struct)

  • Added evhtp_set_pre_accept_cb which calls user-defined function prior to accepting a connection.

  • Added evhtp_set_post_accept_cb which calls user-defined function after accepting a connection.

  • Added evhtp_kv_t structures which are generic key/value queues

  • Added evhtp_kv_new()

  • Added evhtp_kvs_new()

  • Added evhtp_kv_free()

  • Added evhtp_kvs_free()

  • Added evhtp_kvs_for_each() - an iterator for kvs

  • Added evhtp_kv_find() - returns the value (if found) for a key.

  • Added evhtp_header_new()

  • Added evhtp_headers_add_header()

  • Added evhtp_header_key_add()

  • evhtp_hook_type enum's have been been changed to not be in all CAPS:

    • evhtp_hook_on_header = type for a hook after one header is parsed
    • evhtp_hook_on_headers = type for a hook after all headers are parsed
    • evhtp_hook_on_path = type for a hook after path is parsed
    • evhtp_hook_on_read = type for a hook when body data is read
    • evhtp_hook_on_fini = type for a hook just prior to being free()'d
    • evhtp_hook_on_error = type for a hook when a fatal error occurs

  • Currently ssl bufferevents will not cleanly shutdown a SSL connection once bufferevent_free() is called, only SSL_free is done. Since the shutdown process is never taken the session is marked as "bad" and will remove that session for the cache and other various things that can break performance.

  • In order to fix this, before bufferevent_free() is called, I do a non-blocking SSL_shutdown on the sessions SSL ctx. Since I use a set of flags which essentially mark the session as clean, but ignores the full handshake; this breaks TLS RFC. Not the best thing, but it works until a set of features I added to libevent are put into the main base.

  • Added a query_raw string which is the unparsed URI query arguments.

  • SSL session callbacks setting is only done if there are user-defined ones available.

  • libhtparse fixes when dealing with requests with schema data.

• Fixed conditional bug for chunked responses

  • _evhtp_create_reply() was adding a Content-Length 0 header even if the response is chunked (this breaks RFC).

• Added evhtp_kv_rm_and_free which takes an evhtp_kvs_t and evhtp_kv_t set remove the kv_t from kvs_t and free the kv_t structure.

• Added evhtp_kvs_find_kv which acts like kv_find but instead returns an evhtp_kv_t structure.

• Added an on_chunk_complete hook in htparse which is called when a single chunk has been fully parsed.

• Added an on_chunks_complete hook in htparse which is called after all chunks in a request have been parsed.

• Added thread initialization functionality in evthr

• Added streaming reply functionality

  • evhtp_send_reply_start() - creates the initial reply and sends
  • evhtp_send_reply_body() - sends a chunk of data to the client
  • evhtp_send_reply_end() - informs evhtp that the user is done streaming data and figures out whether to keep the connection alive or not.

• Added htparser_get_status to libhtparse

• Added HTTP response parsing in libevhtparsec

• OSX Compatability fixes

• Added evhtp_ssl_use_threads() to make openssl threadsafe

• ssl cfg now has 4 modes of defining how SSL session caching works

  • evhtp_ssl_scache_type_disabled - disable cache entirely
  • evhtp_ssl_scache_type_internal - use OpenSSL's build-in cache
  • evhtp_ssl_scache_type_user - use user-defined caching callbacks
  • evhtp_ssl_scache_type_builtin - use libevhtp's built-in cache (currently doesn't do anything)

• Updated test.c to use the refactored API correctly

• Added x509_verify_cb, max_verify_depth, verify_peer and store_flags option (via Oscar Koeroo [email protected])

• Added HTTP return code 418 (via Oscar Koeroo [email protected])

• Added dummy callbacks and values to the test.c program. (via Oscar Koeroo [email protected])

• Added CA Path option for ssl_cfg. (via Oscar Koeroo [email protected])

• Fixed issue with htparser_should_keepalive() returning wrong values.

v0.3.7

• Due to various problems http-parser (ry/http-parser) I have written my own parser (derived and ispired from various nginx functions) (./libhtparse).

• Re-introduced the pre_accept_cb and post_accept_cb which deprecates the evhtp_set_connection_hooks (which would call a defined function immediately after a connection was accepted, so it was confusing I think).

  The new functions to set the pre/post accepts are: * evhtp_set_pre_accept_cb(evhtp_t *, evhtp_pre_accept cb, void * arg); * evhtp_set_post_accept_cb(evhtp_t *, evhtp_post_accept cb, void * arg); - evhtp_pre_accept functions have the following attributes: * int fd : (the allocated file descriptor) * struct sockaddr * : self-explanitory * int slen : size of sockaddr * void * arg : argument passed to evhtp_set_pre_accept_cb - evhtp_post_accept functions have the following attributes: * evhtp_conn_t * : self explanitory * void * arg : argument passed to evhtp_set_post_accept_cb

• libevhtp now properly honors all return evhtp_res's from defined request hooks. Meaning if you return something other than EVHTP_RES_OK, the proper error handling is done with the exception of EVHTP_RES_PAUSE (see notes on pausing)

• Added [currently only half-working] methods to allow for suspending and resuming parser execution and connection handling.

  A hook which returns an evhtp_res value can now return "EVHTP_RES_PAUSE" which informs libevhtp to stop all request processing on a connection. Alternatively you can use the functions evhtp_request_pause(evhtp_request_t *);

  You may also get a copy of the suspend/resume event timer directly via the function evhtp_request_get_resume_ev()

  To resume execution/processing one must call the function evhtp_request_resume(evhtp_request_t *);

  To completely disable this functionality you may call the function evhtp_disable_pausing(evhtp_t *) before dropping into your event loop

• Removed unnecessary bufferevent_[enable|disable] calls, it was too tedious to work with as it could cause shitty race-like conditions if the client bufferevent was needed outside of the library.

• EVHTP_CLOSE_* flags have been renamed to EVHTP_FLAG_CLOSE_* and length extended to 16 bits.

• added functionality to both set and get user-set htp_request args: evhtp_request_set_cbargs() evhtp_request_get_cbargs()

• added a hook which is called just prior to a evhtp_request_t being free()'d

• Added the ability to define per-callback hooks. Much like per-connection hooks, you can set various hooks which are called when a uri|regex (defined by the set_cb functions) was matched.

  In order to do this properly, set_cb functions now return a evhtp_callback_t which can be passed to the evhtp_set_callback_hook() functions.

  For example: evhtp_callback_t * cb = evhtp_set_cb(htp, "/derp", derp_cb, NULL); evhtp_set_callback_hook(cb, EVHTP_HOOK_HDRS_READ, derp_hdrs_cb, NULL);

  In the case above once evhtp has found that the incoming request is destined for the "/derp" specific callback, it will call "derp_hdrs_cb" after all headers have been read.

  These act just like normal per-connection hooks, but it should be noted that if a per-callback hook has been enabled, the per-connection hook will be ignored for that hook.

v0.3.6

• Removed submodule dependencies
• Added various evhtp_hdr functions
  • evhtp_hdrs_new(): creates a new evhtp_hdrs_t struct
  • evhtp_hdr_copy(): creates a copy of a evhtp_hdr_t struct
  • evhtp_hdrs_copy(): creates a copy of a evhtp_hdrs_t struct
• Added a default request callback if the user has not defined one.
• Added some informational SSL accessor functions

v0.3.5

• Added evhtp_request_t accessors

  evbuf_t * evhtp_request_get_input(evhtp_request_t *); evbuf_t * evhtp_request_get_output(evhtp_request_t *); evbase_t * evhtp_request_get_evbase(evhtp_request_t *); evserv_t * evhtp_request_get_listener(evhtp_request_t *); evhtp_method evhtp_request_get_method(evhtp_request_t *); evhtp_proto evhtp_request_get_proto(evhtp_request_t *); evhtp_conn_t * evhtp_request_get_conn(evhtp_request_t *); evhtp_hdrs_t * evhtp_request_get_headers_in(evhtp_request_t *); evhtp_hdrs_t * evhtp_request_get_headers_out(evhtp_request_t *); evhtp_callback_cb evhtp_request_get_cb(evhtp_request_t *); void * evhtp_request_get_cbarg(evhtp_request_t *); int evhtp_request_get_sock(evhtp_request_t *); const char * evhtp_request_get_path(evhtp_request_t *); const char * evhtp_request_get_uri(evhtp_request_t *); const char * evhtp_request_method_str(evhtp_request_t *); int evhtp_request_get_matched_soff(evhtp_request_t *); int evhtp_request_get_matched_eoff(evhtp_request_t *); int64_t evhtp_request_get_content_length(evhtp_request_t *);

• Better callback return error and response handling. * Callbacks which return an evhtp_res code now must use EVHTP_RES_, where each EVHTP_RES_ definition is a HTTP status code. For example "return EVHTP_RES_400;" will send a 404 error to the client, along with terminating the parser.

• All static functions prefixed with htp* are now just static htp_

• Various small bugfixes.

v0.3.4

• Added optional OpenSSL support (with optional threading abstractions).

  • In order to properly enable SSL for your server two things must happen: evhtp_ssl_cfg structure must be filled with your settings:

    evhtp_ssl_cfg scfg = {
        .pemfile        = ssl_pem,
        .privfile       = ssl_pem,
        .cafile         = ssl_ca,
        .ciphers        = "RC4+RSA:HIGH:+MEDIUM:+LOW",
        .ssl_opts       = <SSL SPECIFIC FLAGS>,
        .enable_scache  = 1,
        .scache_timeout = 1024,
        .scache_init    = evhtp_ssl_scache_builtin_init,
        .scache_add     = evhtp_ssl_scache_builtin_add,
        .scache_get     = evhtp_ssl_scache_builtin_get,
        .scache_del     = NULL,
    };
    

  • please note that .scache_init, .scache_add, .scache_get, and .scace_del are function points to allow for a user to define their own session cache mechanism. Libevhtp has a few default built-in cache functions: evhtp_ssl_scache_builtin_init, evhtp_ssl_scache_builtin_add, evhtp_ssl_scache_builtin_get, evhtp_ssl_scache_builtin_del

    Another side note here is that currently the builtin functions do not expire, this will be fixed in the next beta-release.

  • The second step is to enable the use of the SSL backend by calling evhtp_use_ssl(evhtp_t *, evhtp_ssl_cfg *);

• Regular expression based path callback functions added: evhtp_set_regex_cb(evhtp_t , "^(/path_w_regex/).", callback, args);

  Using the case above, any path/uri in the request with /path_w_regex/*, the API will execute your defined callback function. The output of which is the same as any other defined callback.

• Added _htp_callbacks_find_callback_woffsets(), which (if regex) will set 2 integer variables in the evhtp_rquest_t structure defining the beginning and end of a matched URI/path.

• _htp_callback_new() has been modified to create regex based evhtp_callback_t's

• Made the evhtp_request_t structure private, various get/set functions have been added to deal with the internals.

• Modified compile to support libonigurmura, a BSD licensed cross-platform regular expression library. Currently this is only used to support posix regex functionality.