Lazy-like deserialization or status-only response.

[Pilipets]

Problem: My point is there are a lot of cases where the returned by telegram response isn't fully needed - therefore, avoiding its deserialization or deserializing upon the request can speed up the process significantly.

For example, consider sending the message to some chat with auto resp = api::send_message(connector, args);, where API can return the following JSON - {"ok":true,"result":{"message_id":1197,"from":{"id":..,"is_bot":true,"first_name":"Fun Bot","username":"..."},"chat":{"id":..,"title":"simulation","type":"supergroup"},"date":1623916598,"text":"Hello, world!"}}

What banana does, is deserializing the whole returned JSON into message_t, which is relatively slow, because, in fact, only "ok": true, "result":{} can be significant for the end client.

Solution: My suggestion here is to do partial deserialization of the ok, result fields - similarly to what we can see in the extract_api_result, but parsing certain depth only and postpone the whole deserialization:

Provide a way to receive status-only responses or do lazy-like deserialization;

auto ok_resp = api::send_message(connector, args, /*lazy=*/true); // by default, lazy = false
if (!ok_resp) { // ok = false or error happened }
else {
    cout << string(ok_resp) << "\n"; // json response printed
    process(*ok_resp); // here the whole deserialization happens - we might not even call this
}

Let me know your thoughts about such a proposal.

[Smertig]

Adding bool lazy parameter is a bad idea. For example, calling banana::api::send_message with blocking connector returns banana::message_t. What message should be returned if we pass lazy = true?

However, this laziness could be achieved with a specific connector. Typical blocking connector may look like:

class my_connector {
    // general implementation
    banana::expected<std::string> do_request(std::string_view method, std::optional<std::string> body);

    template <class T>
    T request(std::string_view method, std::optional<std::string> body, banana::expected<T>(*then)(banana::expected<std::string>)) {
        // do "raw" request
        banana::expected<std::string> result = do_request(method, std::move(body));

        // here you have not-yet-deserialized answer - `result`

        // transform `expected<string>` to `expected<T>` and "unwrap" it
        // `then` parses json and converts it to `T` (i.e. `message_t`)
        // note: `then` is an argument
        return then(result).value();
    }
};

One can transform it to "fast" version that doesn't parse the result:

class my_connector_without_parsing {
    // general implementation (same)
    banana::expected<std::string> do_request(std::string_view method, std::optional<std::string> body);

    template <class T>
    bool request(std::string_view method, std::optional<std::string> body, banana::expected<T>(*then)(banana::expected<std::string>)) {
        // do "raw" request
        banana::expected<std::string> result = do_request(method, std::move(body));

        // here you have not-yet-deserialized answer - `result`

        // `MAGIC` should take `expected<string>` and return `bool` if it's json with "ok": true
        return MAGIC(result);
    }
};

// Every API method would return bool
bool result = banana::api::send_message(my_connector_without_parsing{ .. }, { .text = "foobar" });

But now banana doesn't allow implementing MAGIC without moving json parser to public interface. However, I have some thoughts how to fix it and also simplify implementation of banana.

Stay tuned!

[Pilipets]

Hi @Smertig, I saw your commit with the excessive refactoring of serialization logic and introduction of serialized_args_t, which allows sending serialized data.

Overall, great work - the issue I'm thinking of is templates dependency, which introduces boilerplate code if using serialized data of the banana library in the other project.

Let me explain what problems I'm referring to - the client wants:

• To store serialized data in some other data structure before invoking banana::api;
• Pass serialized data through some methods before invoking banana::api;
• Use serialized data as a field of some class.

One needs to use std::any or templates instantiation for a tenth of types, but I believe it would be much better to solve the problem on the banana level by using a non-template base type of the template-based serialized data or tag instead of templates specialization.

For example,

struct serialized_args_t {
    std::string_data;
    api::method_tag tag; // this is not template, but enum
}

or

struct serialized_args_t_base { ... }
template<class T>
struct serialized_args_t : serialized_args_t_base { ... }

Does that make any sense to you?

[Smertig]

@Pilipets banana::serialized_args_t<T> is just a typed (<T>) wrapper over std::string. It stores serialized arguments as a simple string being template at the same time to save information about serialized type. There are two options depending on your case:

1. Extract std::string from serialized_args_t<T>::data member and store it. You can later create new serialized_args_t<U> manually, however you should guarantee (at runtime) that T is equal to U. Otherwise telegram API would return error.
2. Erase type and save corresponding method using std::function if you want to use arguments later with banana::call:

using agent_t = banana::agent::default_blocking; // or anything else

banana::serialized_args_t<T> args = ...;

std::function<void(agent_t&)> type_erased_call = [args = std::move(args)](agent_t& agent) {
  auto result = banana::call(agent, /* no move */ args);
  // optionally use result
});

// store type_erased_call anywhere

One needs to use std::any or templates instantiation for a tenth of types, but I believe it would be much better to solve the problem on the banana level by using a non-template base type of the template-based serialized data or tag instead of templates specialization.

Storing type-erased serialized arguments without additional information basically means first option.

You should also note, that banana::call returns different types depending on T in serialized_args_t<T>, so you can't magically implement banana::non_template_call(agent, non_template_args) because its return value should be type-erased (moreover, agent is templated too..)

[Pilipets]

Thank you for your quick response. Here we will create anonymous classes for each type of T in the serialized_args<T> args - this is something I'm trying to avoid.

std::function<void(agent_t&)> type_erased_call = [args = std::move(args)](agent_t& agent) {
  auto result = banana::call(agent, /* no move */ args);
  // optionally use result
});

I agree about the extraction of the std::string, but I need a way to deduce that type T later in order to create serialized_args<T>. Looks like in C++ there is no simple way to do that because we can't store type...

I was thinking about the possible design of the api::call(..., non_template_args_t args) without losing the performance. Where non_template_args_t contains the tag of the serialized data or virtual method implementation or pointer to the respective conversion-method - aka type erasure.

template<class T>
struct serialzed_args_t : non_template_args_t {...}

Anyway, thanks for your answer.