buffer sizes

[FreedomPrevails]

trafficstars

In the "Tune resource consumption" of Readme, it recommends the following settings if you have a lot of RAM :

{upstream_socket_buffer_size, 20480},
{downstream_socket_buffer_size, 512000}

Which is 20KB and 512KB In the mtproto_proxy.app.src we have these settings (which are commented out):

%% {upstream_socket_buffer_size, 51200},   %50kb
%% {downstream_socket_buffer_size, 512000},   %500kb

So Readme is recommending to reduce the upstream_socket_buffer_size and dont change downstream_socket_buffer_size if I have a lot of RAM ? or default settings are something else ? I searched the code but couldn't find any other default values for buffer sizes.

[seriyps]

https://github.com/seriyps/mtproto_proxy/blob/30f8c3dbf3c5d67a60f54cf4f17752cbac80f625/src/mtp_handler.erl#L27 default for upstream_socket_buffer_size

https://github.com/seriyps/mtproto_proxy/blob/30f8c3dbf3c5d67a60f54cf4f17752cbac80f625/src/mtp_down_conn.erl#L36 default for downstream_socket_buffer_size

[FreedomPrevails]

Thanks, so defaults are 50KB and 500KB. So "Tune resource consumption" section is wrong ?

[seriyps]

Well, in some sort, yes. In README it was more like in contrast to "how to tune if you have not enough ram". But it doesn't make sense to make them more than 512k/50k. Probably I should decrease defaults...

[kooran]

Hello , where should i enter the script you mentioned in README to increase ram usage? i have 32G of ram , less than 1G is using, but 80% of CPU, i tried to add this to config file but i get error while reloading the proxy

{max_connections, 128000}, {upstream_socket_buffer_size, 20480}, {downstream_socket_buffer_size, 512000}, {replay_check_session_storage, on}, {replay_check_session_storage_opts, #{max_memory_mb => 2048, max_age_minutes => 1440}},

[seriyps]

@kooran I don't think increasing buffer sizes will help there. Probably you have lots of connections with wrong secret or something like that. Can you post output of ss -s from your server?

[kooran]

an hour ago i have started using load balancing between few instances so it's better now , but still, i want to use more ram, no ram is using at all, this is the output of ss -s on my first instance:

*         19048     -         -
RAW       0         0         0
UDP       2         2         0
TCP       63723     63720     3
INET      63725     63722     3
FRAG      0         0         0

and i preciate it if you tell me where to increase ram usage, thanks

[seriyps]

Seems it's not full output. I need to see smth like

TCP:   121 (estab 80, closed 5, orphaned 1, timewait 5)

[kooran]

you're , i have missed some of it, sorry

Total: 3918 (kernel 6683)
TCP:   10592 (estab 3759, closed 4122, orphaned 2705, synrecv 0, timewait 4122/0), ports 0

Transport Total     IP        IPv6
*         6683      -         -
RAW       0         0         0
UDP       2         2         0
TCP       6470      6467      3
INET      6472      6469      3
FRAG      0         0         0

[seriyps]

So, the number of "real" connections is 3759 and remaining ones are closed or timewait. It usually happens when lots of clients are trying to connect with wrong secret or protocol. Maybe you made proxy TLS-only and published dd-link or smth like that. In this case proxy have to check if client and protocol is valid and disconnect client with wrong secret or protocol. This check takes CPU

[kooran]

nope , that's not the issue , i have enbaled fake TLS-Only and only published the TLS secret (with ee) and it's working fine, i haven't published any other secret , the issue is only and only too much cpu usage and no ram usage, im just asking how to use more ram as you mentioned in README, is it even possible? this issue was on ubuntu

and other issue in debian is just 2 CPU Cores are working and 6 other cores are at 0% , is there any worker setting like other sources? im not a total newbie and im working on mtproto almost a year, i thing this source has a real issue with cpu usage

[FreedomPrevails]

If you want you can send your config here and I will create a new config with the options that you want. It is not that easy for someone who is not familiar with syntax, if you misplace a comma or something, it will not work.

[kooran]

@FreedomPrevails i appreciate your help , thank you ♥ i want to see if we can make more balance between ram and CPU, and between CPU cores (or threats) in this source or not? my server have 32gb of ram and 8 CPU cores, yes im not so familiar with syntax , just a bit, i tried as that much but i failed! should i send my whole config like secrets and tags , port , ......? or ram and cpu details was enough?

[seriyps]

Well, I had experience running it on 8-core server and always all cores were evenly loaded. If you only have 2 cores loaded it means there is something wrong in your setup. Here is the example from the server I have now:

You can share once again your ss -s output after at least 15-20minutes of uptime (numbers you shown me last time are quite low, I guess you got them soon after you restarted proxy) and you can also show what is in the logs /var/log/mtproto-proxy/application.log

[kooran]

@seriyps Now im gonna scream and run :)) would you please answer my question pleaaaase? this is the third time that im asking, IS IT POSSIBLE TO INCREASE RAM BUFFER OR NOT?

[FreedomPrevails]

@kooran send your whole config. If you want you can redact your sensitive information with x chars.

[seriyps]

@kooran would it be easier to discuss in Telegram? https://t.me/erlang_mtproxy

You can increase buffer sizes, but I'm sure that it won't help. All symptoms look like you have tons of people trying to connect with wrong secret or protocol.