[Bug]: Extra bytes added in case of Protobuf is send/received.

[reversecoderslab]

trafficstars

TLS client version

1.7.5

System information

tls-client-darwin-amd64-1.7.5.dylib 1.7.5, Protobuf version 27.x

Issue description

Extra bytes added in case of Protobuf is send/received.

Steps to reproduce / Code Sample

Bug https://github.com/bogdanfinn/tls-client/issues/99 already exists. Extra bytes in Protobuf are still added also on latest release version.

08efbfbdefbfbdefbfbdefbfbd041002280432efbfbd01efbfbd402b03efbfbd00eca7a61c690637130fefbfbd7defbfbdefbfbdefbfbd722cefbfbdefbfbdefbfbd2c48efbfbd427551efbfbd2defbfbd59360300efbfbd271defbfbd0a514d40efbfbd205d7e206d2a10efbfbdefbfbd622519efbfbd07efbfbdefbfbd2971efbfbdefbfbdefbfbd7e5f5fefbfbd01efbfbdefbfbdefbfbddab755efbfbd39efbfbdefbfbdefbfbdefbfbdd08d6b5601efbfbd627b5148374f76efbfbdefbfbdefbfbdefbfbdefbfbdefbfbdefbfbd0c042defbfbd52efbfbd64345866efbfbdefbfbdefbfbd09efbfbd5c62efbfbdefbfbd60efbfbdefbfbdefbfbdd0a005efbfbdefbfbd50efbfbdefbfbd76efbfbd49efbfbdefbfbdefbfbdd689efbfbd6b5e7aefbfbd425befbfbd462855efbfbdefbfbd123a70efbfbd6ac782efbfbd23efbfbd2a34efbfbd2fefbfbdefbfbdefbfbdefbfbd2b

Death bytes 'efbfbd' still in response hex included.

[shellcmd89]

i use python version of tls-client, the issue still not been solved. and now i can give an example too. url: https://www.nzonscreen.com/title/get-some-sleep-2002

in browser develop tool, we can filter '.key' (not include the quote) to find the key url, it will expire about 30second or 1 minute. we just need refresh the page if the url expired.

example key url:

key_url = 'https://media-src.nzonscreen.com/media/-OFkBzrCNzqkQE9T_CeGCO-Th3gIzufCtSMRwkBdKk2Zq-EzqmBYNmlHRVLTcy33/index.m3u8encryption.key?md5=zR4rSYOtqAY2UJUjrJZxaw&expires=1720511168'

then write a script, make a request to the key url to get the key content, it is raw bytes format. and also this site require a key:value pair in headers.

headers = {'Referer': 'https://www.nzonscreen.com/',}

session = tls_client.Session( client_identifier='chrome120', random_tls_extension_order=True )

r = session.get(key_url, headers=headers)

the response value is: b'\xef\xbf\xbd\xef\xbf\xbd5*p\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd:\x13\x07l\x1e\xef\xbf\xbdp\xef\xbf\xbd'

which is not correct.

base64 encode: 77+977+9NSpw77+977+977+9OhMHbB7vv71w77+9

convert to hex: efbfbdefbfbd352a70efbfbdefbfbdefbfbd3a13076c1eefbfbd70efbfbd

then we use common requests to make same request to the key url. it returns the correct content

r = requests.get(key_url, headers=headers)

the response value is: b'\xe7\x865*p\xf9\xe8\x95:\x13\x07l\x1e\xfcp\xe5'

base64 encode: 54Y1KnD56JU6EwdsHvxw5Q==

convert to hex: e786352a70f9e8953a13076c1efc70e5

i wish the developer can fix this issue, thanks in advance!

[Nintendocustom]

I wasn't able to reproduce either of those issues. On my end, both protobuf data and the provided example give identical data when using Python's requests library and tls-client.

[shellcmd89]

I wasn't able to reproduce either of those issues. On my end, both protobuf data and the provided example give identical data when using Python's requests library and tls-client.

thanks for reply, then perhaps some configure issues on my side. i use win10 and python 3.12.3. python protobuf 4.25.3(also tried 5.27 and 3.20 both give me the wrong response), is it need some extra config? im grateful if you can give some advice.

this is my small script to make test:

import tls_client

key_url = input('\ninput key_url: ')

headers = {'Referer': 'https://www.nzonscreen.com/',}

session = tls_client.Session(client_identifier='chrome120', random_tls_extension_order=True)

r = session.get(key_url, headers=headers) print(r.content)

but the response always be this: b'\xef\xbf\xbd\xef\xbf\xbd5*p\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd:\x13\x07l\x1e\xef\xbf\xbdp\xef\xbf\xbd'

[Nintendocustom]

The issue might be that "isByteResponse" isn't set to True in your version of tls-client.

[shellcmd89]

thank you, although I can't configure it, what you said makes sense

[Nintendocustom]

I've created a fork of Python-Tls-Client that should resolve this issue.

[shellcmd89]

I've created a fork of Python-Tls-Client that should resolve this issue.

my god, i got success, i installed the tls-client you created. and then run the script, it makes some updating:

New version found: v1.7.5. Updating... Downloaded tls-client-x86.dylib from https://github.com/bogdanfinn/tls-client/releases/download/v1.7.5/tls-client-darwin-amd64-1.7.5.dylib Downloaded tls-client-arm64.dylib from https://github.com/bogdanfinn/tls-client/releases/download/v1.7.5/tls-client-darwin-arm64-1.7.5.dylib Downloaded tls-client-arm64.so from https://github.com/bogdanfinn/tls-client/releases/download/v1.7.5/tls-client-linux-arm64-1.7.5.so Downloaded tls-client-x86.so from https://github.com/bogdanfinn/tls-client/releases/download/v1.7.5/tls-client-linux-ubuntu-amd64-1.7.5.so Downloaded tls-client-32.dll from https://github.com/bogdanfinn/tls-client/releases/download/v1.7.5/tls-client-windows-32-1.7.5.dll Downloaded tls-client-64.dll from https://github.com/bogdanfinn/tls-client/releases/download/v1.7.5/tls-client-windows-64-1.7.5.dll Updated to version v1.7.5

then i input the url, i got the correct response. b'\xe7\x865*p\xf9\xe8\x95:\x13\x07l\x1e\xfcp\xe5'

big thanks Nintendocustom

[iamtorsten]

Fixed here https://github.com/iamtorsten/tls-client

Py Package tls-client2