nx-libs
nx-libs copied to clipboard
ArcticaProject
Document compression
Here some lines I wrote in the x2go-user ML. Put those into a document in the distribution.
The code sets tile size and defer parameters depending on the link
speed, but you can also set these parameters on your own, see man
nxagent:
defer=<int>
defer image updates (enabled for all connection types
except LAN), accepts values 0, 1 and 2
The default value can be set via the command line
(-defer). The value provided as nx/nx option is set when resuming a
session, thus it overrides the command line default.
tile=<string>
set the tile size in pixels (<W>x<H>) for bitmap data
sent over the wire
The default value can be set via the command line
(-tile). The value provided as nx/nx option is set when resuming a
session, thus it overrides the command line default.
You can add the desired valued to the X2GO_NXOPTIONS= line in
/etc/x2go/x2goagent.options, e.g. "menu=1,defer=2,tile=128x128"
* Each defer level adds the following rules to the previous ones:
*
* Level 0 Eager encoding.
*
* Level 1 No data is put or copied on pixmaps, marking them always
* as corrupted and synchronizing them on demand, i.e. when
* a copy area to a window is requested, the source is syn-
* chronized before copying it.
*
* Level 2 The put images over the windows are skipped marking the
* destination as corrupted. The same happens for copy area
* and composite operations, spreading the corrupted regions
* of involved drawables.
Here are the defaults as found in the code:
case LINK_TYPE_MODEM:
deferLevel = 2;
tileWidth = 64;
tileHeight = 64;
case LINK_TYPE_ISDN:
deferLevel = 2;
tileWidth = 64;
tileHeight = 64;
case LINK_TYPE_ADSL:
deferLevel = 2;
tileWidth = 4096;
tileHeight = 4096;
case LINK_TYPE_WAN:
deferLevel = 1;
tileWidth = 4096;
tileHeight = 4096;
case LINK_TYPE_NONE:
case LINK_TYPE_LAN:
deferLevel = 0;
tileWidth = 4096;
tileHeight = 4096;
default:
deferLevel = 0;
tileWidth = 64;
tileHeight = 64;
To my surprise I found another parameter today which is currently
undocumented: -irlimit <kB/s>. This will limit the amount of data that
is used to update drawables on the real display. It can only be passed
via command line. To play with it add it to
/etc/x2go/x2goagent.options.
> I asked pretty much the same question here in 2017. The only response
> I got was from another user who, like you and me, was only able to make
> a few observations by trial and error testing:
> <http://lists.x2go.org/pipermail/x2go-user/2017-February/004254.html>
>
> I agree that it would be nice to get a comprehensive overview of the
> different settings.
Well, I have done some research tonight:
First it is worth noting that what we are talking about here is the
way NX will compress images travelling the wire. NX also (delta)
compresses the message format that X11 uses. This is unaffected by the
pack method setting. So if you only have an xterm handling some text
you probably do not see much difference after changing the pack method
because there are not many images involved. Also note that NX uses a
cache so image compression will help in low bandwidth situations, but
maybe less than expected. That's because once the image has traveled
the wire it will be cached on the other side and not travel the wire
again.
x2goclient has three command line options that will influence the compression:
--link Set default link type (modem,isdn,adsl,wan or lan, default: adsl).
--pack Set default pack method (default: '16m-jpeg').
--quality
Set default image quality(0-9, default: 9).
These can also be set in the Connection tab of the chosen session.
The values you use for these parameter are passed down to the nx
software. Some of the parameters there are described in "man nxproxy"
as follows:
link=<string>
An indication of the link speed that is going to be
used between the proxies. Usually the compression and the other link
parameters depend on this setting. The value can be either 'modem',
'isdn', 'adsl', 'wan', 'lan', 'local' or a bandwidth
specification, like for example '56k', '1m', '100m', etc.
data=<int>
Enable or disable the ZLIB data compression. It is
possible to specify a value between 0 and 9. Usually the value is
chosen automatically based on the requested link setting.
stream=<int>
Enable or disable the ZLIB stream compression. The
value, between 0 and 9, is usually determined according to the
requested link setting. Not fully implemented in nx-X11 Agent, yet.
pack=<string>
Determine the method used to compress images.
Now, the pack and quality settings define the compression algorithm
and the quality of the compressed images. In general there are several
groups of compression methods. Most of them will use a user-provided
number of colors from 8 to 16m. Generic form:
<number-of-colors>[-<method>[-<quality>]]
- none (no image compression, lossless)
- masked (reduce number of bits per pixel, lossy): Format of the pack
parameter: "<number of colors>", e.g "16m"
- jpeg (jpeg compression, lossy): <number of colors>-jpeg, e.g. "16m-jpeg"
- png (png compression, lossy): <number of colors>-png, e.g. "4k-png"
- rgb (zlib compression with Z_DEFAULT_STRATEGY, lossless) : "rgb" or
"16ḿ-rgb" /(16m is the only number of colors supported)
- Run Length Encoding (zlib compression with Z_RLE strategy,
lossless): "rle" or "16m-rle" (16m is the only number of colors
supported)
- bitmap (remove each 4th byte, lossless): "bitmap" or "16m-bitmap"
(16m is the only number of colors supported)
For the lossy compressors the string can also have a "-<quality>"
suffix. This suffix will define the pack quality. The default is "9".
There are some high level compression modes, too, that will map to one
from the list above:
- "lossy" will map to "16m-jpeg"
- "lossless" will map to "16m-rle-9" for link=lan and "16m-bitmap" for
all other link types.
- "adaptive" will choose lossy or lossless, depending on the
characteristics of the image
If no pack method is provided it will be set automatically from the
link parameter:
- modem: adaptive-3
- isdn: adaptive-5
- adsl: adaptive-7
- wan: adaptive-9
- lan: adaptive-9
Images that have a width or height of 2 or less pixels or are not
bigger than 512 bytes will not be compressed (but they are still
cached).
Also add information about the differential compression of the X protocol as described in http://www.vigor.nu/dxpc/DESIGN.html
This is very valuable information. It would be great to get it copied or linked in the X2Go user documentation.
But surely this many options cannot be really useful? Would a selection guide be possible?
That's exactly the reason for this ticket. Once this is integrated into nx-libs you'll have it on your system when using x2go.
It looked like this was an suggestion having fallen on dead ears, so I wanted to provide encouragement. But perhaps I should have recognized it as a reminder to self!
You re welcome to provide a patch ;-)
Regarding guidance: I use nx daily and the only thing I tune is the link parameter. Discussions I had showed that testing all those options is too much work for very little gain. For some workloads it might help.
Uli
On Fri, May 29, 2020 at 9:54 PM Anton Driesse [email protected] wrote:
It looked like this was an suggestion having fallen on dead ears, so I wanted to provide encouragement. But perhaps I should have recognized it as a reminder to self!
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ArcticaProject/nx-libs/issues/802#issuecomment-636158950, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQHBZGQBT5VPNNZC3WN2LTRUAHFVANCNFSM4HJ6FH2A .
I should probably stick to things I know about in terms of patching!
My windows X2Go client gives no clue about the relationship between connection speed aka link and compression aka pack. In fact, the terminology is only becoming clear to me just now.
And in the client I cannot not provide a value for pack, so this is the meaning of nopack? For some strange reason I had assumed nopack would mean no compression!
I am not sure. You can check that by configuring and starting the
session and then examine the options file on the server side
(~/.x2go/
HTH
Uli
On Fri, May 29, 2020 at 10:34 PM Anton Driesse [email protected] wrote:
I should probably stick to things I know about in terms of patching!
My windows X2Go client gives no clue about the relationship between connection speed aka link and compression aka pack. In fact, the terminology is only becoming clear to me just now.
And in the client I cannot not provide a value for pack, so this is the meaning of nopack? For some strange reason I had assumed nopack would mean no compression!
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.
Ok, so I've selected ADSL and nopack in the GUI and nopack becomes none.
Based on your table above I should be getting adaptive-7 which becomes either 16m-jpeg-7 or 16m-bitmap depending on the content, but that doesn't really become apparent in the log.
Info: Accepted connection from '127.0.0.1'.
Info: Connection with remote proxy completed.
Info: Using ADSL link parameters 1408/24/1/0.
Info: Using agent parameters 5000/10/50/0/0.
Info: Using cache parameters 4/4096KB/8192KB/8192KB.
Info: Using pack method 'none' with session 'unix-kde-depth_32'.
Info: Using ZLIB data compression 1/1/32.
Info: Using ZLIB stream compression 4/4.
Hmm, should add a description what all those numbers actually mean.
On Sat, May 30, 2020 at 12:39 PM Anton Driesse [email protected] wrote:
Ok, so I've selected ADSL and nopack in the GUI and nopack becomes none.
Based on your table above I should be getting adaptive-7 which becomes either 16m-jpeg-7 or 16m-bitmap depending on the content, but that doesn't really become apparent in the log.
Info: Accepted connection from '127.0.0.1'. Info: Connection with remote proxy completed. Info: Using ADSL link parameters 1408/24/1/0. Info: Using agent parameters 5000/10/50/0/0. Info: Using cache parameters 4/4096KB/8192KB/8192KB. Info: Using pack method 'none' with session 'unix-kde-depth_32'. Info: Using ZLIB data compression 1/1/32. Info: Using ZLIB stream compression 4/4.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ArcticaProject/nx-libs/issues/802#issuecomment-636312446, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQHBZDKGWXYECRGRA5RUSLRUDO6NANCNFSM4HJ6FH2A .
All this is from in nxcomp/src/Loop.cpp:PrintConnectionInfo(); not all lines are always printed depending on th econfiguration.
Using <linkSpeedName> link parameters <TokenSize>/<TokenLimit>/<FlushPolicy+1>/<FlushPriority> Using agent parameters <PingTimeout>/<MotionTimeout>/<IdleTimeout>/<TaintRepliesHideRender> Using cache parameters <MinimumMessageSize>/<MaximumMessageSize>/<ClientTotalStorageSize> <ServerTotalStorageSize> Using pack method <packMethodName> with session <sessionType> Using image streaming parameters <SplitTimeout>/<SplitTotalSize>/<SplitTotalStorageSize>/<SplitDataThreshold>/<SplitDataPacketLimit> Using image cache parameters <ImageCacheEnableLoad>/<ImageCacheEnableSave>/<ImageCacheDiskLimit> Using ZLIB data compression <LocalDataCompressionLevel>/<RemoteDataCompressionLevel>/<LocalDataCompressionThreshold> Using ZLIB stream compression <LocalStreamCompressionLevel>/<RemoteStreamCompressionLevel> Using bandwidth limit of <bitrateLimitName> bits per second Using cache file <PersistentCachePath>/<PersistentCacheName>
-> the only relevant information for your question is <packMethodName> which is "none", derived from you specifying "nopack".
My document states "If no pack method is provided it will be set automatically from the link parameter". But you cannot configure that in x2go, so the compression option provided is the one the user sees in x2go. The link's default compression is ignored in x2go!
A quick test on my side shows: Info: Using LAN link parameters 1536/24/1/0. Info: Using agent parameters 5000/0/50/0/0. Info: Using pack method 'adaptive-8' with session 'unix-kde-depth_24'.
LAN has "adaptive-9" as default but in x2go I selected "adaptive-8".
Ok?
Uli
On Sat, May 30, 2020 at 12:41 PM Ulrich Sibiller [email protected] wrote:
Hmm, should add a description what all those numbers actually mean.
On Sat, May 30, 2020 at 12:39 PM Anton Driesse [email protected] wrote:
Ok, so I've selected ADSL and nopack in the GUI and nopack becomes none.
Based on your table above I should be getting adaptive-7 which becomes either 16m-jpeg-7 or 16m-bitmap depending on the content, but that doesn't really become apparent in the log.
Info: Accepted connection from '127.0.0.1'. Info: Connection with remote proxy completed. Info: Using ADSL link parameters 1408/24/1/0. Info: Using agent parameters 5000/10/50/0/0. Info: Using cache parameters 4/4096KB/8192KB/8192KB. Info: Using pack method 'none' with session 'unix-kde-depth_32'. Info: Using ZLIB data compression 1/1/32. Info: Using ZLIB stream compression 4/4.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.
Sorry, you lost me.
I think you're saying nopack is not the same as "no pack", but did you say what nopack really is?
Anyway, using nopack I do observe that the choice of link makes a difference.
- ADSL gave me reasonable update speed and quality;
- with MODEM I expected quick update and poor quality, but I got slow update and poor quality;
- with LAN I expected and got slow update but high quality.
"nopack" in x2go becomes "none" in NX. And it means exactly what it says: do not compress images. This setting saves CPU cycles but requires more bandwidth. Images are normally cached so this is relevant only for the time an image must be transferred to the other side.
All the things I documented are valid for plain NX sessions (without using x2go but other software). In NX - if you don't specify a compression method explicitly but only a link type - the compression method is derived from the link type. For ADSL you correctly expect adaptive-7 as I listed in the document.
Using x2go things are a bit different. The NX code is the same and so are the built-in defaults. For ADSL this is still "adaptive-7". But x2go does always provide a compression type in addition to the link type. So that default is never used but overridden by the compression settings the user specifies in x2go's session configuration. So regarding compression you always get what you select there, in your case it is "nopack" (which becomes "none" at some point because of the internal naming of the parameter).
In other words: in x2go you always have to select a clever combination of link speed and compression while using plain NX you can stick with the link speed and use the defaults.
If you only touch the link speed setting you will always get 16m-jpeg.
I am using an own software for connection (which I cannot make public for legal reasons) and always stick with the defaults. I was not aware of this x2go behaviour. But this might explain why I don't see the performance problems I sometimes hear about from other people.
So I consider this a bug.
Developers: what's your opinion about this?
Uli
On Sun, May 31, 2020 at 10:27 PM Anton Driesse [email protected] wrote:
Sorry, you lost me.
I think you're saying nopack is the same as "no pack", but did you say what nopack really is?
Anyway, using nopack I do observe that the choice of link makes a difference.
ADSL gave me reasonable update speed and quality; with MODEM I expected quick update and poor quality, but I got slow update and poor quality; with LAN I expected and got slow update but high quality.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.
Late reply on this, but indeed, X2Go Client could be brighter here and honour NX defaults (or know them and adjust compression and packing settings).
I have a prototype x2goxclient implementing this...
Mike Gabriel @.***> schrieb am Sa., 15. Apr. 2023, 11:18:
Late reply on this, but indeed, X2Go Client could be brighter here and honour NX defaults (or know them and adjust compression and packing settings).
— Reply to this email directly, view it on GitHub https://github.com/ArcticaProject/nx-libs/issues/802#issuecomment-1509679424, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQHBZFKXG5J3O2KP2SDIALXBJRU5ANCNFSM4HJ6FH2A . You are receiving this because you authored the thread.Message ID: @.***>