How to use --verify?

[bergfried]

IIRC, a while back and few versions ago, ddcutil seemed to work just fine. However, at some point, a quirk was introduced that resulted in notably buggy behavior when using --verify along with --noverify (or something like that), and values were not always set as intended. The bug seems to be fixed partially, as ddcutil now seems to try and set values in this case but does not always seem to verify them as it used to if both --verify and --noverify are used together.

So, I read the man page and the Synopsis section tells me that there are actually two option areas:

ddcutil [options] command [command-arguments] [options]

Now, I wonder whether --verify is supposed to appear before or after loadvcp filename. I noticed that when I put --verify before it (as I did in the past, IIRC), the call does not always result in the intended outcome anymore (e.g, some color settings remain unchanged). Now, I put --verify both before and after loadvcp filename and according to some tests I did, it seems to work reliably now. This indicates that --verify should appear in the last option area, not the first. Or maybe it needs to appear in both?

This is really confusing. If --verify (or --noverify, for that matter) only works in one of the two option areas, ddcutil should exit with an error code immediately (i.e. without doing anything else) if --verify appears in the wrong option area. Also, the two option areas should be given different names in the man page, and the man page should make clear which options may appear in which option area.

One might also consider providing only one option area but that might have other drawbacks.

While we are at it, contradictory options (like --verify and --noverify) should either be not allowed to appear together or, as I have seen more often, options that appear later in the command line should override options that appear earlier.

Or maybe what I observed is just a bug in the command line parser. EDIT: Or maybe my observations are just incorrect.

EDIT2: It seems that it happened again even with --verify specfiied twice as described above. At that time, the computer was resuming while I manually triggered the script which uses loadvcp to set values. Using ddcutil-2.2.1-1 on Arch.

[rockowitz]

In release 1.4.5 and earlier, if both ---verify and --noverify, --verify took precedence, irrespective of the order in which they were stated. By default, if neither was specified, verification occurred.

Release 2.2.0 introduced a parsing change, but also a bug. As of 2.2.0, if both ***--verify *** and --noverify are specified, the parser emits an error and execution terminates. Again, order is irrelevant. The bug caused verification never to occur. This bug was in the execution layers, and was independent of the parsing changes. That bug has been fixed in branch 2.2.2-dev (commit e925e6d9030fec77768ebf94ef7994f2c24a5bde) .

Options can be specified both before and after the command, or even within the command, e.g. ddcutil getvcp --bus 5 10. The parser strips options out of the command string (and records them) before interpreting the command itself. The man page synopsis just indicates that options can occur both before and after the command and arguments, wh

I apologize that it's taken so long to get back to you on this issue. Let me know if you're still seeing problems with branch 2.2.2-dev.

Finally, I'm interested that you're using loadvcp. I had early on implemented dumpvcp and loadvcp because of my photographic interests, but I haven't had any sense of their actually being used. Please tell me how you're using it. Would it be useful if dumpvcp allowed specification of the features to dumped instead of always dumping those in feature-group color?.

[bergfried]

EDIT: I just noticed that one can provide several feature/value pairs per ddcutil call. My bad, I should have read the man page more carefully. This, of course, would make loadvcp less interesting, and I should probably update my Bash script accordingly. A few points I mention below, especially regarding a hypothetical --expected-value and --check-first, are still valid, though.

---

Thank you for all the information. It matches what I remember and how I now observe and understand it.

I apologize that it's taken so long to get back to you on this issue. Let me know if you're still seeing problems with branch 2.2.2-dev.

No need to apologize. I do not know a reliable way to trigger the bug, and it only happens from time to time, so I think I can wait for an official release.

That bug has been fixed in branch 2.2.2-dev (commit e925e6d) .

That sounds good.

Options can be specified both before and after the command, or even within the command, e.g. ddcutil getvcp --bus 5 10. The parser strips options out of the command string (and records them) before interpreting the command itself. The man page synopsis just indicates that options can occur both before and after the command and arguments, wh

The "wh" at the end makes it look like an incomplete sentence.

Finally, I'm interested that you're using loadvcp. I had early on implemented dumpvcp and loadvcp because of my photographic interests, but I haven't had any sense of their actually being used. Please tell me how you're using it. Would it be useful if dumpvcp allowed specification of the features to dumped instead of always dumping those in feature-group color?.

I think the whole issue is a bit more complicated. I hope it's okay if I explain it by means of pointing out the issues I face (and the solution I dream of) when dealing with my current main monitor.

Regarding dumpvcp, I don't think that specifying any more values is going to help in my particular case. There is at least one setting, 0xF9, that does affect color but is not included in the dump so its value would be missing. On the other hand, including that particular setting and its value would render the entire file unverifyable because input value and output value differ (see #529 for details). One would need to include the correct pair of input and output values in the file somehow but I don't see how ddcutil is supposed to accomplish that.

Also, I seldom use dumpvcp, mostly because the viewing coditions I need are fixed, but also partially because dumpvcp does not include all the settings I am interested in. And as soon as I specify them, I could just as well create a loadvcp-compatible file myself.

And this is what I do, basically. So, why do I use loadvcp instead of setvcp? Because it allows me to set (and verify) several values at once, as in, in a single ddcutil run. And since they are all provided together, there is at least a theoretical chance that ddcutil might be able to speed up the whole procedure. (I don't know the protocols involved or whether pipelining or other optimizations are even possible, but calling ddcutil for each setting separately will render such ideas certainly impossible.)

What I actually do and need is a bit more complicated. In my use cases, there are 3 kinds of settings:

1. Settings like 0x14 that virtually never change. For them, checking the current value first and changing (and verifying) them only if needed instead of setting them each time could improve speed. Most settings are of this kind, so avoiding interacting with them as much as possible is probably best for responsiveness.
2. Settings like 0x1A that often change. For them, setting and verifying them afterwards each time (as is currently the case) is probably best.
3. Settings like 0x10 that are sometimes set relative to their current value. For them, reading them first is necessary, and verifying them is unnecessary as such actions are only ever triggered manually by the user (e.g. via keyboard shortcut) who can just trigger them again if needed.

Orthogonally, there are some settings like 0xF9 where the input value is different from the matching output value. (Luckily, none of the settings of the third kind have this issue, as it would make it impossible to set them in a single ddcutil call.)

I use a Bash script that, for each viewing condition needed, prepares a suitable text file that can be parsed by loadvcp. Since there is currently no way to exploit the nature of settings of the first kind, I group them together with the settings of the second kind. However, since 0xF9 is one of them and I need all values verified, I create one file where input and output match and another where input and output don't match. The first file is processed with --verify. The second file is processed with --noverify and an elaborate Bash script that attempts to set and verify the value up to three times.

So, the file format, as I currently understand it, is insufficient for many of my use cases, but for several quite different reasons.

For comparison, a fictional example of a call to ddcutil with a CLI flexible enough to cover even the most complex of my use cases would look like this:

ddcutil \
  --mfg "$MFG" --model "$MODEL" --sn "$SERIALNO" \
  --verify \
  --no-check-first                      setvcp 16  50 \
                                        setvcp 18  50 \
                                        setvcp 1A  50 \
  --check-first                         setvcp 14  11 \
                                        setvcp 10 100 \
                                        setvcp 12  70 \
                                        setvcp 6C 128 \
                                        setvcp 6E 128 \
                                        setvcp 70 128 \
                   --expected-value  50 setvcp F9  10 \
                                        setvcp F7   2 \
;

In this fictional example,

• several commands can be issued in one call to ddcutil,
• --verify, --verify-first and --no-verify-first are given ahead of all commands where they might apply,
• --expected-value only applies to the very next command (that is, in this example, it only applies to setvcp F9 but not to setvcp F7).

Furthermore,

• --check-first means that for all the following commands where --verify is applicable, an additional verification step is done first, and only if it fails, the normal setting-changing code is entered,
• --no-check-first turns off a previous --check-first for all the following commands,
• --expected-value changes the value used in the verification step to the specific value following it (in this example, the value 10 will be written to 0xF9 but the result must equal 50 to be considered verified).

A command line interface as illustrated above should be mostly compatible with the current one, with the major difference that --verify and maybe other options like --mfg must then always appear early enough.

I think it would be more difficult to make both dumpvcp and loadvcp and the format of the file they use sufficiently expressive, which is why I also think that improving them is unnecessary, or at least less important than improving the CLI as such.

[rockowitz]

Thank you for the detailed explanation of what you're doing.

The most effective way to address what you're trying to do, as well as scripting in general, would be a Python interface to libddcutil. That's a project that has long been on the TODO list, but which has been repeatedly eclipsed by more urgent requirements.

That said, here are some additional comments.

ddcutil has a heavy startup time. Two options can reduce this significantly. Option --bus causes ddcutil to skip searching for displays that implement DDC. Option --skip-ddc-checks skips the tests that determine whether, and how, a monitor actually supports DDC.

Command setvcp can set multiple features on one command call, e.g. ddcutil setvcp 10 95 12 50 would set the value of feature x10 to 95 and the value of feature x12 to 50.

Implementing your sample syntax, given the positionality of the options and commands, is not feasible with the existing parser. However, what you describe could be reduced to three ddcutil calls. One would use an option --check-first, e.g. ddcutil setvcp --check-first 14 11 10 100. A second, with no option, would function as setvcp does now, e.g. ddcutil setvcp 16 50 18 50. A third would use an --expected-value option, e.g. ddcutil setvcp f9 10 --expected-value 50. Unfortunately, command line option -expected-value only makes sense if only one feature is being set.

Correspondingly, the VCP line in the "loadvcp file"" could be extended, e.g.:

VCP 10 95 CHECK-FIRST
VCP F9 10 EXPECTED 50 

Having said that, I'm not sure how widely useful the new options would be. A Python interface would be a more general purpose solution.

Finally, the dangling ", wh" was probably the vestige of a convoluted comment that went too far into the weeds.

[bergfried]

The most effective way to address what you're trying to do, as well as scripting in general, would be a Python interface to libddcutil.

My observations tell me that starting up a Python instance also comes with quite a bit of workload. And when finally running (in the background, for example), it needs about 10 times as much memory as a Bash instance entrusted with a similar task. In other words, I doubt that Python is going to make the whole thing more performant than a single call to a (more or less) specialized executable. I might be wrong, of course, and there might be other use cases where Python is the most reasonable option. (For example, the current workaround I use in Bash is anything but performant. Complex interactions are certainly easier written in Python than Bash, and maybe also more performant.)

ddcutil has a heavy startup time. Two options can reduce this significantly. Option --bus causes ddcutil to skip searching for displays that implement DDC. Option --skip-ddc-checks skips the tests that determine whether, and how, a monitor actually supports DDC.

Thank you, I did not notice the options you mentioned. If I provide the options --edid, --bus and --skip-ddc-checks together, will ddcutil still make sure that the device it is about to communicate with is the monitor identified by EDID (or similar options like --mfg, --model and --sn)? As in, is it safe? Is it the same in case the monitor is identified by the header of a file processed by loadvcp? For me, safety comes first, then correctness (think --verify), then speed.

Correspondingly, the VCP line in the "loadvcp file"" could be extended, e.g.:

VCP 10 95 CHECK-FIRST
VCP F9 10 EXPECTED 50

I was thinking of something very similar myself, but the main problem I could not solve in my head was how dumpvcp is supposed to create such a file without a lot of help coming from the user. CHECK-FIRST is doable, but EXPECTED is a problem. Even if dumpvcp determines that 0xF9 is currently 50, how it is supposed to know that the input value is 10 and not 50? And even if the user wanted to help, she could only do so by first being told by dumpvcp (or other means) what the current value is.

As a side note, from my experience, parsing command line arguments is easier than parsing files. On the other hand, of course, it is probably easier to add a feature to a sufficiently powerful parser than writing a new one from scratch.

Implementing your sample syntax, given the positionality of the options and commands, is not feasible with the existing parser. However, what you describe could be reduced to three ddcutil calls.

This is correct, indeed, but as you have said yourself, ddcutil is slow to start so it would be great if a single call could cover all cases.

Unfortunately, command line option -expected-value only makes sense if only one feature is being set.

As you have indicated, this is only an issue if options aren't positional. Luckily, in my specific case, only one option is affected anyway, so there would be no difference.

Having said that, I'm not sure how widely useful the new options would be.

Good question. Well, there is at least one user asking for it. :-)

A Python interface would be a more general purpose solution.

I agree.

[rockowitz]

Let me try to address your comments as a whole rather than line by line.

First, in creating this reply, I ran some tests for confirmation and found there was a bug that had crept in when the --bus option is used with loadvcp. So for now you'll need to use branch 2.2.2-dev.

Regarding monitor selectors:

Internally, monitors are identified by the communication path, which is normally a I2C bus number. (It will be a USB hiddev device number for those rare monitors that use USB to communicate with the monitor's Virtual Control Panel, which I'll ignore for clarity.) (In case you're curious, it's struct DDCA_IO_Path in file ddcutil_types.h.) On startup, ddcutil inventories all attached monitors. Options --edid etc. search through the list of detected monitors, revolving to the I2C bus number (or USB hiddev number). Once the I2C bus number is found, there's no further use of the values for --edid etc. Option --bus is special, in that it short-circuits this process, simply verifying that there is a monitor on the specified I2C bus.

Command loadvcp is special in that the the vcp file contains monitor identifiers. If no display selection options are given on the command line, the monitor identifiers in the vcp file are used to find the monitor. If display selection options are given on the command, the I2C bus number is determined as above. The monitor identifiers in the vcp file must match those in the monitor's EDID.

Regarding extending the VCP line in the "loadvcp file":

This would be a loadvcp extension only, for use by someone like yourself who is comfortable editing the file. There's no meaningful way to extend dumpvcp correspondingly.

Regarding performance:

The bulk of ddcutil's elapsed execution time is spent in sleeps mandated by the DDC/CI protocol. A Python front-end to libddcutil would incur the startup overhead once, and since sleeps dominate elapsed time I don't see Python adding significantly to elapsed time. But of course, that conjecture remains to be tested.

[bergfried]

First, in creating this reply, I ran some tests for confirmation and found there was a bug that had crept in when the --bus option is used with loadvcp. So for now you'll need to use branch 2.2.2-dev.

I see. No need to hurry, I can wait for an official release.

Once the I2C bus number is found, there's no further use of the values for --edid etc. Option --bus is special, in that it short-circuits this process, simply verifying that there is a monitor on the specified I2C bus.

Let me rephrase that, just to make sure. Options like --edid are used to identify a matching I2C bus number that corresponds to the monitor. However, --bus forces the I2C bus number to be the specified one, and ddcutil will merely verify that the number corresponds to a monitor. Is this correct?

Furthermore, according to some experiments I just performed, adding options like --model to a command line with --bus will result in an error and the message "Monitor specified in more than one way". Why not allowing adding --model, --edid and similar options so the user can make sure that the I2C bus number points to the monitor specified? Using such a combination, the user could skip detecting all monitors but still make sure he is communicating with the correct monitor (assuming the bus number is known for other reasons beforehand).

Command loadvcp is special in that the the vcp file contains monitor identifiers. If no display selection options are given on the command line, the monitor identifiers in the vcp file are used to find the monitor.

The man page reads

The monitor to which the values will be applied is determined by the monitor identification stored in the file.

but says nothing about monitor selection options in the command line taking precedence. Maybe that should be added to or corrected in the man page then.

This would be a loadvcp extension only, for use by someone like yourself who is comfortable editing the file. There's no meaningful way to extend dumpvcp correspondingly.

Thank you, that would be great!

The bulk of ddcutil's elapsed execution time is spent in sleeps mandated by the DDC/CI protocol. A Python front-end to libddcutil would incur the startup overhead once, and since sleeps dominate elapsed time I don't see Python adding significantly to elapsed time. But of course, that conjecture remains to be tested.

I like your scientific approach. :-)

[rockowitz]

Let me rephrase that, just to make sure. Options like --edid are used to identify a matching I2C bus number that corresponds to the monitor. However, --bus forces the I2C bus number to be the specified one, and ddcutil will merely verify that the number corresponds to a monitor. Is this correct?

Yes.

Furthermore, according to some experiments I just performed, adding options like --model to a command line with --bus will result in an error and the message "Monitor specified in more than one way". Why not allowing adding --model, --edid and similar options so the user can make sure that the I2C bus number points to the monitor specified? Using such a combination, the user could skip detecting all monitors but still make sure he is communicating with the correct monitor (assuming the bus number is known for other reasons beforehand).

Until now, ddcutil has used what are referred to internally and in the API as Display_Identifiers. There are several types of Display_Identifiers: i2c bus number, usb hiddev device number, edid, monitor/model/serial-number. The parser currently allows for specification of exactly one Display_Identifier, so for example edid cannot be combined with bus number.

Your comments led me to begin a long contemplated, complicated, change to what I call Display_Selectors. Some of the preliminary changes for this transition are in branch 2.2.2-dev. A selector can include all possible criteria, so long as they are not mutually exclusive. For example, I2C bus number and USB hiddev device number cannot both be included in the criteria. I do not expect this change will be visible in release 2.2.2. Again, in this scheme, bus number is special. If it is the only criterion specified then the search process is skipped. If it is specified along with another criterion, e.g. model number, the normal search process will occur.

Your suggestion that if bus number is specified then the short circuit always occurs with any additional criteria used for verification is a reasonable one, but I'm not sure how widely useful it would be. Likely would not be part of the initial transition to Display_Selectors.

Command loadvcp is special in that the the vcp file contains monitor identifiers. If no display selection options are given on the command line, the monitor identifiers in the vcp file are used to find the monitor.

The man page reads

The monitor to which the values will be applied is determined by the monitor identification stored in the file.

but says nothing about monitor selection options in the command line taking precedence. Maybe that should be added to or corrected in the man page then.

An oversight. The man page has not kept up with enhancements.

[bergfried]

However, --bus forces the I2C bus number to be the specified one, and ddcutil will merely verify that the number corresponds to a monitor. Is this correct?

Yes.

Okay, so it is always at least a monitor. Good to know!

The parser currently allows for specification of exactly one Display_Identifier, so for example edid cannot be combined with bus number.

I see.

Again, in this scheme, bus number is special. If it is the only criterion specified then the search process is skipped. If it is specified along with another criterion, e.g. model number, the normal search process will occur.

By "normal search process" you mean "normal search process but limited to the specified bus number", right? Otherwise, there would be no real change compared to the current (old) behavior, as it would run a normal search and simply ignore the specified bus number.

Your suggestion that if bus number is specified then the short circuit always occurs with any additional criteria used for verification is a reasonable one, but I'm not sure how widely useful it would be.

Well, I see it the other way around. Having read horror stories like https://bbs.archlinux.org/viewtopic.php?id=193048, I would never use --bus the way it is currently implemented. It is good to know, as you have explained here, that ddcutil will always make sure that it talks to a monitor, but the man page does not explain that to the cautious user. And, of course, verifying that it is not only a but the monitor would be even better. Who knows what might happen if you setvcp a vendor-specific setting on the wrong monitor!

[rockowitz]

    However, --bus forces the I2C bus number to be the specified one, and ddcutil will merely verify that the number corresponds to a monitor. Is this correct?

Yes.

Okay, so it is always at least a monitor. Good to know!

The parser currently allows for specification of exactly one Display_Identifier, so for example edid cannot be combined with bus number.

I see.

Again, in this scheme, bus number is special. If it is the only criterion specified then the search process is skipped. If it is specified along with another criterion, e.g. model number, the normal search process will occur.

By "normal search process" you mean "normal search process but limited to the specified bus number", right? Otherwise, there would be no real change compared to the current (old) behavior, as it would run a normal search and simply ignore the specified bus number.

Normal search process:

• inventory all /dev/i2c devices that connect to a monitor
• apply the criteria specified to select a monitor from the list; In the new (not yet public scheme), this means AND all display identifiers, e.g. bus, model, edid, etc.

Your suggestion that if bus number is specified then the short circuit always occurs with any additional criteria used for verification is a reasonable one, but I'm not sure how widely useful it would be.

Well, I see it the other way around. Having read horror stories like https://bbs.archlinux.org/viewtopic.php?id=193048, I would never use --bus the way it is currently implemented. It is good to know, as you have explained here, that ddcutil will always make sure that it talks to a monitor, but the man page does not explain that to the cautious user. And, of course, verifying that it is not only a but the monitor would be even better. Who knows what might happen if you setvcp a vendor-specific setting on the wrong monitor! Option "--bus" is an accelerator. You don't have to use it. It is most often used in scripts that people create for their stable environments. It is no more dangerous for identifying the wrong monitor than using "--display". The archlinux horror story baffles me. eDP devices implement I2C slave address x50 (EDID) but not slave address x37 (DDC). These are the slave addresses that ddcutil uses. lmsensors probes other slave address used by sensors.

ddcutil limits its probes to devices that could possibly be monitors. The same check is used whether or not option "--bus" is specified. See function sysfs_is_ignorable_i2c_device() in file src/sysfs/sysfs_base.c

Finally, there's lots of information missing from the man page. Trying to include everything, including every corner case, would make it incomprehensible. That's what the web site is for. Having said that, the web site does need attention. There's too much information in the release notes that hasn't made it into the body of the site.