ava icon indicating copy to clipboard operation
ava copied to clipboard

ava's colored diff doesn't allow understanding diffs on plain text only logger interfaces like GitHub Actions

Open TimDaub opened this issue 3 years ago • 10 comments

  • as you can see from this piece of test code: https://github.com/neume-network/strategies/runs/6791138479?check_suite_focus=true#step:6:86 we're comparing two objects with each other using t.is
  • And then below notice the diff https://github.com/neume-network/strategies/runs/6791138479?check_suite_focus=true#step:6:91
  • Unfortunately, this specific error only seems to happen for us within the CI, so I can't reproduce it locally.
  • But on GitHub Actions, as you can see for yourself, I can also not understand the diff as GitHub Actions don't log the colors in ava's original output
  • Here's an example of that on my local machine. See how "hello" is marked in green
Screenshot 2022-06-08 at 16 01 23

To solve: Is this a general issue with ava, or do I maybe just have to select another diff reporter?

TimDaub avatar Jun 08 '22 14:06 TimDaub

Did you try to use t.deepEqual? From what I see - it looks like you are actually comparing two strings. Maybe they are stringified objects - but in the end, compared as strings. I'm not sure if that 100% helps with the issue with colors, but maybe it is worth trying?

sculpt0r avatar Jun 08 '22 14:06 sculpt0r

thanks for the tip for the work around. But yeah I think we're comparing strings. But in general, I'm also interested in solving this problem for good if it is indeed with ava and not my setup.

TimDaub avatar Jun 08 '22 16:06 TimDaub

You're right that the string diffs are color-only. I don't think we can do any other text effect in CI's either (such as underlines). @sindresorhus is that correct?

I'm not sure what we can do at this stage. One solution would be to add text markers underneath the changes (on their own line) but that's not an easy change.

novemberborn avatar Jun 09 '22 12:06 novemberborn

A work around that could get merged is IMO just printing both strings below each other for now. Then at least users have functionality in GH Actions until a fancy underline text algorithm is merged here. Although, with very big diffs that can also be challenging...

TimDaub avatar Jun 09 '22 12:06 TimDaub

@TimDaub I think it should print both lines, like you have locally. Any chance one of the values is an empty string?

novemberborn avatar Jun 09 '22 12:06 novemberborn

@novemberborn It really depends on what the terminal renderer in GitHub Actions supports. All terminals support bold, but only some support underline/strikethrough/italic. You kinda just need to test and see.

sindresorhus avatar Jun 09 '22 12:06 sindresorhus

@TimDaub I think it should print both lines, like you have locally. Any chance one of the values is an empty string?

Then I would have expected (or vice versa):

+ 
- stringtodiff

but it doesn't print - from what I can tell

TimDaub avatar Jun 09 '22 13:06 TimDaub

It really depends on what the terminal renderer in GitHub Actions supports. All terminals support bold, but only some support underline/strikethrough/italic. You kinda just need to test and see.

@sindresorhus interesting. Sounds like that support is not detectable, so we'd need a way to identify CI's and refine based on our knowledge?

If you think that's a direction worth pursuing I can write it up in a ticket.


it doesn't print - from what I can tell

@TimDaub indeed, that is pretty weird. Locally with empty strings or undefined it prints it just fine. It seems like we have two different problems here?

novemberborn avatar Jun 12 '22 18:06 novemberborn

GitHub Actions supports colors though, so not sure why it's not used. Maybe you need to upgrade Chalk for that.

sindresorhus avatar Jun 12 '22 18:06 sindresorhus

so we'd need a way to identify CI's and refine based on our knowledge?

Any terminal and CI I know about that can display bold/underline, can also display colors.

sindresorhus avatar Jun 12 '22 18:06 sindresorhus

@TimDaub idk if it'd work but maybe try the --color cli option to Ava to force color output. I'm like 90% sure I read in the docs that GitHub actions supports color output in its logs via the standard/usual ANSI escape codes. Also I don't think GitHub Actions automatically allocates a tty or pseudo-tty for you either, and most utilities will automatically disable color/escape code output if they can't detect a tty device on stdout (although I'm not totally sure how Ava handles this). So perhaps forcing color output will help? Usually such an option (to force color output) is provided for exactly this type of scenario (i.e. you want/need color output even though you're in a "headless" environment with no detectable tty or gui at runtime).


EDIT: although Ava doesn't currently emit colored output automatically within GitHub actions, forcing color output via ava's --color cli option does in fact appear to work

Screen Shot 2022-11-28 at 8 52 12 AM

tmillr avatar Nov 27 '22 05:11 tmillr