[feature] Show only added/removed lines in notifications

[unknownsoldier08]

Version and OS 0.39.18 on linux/docker

Is your feature request related to a problem? Please describe. When I get notifications to discord, it shows me all of the grabbed content (including all additions and removals)

Describe the solution you'd like Is it possible to have it show only what has been added and removed and not include all of the information that remains the same?

Describe the use-case and give concrete real-world examples For a lot of websites, like forums, feeds full of line items, there's usually 20+ lines. If there was only 1 new addition, we'd see ~20 lines in the notification.

For lists, sometimes we're only interested in that new addition, so instead of including all the extracted text, I'd like to just show what has been added or removed.

I've tried {diff}, but this seems to extract everything that was caught in my filter (ie. .postItem).

Is something like this possible currently or can it be added as a feature?

[dgtlmoon]

I think this is maybe a dupe of https://github.com/dgtlmoon/changedetection.io/pull/749 ?

[dariusaurius]

even though the filter is activated to only trigger on new lines, the token {diff} still shows all diff-categories inside the notification. having more tokens available, would be so awesome, e.g. {diff_added}, {diff_changed} and {diff_removed}

[Suddow]

Yes I'd also like more granular notification settings. 'Added', 'changed', 'removed', also maybe ways to exclude "ignored text" or "removed elements" from the filters/triggers page in the notification itself as well.

[MaRmARk0]

I was thinking about this too. I watch some eshops for newly added "sale" items. So I don't need to see removals, only additions or maybe (price) changes. So having {diff_added}, {diff_changed} and {diff_removed} would be nice.

[StarvingDeveloper]

In my case I am watching over big list with pagination. So when new items gets added some some old get moved to 2nd page and I am always getting "removed" lines which only distract. So {diff_added} implementation sounds like perfect idea for me.

[StarvingDeveloper]

@dgtlmoon I checked #749 and you abandoned it in favour of #1071, but what we need is totally different - in our reports we have added and removed lines, but we dont want to see what got removed and want to read only what got added, so {diff_added} as @MaRmARk0 suggested perfect solution for many, not only us.

[dgtlmoon]

@StarvingDeveloper i'm open to a pull request if you want todo it (must include tests), if you bother to read that thread, there was a very good reason why it was abandoned https://github.com/dgtlmoon/changedetection.io/pull/749#issuecomment-1300508160

[StarvingDeveloper]

@StarvingDeveloper i'm open to a pull request if you want todo it (must include tests), if you bother to read that thread, there was a very good reason why it was abandoned #749 (comment)

@dgtlmoon If I would capable of such PR I would not discus in issues, but would try make changes. As about #749, I think you still dont understand what we want to have in this and in many other threads as I seen: we just want change notifications, not detection. as I said on early message in this thread that addition would be very useful for every page with pagination, because when in such pages new content appear, older entries get moved to next page and in notification we get same amount of (removed) lines as (added) and that distract a lot - you see something you seen before and get confused till you notice it's removed, not added.

so tl,dr: improve only notifications with option {diff_added}. Some users seen asking for {diff_removed}. Bet for me actual only {diff_added}.

[ItIsSeven]

Has this issue moved forward at all, this is essential.

[reecespieces]

I could really use this. Here's a visual representation to help get the point across.

The change detection itself is working how I want.

This is what my notification looks like using the {{diff_full}} template token.

There should be 3 more template tokens added for notifications.

{{diff_added}} should only show the lines starting with (added ) {{diff_changed}} should only show the lines starting with either (removed) or (added ) {{diff_removed}} should only show the lines starting with (removed)

---

Side topic: "(added )" shouldn't have that space. It should be "(added)".

[reecespieces]

I've dug through the code-base and done some testing, and it seems like the {{diff}} token will show only lines starting with (added ), (removed), (changed), or (into ), so that excludes all the lines that stayed the same.

I was a little confused because {{diff}} wasn't actually in the list of valid tokens in the edit notification page.

I've submitted a PR which adds {{diff}} to that page, along with 2 more tokens that I have added: {{diff_removed}} and {{diff_added}}.

{{diff_removed}} will only show lines starting with (removed), (changed), or (into). {{diff_added}} will only show lines starting with (added), (changed), or (into).

I also got rid of the trailing spaces in "(added )" and "(into )", so now it will just print "(added)" and "(into)" instead.

I don't completely understand the part that generates the (changed) and (into) lines. Even before my modifications, sometimes my notifications would have multiple (changed) or (into) lines in a row, and there would be more of one than the other. Other times, it will work completely as expected, with each (into) line being immediately preceded by a (changed) line. So for now, I've decided to leave that part alone.

Example of what my notifications look like now that I am using {{diff_added}}:

[StarvingDeveloper]

So it is impossible leave only (added) without (changed) and (into)? In my case it still will be mess and I was interested in only (added)... But thanks for effort - I hope for must your PR will work.

[reecespieces]

I could make another token that only shows (added) lines.

And another that only shows (removed) lines.

In my use cases though, I have noticed, a removal + an addition will get the (changed) (into) lines. So if you are ignoring all the (into) lines, you might actually miss something new being added if something else was removed at the same time.

Example:

Version 1.19.2-4.5.5 was removed from the page and Version 1.19.2-4.5.6 was added to the page. If I was ignoring the (into) lines, then I wouldn't have been notified that version 1.19.2-4.5.6 came out.

[dgtlmoon]

So if you are ignoring all the (into) lines, you might actually miss something new being added if something else was removed at the same time.

yeah into and added are allllllllllll very dependant on what the python diff() algorithm feels like at the time...

there are phd papers written on this and still no one can decide :)

[reecespieces]

So if you are ignoring all the (into) lines, you might actually miss something new being added if something else was removed at the same time.

yeah into and added are allllllllllll very dependant on what the python diff() algorithm feels like at the time...

there are phd papers written on this and still no one can decide :)

So hey, I have a bit of a question on the "Only trigger when new lines appear" option.

Will it trigger when a new line gets the replace tag and gets shown as (into)? Like in my example above.

EDIT: Answering my own question. Yes it will trigger.

[dgtlmoon]

@reecespieces now we need another token such as {{ diff_added_triggers }}``, {{ diff_changed_triggers }}, `{{ diff_removed_triggers }} so we can just send a notification containing only the lines that changed that also matched our triggers, instead of all of the noise

[StarvingDeveloper]

After recent update with diff_added my reports much, much more useful! Thank you @reecespieces for PR and @dgtlmoon for awesome tool.