Purging Services completely broken in current master

[friesoft]

Expected Behavior

Using current master purging services imported previously using Import/Sync (same Import/Sync rules) is broken. New Services are imported/synced just fine but old ones (not returned anymore from the datasource) aren't purged.

Current Behavior

Old Services which aren't reported anymore by the Import source are still kept around and not purged

Possible Solution

Services should be purged

Steps to Reproduce (for bugs)

• Create Import Source using e.g. MySQL/REST API (both not working for us)
• Create Sync Rule creating Services ** Update Policy: Replace (before we had Merge, which didn't work either) ** Purge: Yes ** Purge Action: Delete
• Run Import + Sync
• Modify data returned by Import Source (some old rows removed, maybe add some new)
• Run Import + Sync again
• New Services are created but removed rows don't delete Services

Your Environment

• Director version (System - About): 26e76c611e59d476ceac44150de3c156f9c97ccf
• Icinga Web 2 version and modules (System - About): 2.10.1
• Icinga 2 version (icinga2 --version): 2.13.3-1
• Operating System and version: RHEL 7
• Webserver, PHP versions: Apache httpd 2.4.6 (RHEL default), PHP 7.3.33

[dbecker1234]

We have the same problem but with hosts since upgrading from 1.9.1 to 1.10.0. Could it be a general problem with the same cause? The deleted hosts disappear from the preview and the counter in the history decreases, but the sync job only creates new hosts. Configuration is like above, we tested MySQL and the Netbox-module as source.

Environment:

• Director version: 1.10.0
• Icinga Web 2 version: 2.11.1
• Icinga 2 version: 2.13.5-1
• Operating System: Debian 11.5
• Webserver, PHP version: Apache 2.4.54, PHP 7.4.30

[Thomas-Gelf]

Can be reproduced. Problem seems to be, that the Sync Preview refreshes the "Last Sync" timestamp, which then has an influence on which Import Runs are considered for diffing former objects

[friesoft]

Thanks for the fast fix :) I've just updated to current master.

Should this also cleanup (remove) services which would have previously been cleaned up? If yes - it's not working 😆 I'm currently looking into testing if purge works now.

[Thomas-Gelf]

Cleanup depends on whether a successful run without purge took place in the meantime. Schema upgrade wipes invalid Sync history entries, but not valid ones.

Purge works as follows:

• pick the Import Run related to the last Sync Run
• diff object keys with the current/last Import Run
• remove related objects

You have two possibilities:

• manually remove related objects
• remove sync rule history entries that took place AFTER upgrading to v1.10 and sync again

This has been released as v1.10.1, thanks for testing it!

[friesoft]

Okay then I'll have to remove those objects manually over time as there are thousands of changes in between..

Purging services with current master is still not working for me though :(

Import source (MySQL) doesn't return some rows for which services still exist, triggered import, triggered sync => services still exist

Sync rule is configured with "Update Policy": "Merge", "Purge": Yes and "Purge Action": "Delete"

[Thomas-Gelf]

Please test:

• Trigger a Sync run which modifies (or creates/deletes) some services
• Remove a service from your source
• Sync again

Then verify:

• does the Sync preview show, that the service will be removed?
• does the Sync run remove the service?

[dbecker1234]

Also thanks for the fast fix! :-) We just did upgrade to 1.10.1 and have still problems with host purging. Should I create a separate issue or is it related? Don't want to hijack this one, but seems to be identically to friesoft's problem, just hosts instead of services...

I did create a complete new host after the upgrade, it gets created but after deleting at the source not purged. The Sync preview shows nothing and status of the rule is still "This Sync Rule was last found to be in Sync at ...". The Import rule shows the change in the preview and the counter in the history decreases.

Did try to truncate the sync_run table (system isn't productive yet), but without success. Anything I can test or informations that could help you?

[friesoft]

Triggered Sync run created 1 new service Removed 2 Rows from the import source Triggered Import Source Triggered Sync run Sync run reports nothing to do

Sync preview reports "This Sync Rule is in sync and would currently not apply any changes" Nothing removed :(

[Thomas-Gelf]

Strange... works for me. Could you please put Import Source and Sync rule definition in a basket and send me that basket via mail?

[Thomas-Gelf]

Oh, and please run one more test:

• create a new service at your source
• sync
• remove that service at your source
• preview/sync again

[friesoft]

Strange... works for me. Could you please put Import Source and Sync rule definition in a basket and send me that basket via mail?

I've created a ticket in the support tracker with the basket attached: 765475

[Thomas-Gelf]

Thank you! Is there a reason you're using "host service" and not the suggested "host!service"? And which Director version was the last one, where purge worked for you?

[dbecker1234]

Did also sent you mine to the mail address in your profile.

[Thomas-Gelf]

I got both of them, @dbecker1234: as you have the problem with Hosts, did you try with v1.10.1, and does this sequence work for you:

• create a new service at your source
• sync
• remove that service at your source
• preview/sync again

[friesoft]

Thank you! Is there a reason you're using "host service" and not the suggested "host!service"? And which Director version was the last one, where purge worked for you?

To be honest I don't really understand what you mean by "host service" vs "host!service".

Previsouly we were on ccd6f4266a23fbee8f33b1310e58cd72ddd77c45 which was working fine

[friesoft]

Oh, and please run one more test:

• create a new service at your source
• sync
• remove that service at your source
• preview/sync again

Import source reported new data after removing the service, as expected. Sync rule still reports nothing to do

[Thomas-Gelf]

@friesoft: diffing your previous version with the v1.10.1: 4,495 additions and 1,807 deletions in 131 files :see_no_evil: Will fix this as soon as I'm able to reproduce your problem, v1.10.1 still works fine for me. Could you please download a preview JSON for your Import source, keep just 5 lines or so (clone the Import Source and add LIMIT 5 to the query), remove sensible data - and send me that file? I'd love trying to reproduce this with an Import Source looking like yours, shipping similar data

[dbecker1234]

I got both of them, @dbecker1234: as you have the problem with Hosts, did you try with v1.10.1, and does this sequence work for you:

• create a new service at your source
• sync
• remove that service at your source
• preview/sync again

Yes, we upgraded from 1.10.0 to 1.10.1 right after you released it today. I tested with that sequence, new hosts are getting created in the director. But when removing them on the source the sync rule just says it's already in sync, so also no preview. On our site I'm pretty sure it was working with 1.9.1 (we followed the releases in the past and not git master).

[friesoft]

@friesoft: diffing your previous version with the v1.10.1: 4,495 additions and 1,807 deletions in 131 files 🙈 Will fix this as soon as I'm able to reproduce your problem, v1.10.1 still works fine for me. Could you please download a preview JSON for your Import source, keep just 5 lines or so (clone the Import Source and add LIMIT 5 to the query), remove sensible data - and send me that file? I'd love trying to reproduce this with an Import Source looking like yours, shipping similar data

Well thats quite a lot of diff 😆 I've attached an import source preview with 5 rows to the ticket.

[friesoft]

I guess this issue should be reopened and added to the 1.10.2 milestone

[friesoft]

We've downgraded to 1.9.1 now as current master/1.10 is broken for us. With 1.9.1 purging services works as intended.

[Thomas-Gelf]

Hint: affects objects with uppercase characters only

[dbecker1234]

Latest fix worked for us, thanks!

[Thomas-Gelf]

Thanks for letting me know, @dbecker1234! This one was easy to fix, but hard to track down :see_no_evil: Will cherry-pick this to support/1.10 and release it with v1.10.2