Error calling GET https://vendor-list.consensu.org/v3/vendor-list.json

[matthieularere-msq]

Hello,

When starting prebid-server, I always have a bunch of output error like these :

E1024 19:36:51.458683 1100157 vendorlist-fetching.go:127] Error calling GET https://vendor-list.consensu.org/v2/archives/vendor-list-v98.json. Cookie syncs may be affected: context deadline exceeded
(...)
E1024 19:36:51.458711 1100157 vendorlist-fetching.go:127] Error calling GET https://vendor-list.consensu.org/v2/archives/vendor-list-v217.json. Cookie syncs may be affected: context deadline exceeded
E1024 19:36:51.458715 1100157 vendorlist-fetching.go:127] Error calling GET https://vendor-list.consensu.org/v2/archives/vendor-list-v218.json. Cookie syncs may be affected: context deadline exceeded
E1024 19:36:51.458720 1100157 vendorlist-fetching.go:127] Error calling GET https://vendor-list.consensu.org/v2/archives/vendor-list-v219.json. Cookie syncs may be affected: context deadline exceeded
E1024 19:36:51.458726 1100157 vendorlist-fetching.go:127] Error calling GET https://vendor-list.consensu.org/v3/vendor-list.json. Cookie syncs may be affected: context deadline exceeded

I've quickly added a few more output by adding an else in gdpr/vendorlist-fetching.go :

if err != nil {
	glog.Errorf("Error calling GET %s. Cookie syncs may be affected: %v", url, err)
	return 0
} else {
	glog.Errorf("Found GET %s. ", url)
}

Which confirm me that files from v2/vendor-list.json to v2/archives/vendor-list-v97.json are found, but none of the following ones.

I did not really paid much attention to that sofar as the first to load is the main one to use, https://vendor-list.consensu.org/v2/vendor-list.json. However it kind of bother me that the new https://vendor-list.consensu.org/v3/vendor-list.json is impacted too. Is there any way to fix this ?

[bsardo]

Hi @matthieularere-msq, We're aware of this general problem and are planning to address it soon as part of a fetchers package refactor. As for your specific concern regarding preloading the cache with the latest list version of GVL spec version 3, I can look at tweaking the order in which the GVLs are loaded into cache to address your concern in the short term.

[matthieularere-msq]

Hi @bsardo Without a fix, wouldn't it have an impact once tcf v2.2 is mandatory ? Could a fix on branch 0.x be made ? Thanks

[bsardo]

Hi @matthieularere-msq, This point has been raised before, see https://github.com/prebid/prebid-server/issues/1687#issuecomment-770934307, and is top of mind. The first part of my fetcher package refactor will be focused on this specific fetcher. This shouldn't really impact us when 2.2 is mandatory as anything that isn't successfully preloaded will be loaded and cached on demand.

[bretg]

Is this a duplicate of https://github.com/prebid/prebid-server/issues/3336 ?