dcache
dcache copied to clipboard
dCache
info output empty every now and then since 4.2
Hi.
Since upgrading to 4.2.x we're having a problem that our monitoring shows interruptions in the data every now and then like this:
(both show the used size of a space reservation, each at a different scale)
Turns out that our montoring tool, which downloads http://lcg-lrz-dcache0.grid.lrz.de:59998/info/reservations every 5 mins gets back an empty XML reply like this:
<?xml version="1.0"?>
<dCache xmlns="http://www.dcache.org/2008/01/Info"/>
every time the graph gets no data.
Port 59998 is the httpd, the exact version is 4.2.23-1.
There's nothing special in the info log, apart from the usual spaming happening since… (#1970):
Feb 10 20:12:54 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:12:54+01:00 (info) [] DGA CellInfoDga triggering too quickly (-23ms): engaging safety.
Feb 10 20:14:05 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:14:05+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-22ms): engaging safety.
Feb 10 20:14:54 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:14:54+01:00 (info) [] DGA CellInfoDga triggering too quickly (-23ms): engaging safety.
Feb 10 20:15:05 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:15:05+01:00 (info) [] DGA ListBasedMessageDga[PoolManager, pools, psux ls pool] triggering too quickly (41ms): engaging safety.
Feb 10 20:16:05 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:16:05+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-16ms): engaging safety.
Feb 10 20:16:54 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:16:54+01:00 (info) [] DGA CellInfoDga triggering too quickly (-23ms): engaging safety.
Feb 10 20:17:34 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:17:34+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (-37ms): engaging safety.
Feb 10 20:18:05 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:18:05+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-24ms): engaging safety.
Feb 10 20:18:54 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:18:54+01:00 (info) [] DGA CellInfoDga triggering too quickly (-25ms): engaging safety.
Feb 10 20:20:05 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:20:05+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-24ms): engaging safety.
Feb 10 20:20:54 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:20:54+01:00 (info) [] DGA CellInfoDga triggering too quickly (-22ms): engaging safety.
Feb 10 20:22:06 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:22:06+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-25ms): engaging safety.
Feb 10 20:22:34 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:22:34+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (-18ms): engaging safety.
Feb 10 20:22:54 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:22:54+01:00 (info) [] DGA CellInfoDga triggering too quickly (7ms): engaging safety.
Feb 10 20:24:06 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:24:06+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-25ms): engaging safety.
Feb 10 20:24:54 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:24:54+01:00 (info) [] DGA CellInfoDga triggering too quickly (-23ms): engaging safety.
Feb 10 20:25:06 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:25:06+01:00 (info) [] DGA ListBasedMessageDga[PoolManager, pools, psux ls pool] triggering too quickly (44ms): engaging safety.
Feb 10 20:26:06 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:26:06+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-22ms): engaging safety.
Feb 10 20:26:54 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:26:54+01:00 (info) [] DGA CellInfoDga triggering too quickly (-16ms): engaging safety.
Feb 10 20:27:35 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:27:35+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (-41ms): engaging safety.
Feb 10 20:28:06 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:28:06+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-24ms): engaging safety.
Feb 10 20:28:55 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:28:55+01:00 (info) [] DGA CellInfoDga triggering too quickly (-23ms): engaging safety.
Feb 10 20:30:06 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:30:06+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-19ms): engaging safety.
Feb 10 20:30:55 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:30:55+01:00 (info) [] DGA CellInfoDga triggering too quickly (-18ms): engaging safety.
Feb 10 20:32:06 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:32:06+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-24ms): engaging safety.
Feb 10 20:32:35 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:32:35+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (-39ms): engaging safety.
Feb 10 20:32:55 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:32:55+01:00 (info) [] DGA CellInfoDga triggering too quickly (-22ms): engaging safety.
Feb 10 20:34:06 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:34:06+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-26ms): engaging safety.
Feb 10 20:34:55 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:34:55+01:00 (info) [] DGA CellInfoDga triggering too quickly (-24ms): engaging safety.
Feb 10 20:36:06 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:36:06+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-24ms): engaging safety.
Feb 10 20:36:55 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:36:55+01:00 (info) [] DGA CellInfoDga triggering too quickly (-21ms): engaging safety.
Feb 10 20:37:35 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:37:35+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (-15ms): engaging safety.
Feb 10 20:38:07 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:38:07+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-13ms): engaging safety.
Feb 10 20:38:55 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:38:55+01:00 (info) [] DGA CellInfoDga triggering too quickly (-18ms): engaging safety.
Feb 10 20:40:07 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:40:07+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-12ms): engaging safety.
Feb 10 20:40:55 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:40:55+01:00 (info) [] DGA CellInfoDga triggering too quickly (-21ms): engaging safety.
Feb 10 20:42:07 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:42:07+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-18ms): engaging safety.
Feb 10 20:42:35 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:42:35+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (-21ms): engaging safety.
Feb 10 20:42:55 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:42:55+01:00 (info) [] DGA CellInfoDga triggering too quickly (-22ms): engaging safety.
Feb 10 20:44:07 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:44:07+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-22ms): engaging safety.
Feb 10 20:44:56 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:44:56+01:00 (info) [] DGA CellInfoDga triggering too quickly (-37ms): engaging safety.
Feb 10 20:46:07 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:46:07+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-22ms): engaging safety.
Feb 10 20:46:56 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:46:56+01:00 (info) [] DGA CellInfoDga triggering too quickly (-18ms): engaging safety.
Feb 10 20:47:35 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:47:35+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (-8ms): engaging safety.
Feb 10 20:48:07 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:48:07+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-14ms): engaging safety.
Feb 10 20:48:56 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:48:56+01:00 (info) [] DGA CellInfoDga triggering too quickly (-23ms): engaging safety.
Feb 10 20:50:07 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:50:07+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-22ms): engaging safety.
Feb 10 20:50:56 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:50:56+01:00 (info) [] DGA CellInfoDga triggering too quickly (-19ms): engaging safety.
Feb 10 20:52:07 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:52:07+01:00 (info) [] DGA StaticDomainDga triggering too quickly (5ms): engaging safety.
Feb 10 20:52:08 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:52:08+01:00 (info) [] DGA ListBasedMessageDga[PoolManager, units, psux ls unit] triggering too quickly (-61ms): engaging safety.
Feb 10 20:52:35 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:52:35+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (-10ms): engaging safety.
Feb 10 20:52:56 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:52:56+01:00 (info) [] DGA CellInfoDga triggering too quickly (-13ms): engaging safety.
Feb 10 20:54:08 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:54:08+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-22ms): engaging safety.
Feb 10 20:54:56 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:54:56+01:00 (info) [] DGA CellInfoDga triggering too quickly (-36ms): engaging safety.
Feb 10 20:56:08 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:56:08+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-23ms): engaging safety.
Feb 10 20:56:56 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:56:56+01:00 (info) [] DGA CellInfoDga triggering too quickly (-21ms): engaging safety.
Feb 10 20:57:36 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:57:36+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (-20ms): engaging safety.
Feb 10 20:58:08 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:58:08+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-19ms): engaging safety.
Feb 10 20:58:56 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 20:58:56+01:00 (info) [] DGA CellInfoDga triggering too quickly (-38ms): engaging safety.
Feb 10 21:00:08 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 21:00:08+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-6ms): engaging safety.
Feb 10 21:00:56 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 21:00:56+01:00 (info) [] DGA CellInfoDga triggering too quickly (-7ms): engaging safety.
Feb 10 21:01:08 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 21:01:08+01:00 (info) [] DGA ListBasedMessageDga[PoolManager, poolgroups, psux ls pgroup] triggering too quickly (-330ms): engaging safety.
Feb 10 21:01:08 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 21:01:08+01:00 (info) [] DGA ListBasedMessageDga[PoolManager, pools, psux ls pool] triggering too quickly (-48ms): engaging safety.
Feb 10 21:02:08 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 21:02:08+01:00 (info) [] DGA StaticDomainDga triggering too quickly (-19ms): engaging safety.
Feb 10 21:02:36 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 21:02:36+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (-17ms): engaging safety.
Feb 10 21:02:57 lcg-lrz-dcache0 dcache@info0[219026]: 2019-02-10 21:02:57+01:00 (info) [] DGA CellInfoDga triggering too quickly (-17ms): engaging safety.
Nothing in the httpd log.
Any ideas?
Cheers, Chris.
And it also affects critical stuff, like the output from /usr/sbin/dcache-storage-descriptor is broken all few invocations
Sorry for the delay.
Could you check the space-manager log file, to see if there are any errors reported there?
Cheers, Paul.
Could you check the space-manager log file, to see if there are any errors reported there?
No:
/var/log/dcache# grep -i SpaceManager storagesystem0.log
/var/log/dcache#
but we do have a permanent:
Nov 16 16:27:25 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:27:25+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (28ms): engaging safety.
Nov 16 16:28:50 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:28:50+01:00 (info) [] DGA StaticDomainDga triggering too quickly (26ms): engaging safety.
Nov 16 16:29:18 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:29:18+01:00 (info) [] DGA CellInfoDga triggering too quickly (27ms): engaging safety.
Nov 16 16:30:50 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:30:50+01:00 (info) [] DGA StaticDomainDga triggering too quickly (25ms): engaging safety.
Nov 16 16:31:18 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:31:18+01:00 (info) [] DGA CellInfoDga triggering too quickly (26ms): engaging safety.
Nov 16 16:32:25 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:32:25+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (26ms): engaging safety.
Nov 16 16:32:50 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:32:50+01:00 (info) [] DGA StaticDomainDga triggering too quickly (25ms): engaging safety.
Nov 16 16:33:18 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:33:18+01:00 (info) [] DGA CellInfoDga triggering too quickly (25ms): engaging safety.
Nov 16 16:34:50 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:34:50+01:00 (info) [] DGA StaticDomainDga triggering too quickly (26ms): engaging safety.
Nov 16 16:35:18 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:35:18+01:00 (info) [] DGA CellInfoDga triggering too quickly (25ms): engaging safety.
Nov 16 16:36:50 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:36:50+01:00 (info) [] DGA StaticDomainDga triggering too quickly (26ms): engaging safety.
Nov 16 16:37:18 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:37:18+01:00 (info) [] DGA CellInfoDga triggering too quickly (27ms): engaging safety.
Nov 16 16:37:25 lcg-lrz-dcache0 dcache@info0[17942]: 2021-11-16 16:37:25+01:00 (info) [] DGA RoutingMgrDga triggering too quickly (26ms): engaging safety.
as reported in #1970. Though this may be unrelated, as we get the warnings even when the data is not missing from info.
Also not, that we're running a bit outdated (6.0.18-1).
Cheers, Chris
Thanks for checking, Chris.
Do you think the SpaceManager database is running slowly?
Perhaps you could enable PostGreSQL's slow query logging feature for the space-manager database and see if there are any queries taking longer than 100 ms (just to pull a duration out of thin air)?
You should also upgrade your dCache version.
I strongly doubt upgrading from 6.0 to 6.2 (or newer) will fix this problem; however, that version is now out of support. Therefore, you'll need to upgrade anyway, once this problem is fixed :wink:
Cheers, Paul.
No, I don't think there are DB load issues... the node has generally a very low load... and I think we have slow query logging even on already and it doesn't give any.
Especially since the problem occurred right after upgrading to 4.2.x without any other major config changes, it seems rather like a regression.
Cheers, Chris.
This is also still a bug in 2.7.10... and numerous sites suffer from it, with output like that of dcache-storage-descriptor being broken, leading to errors on other ends...
Isn't there anything one can do to fix this?!?
Can it be that /info/reservations is completely gone from the info’s XML?!?!!?!
This might be another issue:
[dCache0] (info@info2) admin > watchers ls
State Watchers:
PoolgroupSpaceWatcher (enabled, triggered: 862)
PoolsSummaryMaintainer (enabled, triggered: 68)
LinkgroupTotalSpaceMaintainer (enabled, triggered: 44)
LinkSpaceMaintainer (enabled, triggered: 69)
NormalisedAccessSpaceMaintainer (enabled, triggered: 69)
ReservationByDescMaintainer (enabled, triggered: 0)
Seems it doesn't upgrade the Reservations.. but why?
ping @paulmillar @kofemann ... I was told today that numerous ATLAS sites suffer from this.
Hi Chris,
can you elaborate on ATLAS sites suffer from this?Any details? We have seen any report.
Hey Tigran.
I've asked Günter/Rod, which sites they were exactly talking about.
But I see at least CSCS, for which a ticket has been filed in GGUS,... I know Freiburg had at least the every-now-and-then interruptions, cause they wanted my workaround for it (not sure, whether they suffer from the complete blackout - probably not, if they still run an older version).... and in your RT, there is BNL which reported the same issues (ticket 10285, next to my 10289).
Thanks, Chris.
And I just received mail from Rodney, who claims that it was said in some ATLAS meeting that these suffer from the issues:
Non-operational storage reporting at CSCS-LCG2, DESY-ZN, GoeGrid, IEPSAS-Kosice, ifae, LRZ-LMU, pic (dCache SRR)
But I'm not exactly sure which exact issue all these are suffering from... and whether it's the same complete blackout (that we have since 7.2.x) or the every-now-and-then-issues, we've had since 4.2.
Hi Tigran,
can you elaborate on
ATLAS sites suffer from this?Any details? We have seen any report.
see GGUS ticket https://ggus.eu/index.php?mode=ticket_info&ticket_id=156280 for DESY-ZN.
I should perhaps add, there this issue may likely be actually two independent issues:
- the one I see since 4.2 / 2019, which caused the info output to be missing every now and then
- the one that I see, since I've upgraded from 6.0.18 to 7.2.x, which causes it to be always empty
fixing either may or may not fix the other.
For SRR reporting please use the frontend service as described at:
https://github.com/dCache/dcache/blob/master/docs/TheBook/src/main/markdown/srr.md#wlcg-storage-resource-reporting
Please note, that by default, srr information is limited to localhost only. Moreover, the frondend service itself can be limited to localhost
[srrDomain]
[srrdDomain/frontend]
frontend.authn.basic=true
frontend.authn.protocol=http
frontend.authz.anonymous-operations=READONLY
frontend.srr.shares=user:/cms,store:/cms
frontend.net.listen=127.0.0.1
I assume that means that this bug is not going to be fixed?!
As I've already said, we're using the info provider (which seems the broken part here) not only for SRR, so just switching to frontend/REST isn't really a help. Also, as described by Andreas in #6534, frontend seems to have it's own issues.
Plus, as mentioned before, many sites still use dcache-storage-descriptor, and I don't remember that there'd have been any deprecation notice of that or of the info provider (and usually I read the release notes quite thoroughly)... given that in the past, dcache.org has always spent quite some effort to keep things stable (just think about the many legacy options that were kept just in order to not force admins for 1-2 further release to switch to their new names)... it would be more than strange that now suddenly out of the blue a component breaks within a major release, and people a forced to migrate to something completely different.
Hi Chris,
There's nothing (said so far) so say this problem will not be fixed. The issue has not been closed!
Tigran commented noted that the problem "ATLAS sites suffer from this" is actually about specifically SRR. For this specific case, there is a preferred solution, as he describes.
Cheers, Paul.
Well but as you see from Andreas' #6534 and my #6554, this doesn't really work either - apart from the fact that there are sites which may not be prepared/willing to enable frontend right now.
@drfalken ,
Thanks for reporting your experience.
When you say you're seeing "the same issue", is this the occasional missing information that Chris originally reported, or the complete absence of information he found when upgrading to 7.2?
Cheers, Paul.
Hi Paul, yes this is absence of information when upgrading to 7.2. Cheers.
Il giorno dom 20 mar 2022 alle ore 09:40 Paul Millar < @.***> ha scritto:
@drfalken https://github.com/drfalken ,
Thanks for reporting your experience.
When you say you're seeing "the same issue", is this the occasional missing information that Chris originally reported, or the complete absence of information he found when upgrading to 7.2?
Cheers, Paul.
— Reply to this email directly, view it on GitHub https://github.com/dCache/dcache/issues/4622#issuecomment-1073199601, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHLRKTR6VDSKFTWJRHOGB3VA3P6XANCNFSM4GWN2GGA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.***>
-- "Greetings, Professor Falken. A strange game. The only winning move is not to play. How about a nice game of chess? "
Seems this has been fixed by 253e07ab7d ... well better late than never ;-) though it unfortunately caused quite some breakage since then and has required lots of code-rewrite for quite some people that relied on info...
And now it's still uncertain... how long will info continue to work?