finos
Question: Strategy for Latest Context between two types: `fdc3.instrument` and `fdc3.instrumentList` while ignoring others
Question Area
- [ ] App Directory
- [ ] API
- [x] Context Data
- [ ] Intents
- [ ] Use Cases
- [ ] Other
Question
Many of the apps in our suite handle both fdc3.instrument and fdc3.instrumentList and currently it's unclear how an app joining a channel with context data available can get the "latest" between two types, while ignoring other context types.
Three apps — A, B and C:
App A and App B joins user channel 1
App A broadcasts an fdc3.instrument context
App B broadcasts an fdc3.instrumentList context
App A broadcasts and fdc3.contact context
App C joins user channel 1
App C adds a context listener (using fdc3.addContextListener(null, callback) )
Is there a way to determine which one is the most recent context between instrument and instrumentList?
If you add a null typed context listener to a user channel, you'll receive the most recent context (of any type) immediately - so checking its type will tell you. If you add a typed listener, you'll get the most recent context of that type instead - you can also manually retrieve those types with channel.getCurrentContext.
If you needed the most recent type + the most recent context of each type you could do this in app C:
await fdc3.addContextListener(null, handler); //will receive most recent type
const currentChannel = await fdc3.getCurrentChannel();
const lastestInstrument = null, latestinstumentList = null;
if (currentChannel) {
latestInstrument = await currentChannel.getCurrentContext("fdc3.instrument");
latestInstrumentList = await currentChannel.getCurrentContext("fdc3.instrumentList");
}
I'd do it that way rather than adding 3 different listeners as I'm assuming you want a single listener to handle all types, and through that to know which was most recent.
Does that help?
Hi @kriswest , thank you for the prompt reply.
Invoking the getCurrentContext does not let us infer which is the latest, so that's not an option we're considering.
The Standard says that a context listener for null should return the latest context regardless of type, but does that mean only the latest should get sent or all types ordered by most recent?
In the below screenshot we can observe that when you broadcast a context type the first time, it gets added to some internal dictionary in the order they were first sent.
But broadcasting the same type a second time , makes no difference to the output.
(here i have a simple fdc3.addContextListener(null, console.log) that I remove and add to the channel between broadcasts)
We are trying to evaluate if this is a gap in the Standard or a bug in the Desktop Agent implementation.
@Bruno-LSEG yes fdc3.getCurrentContext(someType) doesn't tell you what was most recent, just the most recent of that type. However, fdc3.getCurrentContext(null) should give you the latest of any type, as will just adding a context listener with a null type with fdc3.addContextListener(null, handler);.
Its worth noting that (unlike user channel listeners) app channel listeners do NOT give you the latest context (i.e. const chan = await fdc3.getCurrentChannel(); chan.addContextListener(null, handler)). behaves differently to àwait fdc3.addContextListener(null, handler).
The Standard says that a context listener for null should return the latest context regardless of type, but does that mean only the latest should get sent or all types ordered by most recent?
Only the most recent context (of the specified type or any type if null was specified). If your handler is being passed anything other than a single context object, then the DA implementation is not conformant to the Standard.
In the below screenshot we can observe that when you broadcast a context type the first time, it gets added to some internal dictionary in the order they were first sent.
But broadcasting the same type a second time , makes no difference to the output.
That will be a map internal to the FDC3 client used to cache the most recent context of each type so that it can handle queries for the most recent message of each type - hence, its not changing as its just replaced the entry (in your example the second contact's ric is different, you can check that that changed). There will also be a flag, timestamp or duplicate tracking which was the most recent of any type I imagine.
We are trying to evaluate if this is a gap in the Standard or a bug in the Desktop Agent implementation.
From what you've said I think your DA is working correctly according to the standard. the FDC3 API doesn't give you back an ordered list of the latest contexts, it will only give you the latest of each type and of any type. If you need something more than that, please outline the use case or problem you are tying to solve and I'll see if i can suggest something.
Thanks for the clarification @kriswest.
What we are aiming for is making sure we can offer a consistent user experience, imagine this scenario:
The user selects multiple instruments from a Portfolio View App, broadcasting an fdc3.instrumentList with, for example, 15 instruments.
A News App picks that up and shows the list. The user then clicks on an individual entry within the News App, broadcasting an fdc3.instrument.
Now all the apps in the screen are showing that single fdc3.instrument since they all are on the channel.
Finally, the user opens a Chart App and joins it to the channel. The chart displays 15 lines in it, one for each of the instruments in the fdc3.instrumentList (which is now "stale"). The user has to click on the single instrument again to bring the Chart App back to a synced state with the rest of the platform.
It would be very useful if the standard could encourage or enforce some form of ordering in the context types sent to an app joining a channel.
That way, in our example, the Chart App would know how to differentiate which context is latest and could show the user the appropriate data the first time.
Finally, the user opens a Chart App and joins it to the channel. The chart displays 15 lines in it, one for each of the instruments in the fdc3.instrumentList (which is now "stale"). The user has to click on the single instrument again to bring the Chart App back to a synced state with the rest of the platform.
It would be very useful if the standard could encourage or enforce some form of ordering in the context types sent to an app joining a channel.
We do enforce a very limited ordering, what was the most recent context? Hence, in your example, when the chart app adds a null typed listener it should receive the instrument that was broadcast later than the list. If thats not happening - i.e. the you receive the wrong context type back then take that up with your DA vendor, as that would not be conformant to the Standard and wouldn't pass the conformance tests. If you can't resolve that with them you can inform FINOS who can mandate a retest of the Desktop Agent's conformance.
However, the order that you are adding the listeners may matter. If you add the null listener first, it will probably fire the handler with the instrument immediately, then if you add a typed listener, you get the instrumentList and fire that handler...
You can work around by either customizing the order that you add the listeners, only adding a null type listener (filter inside the handler for types) OR call fdc3.getCurrentContext(null) after adding them and prefer what it gives you for the last context.
@Bruno-LSEG does the above help? I strongly suspect that the number and order of listeners that you add is what's causing the issue that your dealing with. Please close this issue if you're all set - otherwise I'd be happy to bring this into a meeting to address further.
This could also be solved by my proposal, #1290 . This involves sharing metadata along with contexts, which includes the timestamp of when a context was raised. If a firm wants to determine whether an fdc3.instrument or fdc3.instrumentList context was called first, they would be able to look at that context's originating timestamp. That is, once #1290 was adopted.
Thanks @julianna-ciq - yes a timestamp would provide another solution, that's easy to intuit.
In the mean time I suspect @Bruno-LSEG needs to switch to a single context listener for the null type to process all context types. If I'm reading the issue right that is - let us know!
I also assumed that getCurrentContext would solve this but in a situation where the current context is something not supported by your app eg an fdc3.contact in the example above - you would not be able to tell which of the fdc3.instrument or fdc3.instrumentList was actually the latest sent...
So, I think the request for an ordered list to be retuned when adding a context listener for the null type makes sense. Yes, if originating timestamp was included you could compare those but perhaps it's easier and/or cleaner to just go through the list and pick the first or last context type that your app support. Perhaps the first in the list should be latest so that you don't need to process more than necessary?
@Bruno-LSEG - please let us know if this would solve your need?
Thank you for the comment @julianna-ciq , a timestamp could help greatly, even if the Platform Provider doesn't do any sort of ordering the apps themselves could look at that to make a choice.
@openfin-johans , yes that would solve our need. Whether it's first or last, as long as it's standardized so that we know what to pick.
I also assumed that getCurrentContext would solve this but in a situation where the current context is something not supported by your app eg an fdc3.contact in the example above - you would not be able to tell which of the fdc3.instrument or fdc3.instrumentList was actually the latest sent...
This is not the case. Both fdc3.addContextListener(null) and fdc3.getCurrentContext(null) are required to return you the latest context of any type. This is tested as part of the conformance tests for Desktop Agents and any agent that doesn't respond correctly fails that test.
At present this issue does not clearly articulate a problem that is not solved by the current API standard. None of the examples demonstrate how listeners are being added to the app and how it is being mislead as to the most recent context on the channel. These are some of the oldest API calls and most core use cases for FDC3 and have been used by apps to display the most recent context (potentially from a number of types) for many years.
The change @julianna-ciq flags will provide an alternative way for you to look at this data, if adopted and implemented, in future.
@Bruno-LSEG If you post an example of how you are adding listeners the community will be able to provide support on where its going wrong or should be adapted. Otherwise, I believe this issue should be closed as multiple proven strategies are already provided.
@openfin-johans If memory serves we talked about retaining a history of contexts on a channel beyond the most recent of each type some years ago. The idea raises a whole load of question:
- How many messages should be retained, and for how long?
- Does this lead to memory bloat (as that history may end up existing in many different app contexts in addition to the Desktop Agent)?
- Should an app joining a channel really be able to access its history beyond the 'current' context?
- How will implementations end up varying between agents?
In discussion we determined that:
- almost all the use cases presented are solved for by being able to access the most recent context of any type and the most recent context of each type.
- the only exception being a widget or app for displaying the context history - although this would typically be solved by having an ever present UI component or service app listen on the channel from startup so it can record that history.
- the current solution avoids all the questions (and corresponding added complexity in FDC3) above.
Hey @kriswest - it's very likely that I am missing something - it's Friday and the hottest day of the year :)
11:00 App A and App B both join channel green 11.01 App A broadcasts an instrument 11.02 App A broadcasts an instrumentList 11.03 App B broadcasts a contact 11.04 App C joins channel green
App C only supports instrument and instrumentList - but how can it determine that instrumentList was sent after instrument in this case?
Thanks
Johan
@openfin-johans That is a more difficult example than the original question posed. If the app in question doesn't support one of the types and wishes to ignore it, and it was the most recent type, then while you can retrieve the latest of either type all you know is that that neither was the latest on the channel. This is enough for most usecases, including filter setting on blotters etc.. but I can see that you can construct flows that might depend on being able to ignore an unsupported type.
If that is the use case, the issue title and description should be updated to mention ignoring an unsupported type.
I remain of the opinion that retaining a history of contexts on a channel is not the best idea for the reasons given previously - although it is of course possible to control the complexity/bloat with work. E.g. If we limit the history to one per type there would still be oddities to handle (e.g. multiple of the same type interleaved with other types would be collapsed to the most recent).
Context metadata containing a timestamp seems a simpler solution for establishing the latest type initially. However, I'd probably want to use fdc3.getCurrentContext(type) to retrieve each type, to work around the event handler style of fdc3.addContextListener(type) - but it doesn't currently return ContextMetadata and would need a breaking change to do so - @julianna-ciq see https://fdc3.finos.org/docs/api/ref/Channel#getcurrentcontext. Having to add separate handlers for each context type, then wait for them to maybe each call their callbacks (i.e. if there is a current context), before bootstrapping your component is going to be a mess of setting timeouts. Perhaps another approach would be to return the current context and contextMetadata as a property of the Listener object returned by fdc3.addContextListener (and similar functions)?
Finally, none of this provides an answer usable today. A workaround is possible with a service app thats hanging out on channels and maintaining a history datastructure for each on an app channel might provide a workaround... and perhaps some feedback on what approach to keeping the history is most workable.
It is worth noting that the implied 'equivalence' between an Instrument context and an Instrument List context, in the original question from Bruno-LSEG, is an application level decision. So I think just adding a timestamp to each typed context allows applications to decide how to interpret the Desktop Agent context i.e. the Dictionary of Context Types available when an application instance connects to a channel.
Hi @kriswest , i don't believe retaining the whole history of contexts is the way to go. We just need to have the fdc3.addContextListener(null) invoking the callback in an ordered manner, the order being based on when each context type was last sent.
The example from @openfin-johans is simple and shows the situation quite clearly.
So from @openfin-johans 's example, App C would add a context listener for
fdc3.addContextListener(null (c) => {
if(c.type === 'fdc3.instrument' || c.type === 'fdc3.instrumentList') {
// 100ms debounce where it ignores other contexts to execute workflowLogic
debounceEager(100, () => workflowLogic(c));
}
});
and the order of callbacks would be:
contact
instrumentList
instrument
That way AppC would take the first accepted context it receives and ignore the others for the duration of the debounce.
We cannot rely on calling fdc3.getCurrentContext() due to having many different context types flowing in the platform and not all apps support all context types or execute the same logic for all context types.
I have updated the title and description to be more factual.
@Bruno-LSEG fdc3.addContextListener(null, handler) is already ordered in that it is called for the last context of any type only. What I think you are asking for is for it to be called for every current type on the channel. That's not likely appropriate for every application and would be a hefty, likely breaking change in behaviour for existing apps. Thats very likely to lead to a flickering effect in some apps as they re-render multiple times on receiving those callbacks in quick succession. Requiring every app to the debounce those calls adds complexity for every app and would require a major revision to FDC3 version numbering to do (as its not an addition to functionality, but a change to existing functionality).
We cannot rely on calling fdc3.getCurrentContext() due to having many different context types flowing in the platform and not all apps support all context types or execute the same logic for all context types.
You'd only need to call getCurrentContext(type) for types that you support (I'm not propose you call it with a null type, rather call it for each type you want to know about). In fact, I think you are identifying an issue with what you propose for fdc3.addContextListener(null, handler) - the handler will get called for every type including those you don't support, without you knowing in advance how many times its going to get called. Whereas with channel.getCurrentContext(type) you know what types you are getting and exactly how many calls you are making, exactly what logic you will apply to each. However, you also need the timestamp info to get the ordering - which again is likely to be a breaking change to the return type.
That makes our options (at least in my opinion):
- Triggering an FDC3 3.0 release with a breaking solution.
- Creating something new (e.g. a
channel.getAllCurrentContext, with a return type that includes the metadata).
@julianna-ciq we probably need to think about the channel.getCurrentContext issue for metadata anyway...
Should I pop this issue onto the agenda for Thursday's Standards Working Group meeting @Bruno-LSEG @openfin-johans @julianna-ciq ?
Based on discussion both here and during the SWG meeting today, I'd like to propose 2 potential solutions. Note that these are provided on the assumption that the #1290 issue moves forward, so it will include a metadata.timestamp property:
Option 1
Extend the addContextListener to support an array of types. For example:
fdc3.addContextListener(["fd3.instrument", "fdc3.instrumentList"], (contextList, metadataList) => {
/* contextList (ordered by most recent sent): [
{
"type": "fdc3.instrumentList",
// ...
},
{
"type": "fdc3.instrument"
// ...
}
]
*/
/* metadatasList (ordered to match the contextList): [
{timestamp: ..., source: ..., ... },
{timestamp: ..., source: ..., ... }
*/
});
This would be a matter of expanding the addContextListener. Whether or not the returned list should be in the order of recency, versus the order of the types provided in the type list, would be open for discussion.
Option 2
Add a new method, getAllCurrentContext
I could imagine this called as await fdc3.getAllCurrentContext(), and then returns an object like this:
{
"fdc3.contact": [
{
"type": "fdc3.contact",
"id": {
"ric": "[email protected]"
}
},
{
"timestamp": ...,
"source": ....
}
],
"fdc3.instrument": [
{
"type": "fdc3.instrument",
"id": {
"ric": "LSEG.L"
}
},
{
"timestamp": ...,
"source": ...
}
],
"fdc3.instrumentList": [...]
}
In this case, every context would be returned as a dictionary of type and tuple (context, metadata). This would provide all the information necessary to determine which came first, while also being fairly extensible for other use cases.
How do people feel about either of these options? I believe, based on the conversation with Albert and Bruno, that both would solve LSEG's current issue.