bee
bee copied to clipboard
ethersphere
Often returned feeds are not the last published on network
Context
I've different nodes, I can publish a feed update on the first, and require from the second.
Summary
Often the feed index changes based on the node that receive the request.
Expected behavior
If I ask for the last, it should be always the last
Actual behavior
Having load balanced nodes, requiring the feed can produce unexpected behaviors.
Steps to reproduce
Try to push several feeds updates on a node, and try to receive from the receiving one, and another
Possible solution
Don't know.
A possible solution could be to pass to bee an optional parameter with a minimum feed index to find. If I know (me, the client) that an already seen feed index was at least a specific value, I also know that every response with the index less than it is invalid. And I can pass this information to bee. In this way bee can look for any index >= to it, and return 404 if doesn't found anything. That is a better response than an invalid one. If I don't pass this parameter, any feed-index response is valid, as now.
It is possible to make this change in the client side if you are using bee-js. Looking up the feed update is just using the index that is returned and the topic of the single owner chunk is calculated based on that.
Also as a workaround I may suggest to use a persistent cookie for the loadbalancer to return, so that subsequent requests in the same session would be served by the same server in the loadbalancer. This could mitigate the data inconsistencies which may happen during reads, but this gets especially important when writing, otherwise it's possible to end up with different single -owner chunks for the same address.
Yes, writing is already addressed with the postage batch, that will find the exact node owning it. We need to find a solution with downloads, but I need to get more into how bee-js works.
It is possible to make this change in the client side if you are using bee-js. Looking up the feed update is just using the index that is returned and the topic of the single owner chunk is calculated based on that.
I think the problem here is that the feed index returned from one node is not the most recent, so I don't think it is possibile to resolve this using bee-js
It is possible to make this change in the client side if you are using bee-js. Looking up the feed update is just using the index that is returned and the topic of the single owner chunk is calculated based on that.
I think the problem here is that the feed index returned from one node is not the most recent, so I don't think it is possibile to resolve this using bee-js
The proposal from @tmm360 was to pass an optional parameter with a minimum feed index to find. and if the returned index is less than that ignore that. I wrote answering to this that this is possible to do with bee-js / on the client side if it uses the feed API and not the bzz API.
Actually the complete feed lookup algorithm is possible to do in bee-js, only by using the /soc endpoints.
Unfortunately it's difficult to use only bee-js in our situation. We provide a gateway that can see traffic through it. It can store for each returned feed it's maximum returned index, and can inject that value for each new request as I suggested.
Instead, in a case of a pure bee-js use, the final user requests a feed, and we don't do any request modification. Problem is that user, opening the page, can't know what the last feed-index was, so can't request it specifically. We don't have any way to inform users on last feed indexes, except to provide it with a specific query on our gateway, but it's not a very elegant solution.
We was using feeds wrong. Closing this now, will open again if we will found more specific issues.