access $by_category category name with Pinned Persistent Subscription & stream selection for pinning strategy

[rossbuggins]

When using a persistent subscription in Pinned mode with a stream such as $ce-MyStream, which has MyStream-one, MyStream-two, in C# I can't find reference to -one or -two, only $ce-MyStream.

For some background In this use case a projection is generating MyStream-one, MyStream-two by emiting links to other events. There is logic in the projection that is loading incoming events into their grouped streams. (The original event details is showing this, not this stream name that the projection links them to).

[StevenBlair123]

Ross,

Is MyStream-one, MyStream-two being created from within a projection using linkTo?

[rossbuggins]

Hi Steve, it Sure is Thanks , Ross

[rossbuggins]

I've found it hidden away in the original events meta data:

var metaStr = Encoding.Default.GetString(e.OriginalEvent.Metadata);
var metaObj = Newtonsoft.Json.JsonConvert.DeserializeObject<JObject>(metaStr );
var path = metaObj .Value<string>("$o");

Now I'm seeing the stream name generated by the linkto, it has then lead on to the question of:

The pinning on the server for the persistent subscription , doesn't seem to be pinning to each client based on this value? Example data that is a bit clearer:

Pony-123 {data: {Rider:A}}
Pony-345 {data: {Rider:B}}

A projection is then looking in the data of the Pony events and writing out with linkto

Rider-A
Rider-B

Pinned Persistent Subscription on $ce-Rider

Starting up two consumers and I thought it would then send Rider-A stream events to one client and and Rider-B stream events to the other?

Update: Reading through https://eventstore.org/docs/dotnet-api/competing-consumers/index.html

it says:

Pinned For use with an indexing projection such as the system $by_category projection.

Event Store inspects event for its source stream id, hashing the id to one of 1024 buckets assigned to individual clients. When a client disconnects it's buckets are assigned to other clients. When a client connects, it is assigned some of the existing buckets. This naively attempts to maintain a balanced workload.

The main aim of this strategy is to decrease the likelihood of concurrency and ordering issues while maintaining load balancing. This is not a guarantee, and you should handle the usual ordering and concurrency issues.

I think the key line is

Event Store inspects event for its source stream id

meaning that it will always look at Pony- rather than Rider- for the persistent subscription pinning?

Update 2

By changing my projection to emit to a new stream, rather than link, the persistent stream works as the documentation is written (see update 1).

So the final question is, is there any way to get the persistent subscription to work as I want it to, by using linkto instead of emit (as i don't want to be re writing whole new events, but rather link to the existing ones)?

[oskardudycz]

@ylorph, @hayley-jean, as we discussed, we should try that and document/verify how it's working and how we'd like that to work.