Transform a document before publishing

[steph643]

As you know, there are two methods to implement a publish function:

1. By returning a cursor (or an array of cursors)
2. By using this.added() / this.changed() / this.removed(), usually from cursor.observeChanges() callbacks

Let's consider these three use cases for method 2: a. You want to add a field to a document before publication b. You want to remove a field from a document before publication c. You want to do both, for example to combine several secret fields into one public field

Does meteor-publish-composite allow for these three use cases?

[benstr]

+1. I found a useful how-to on the subject but would like to know if publish-composite has an easier way of accomplishing this. My ultimate goal is sort: on my subscriptions need to include fields from children publications. By merging documents during publication into a custom collection would solve the issue for me.

[reywood]

Unfortunately, there is no way to do this with publishComposite right now. I'll leave this open as an enhancement request for now.

[serkandurusoy]

Please take a look at https://forums.meteor.com/t/add-subscriptionid-to-documents-during-subscription/8051/12?u=serkandurusoy

I've actually been using transform within the find functions, just to avoid using verbose observe syntax. And it turns out, I actually can't use transform in regular publish and observe with publishComposite.

To be frank, I really don't mind (at least currently with my usecases), but I would like to read an explanation about the inner workings of publishComposite, precisely about the difference between a regular publish function.

And I'd really very much like to hear your opinions on the use of transform within publishComposite find functions. Is it a bad idea?

[reywood]

This is definitely tricky. publishComposite uses both observe and observeChanges. It uses observe because it needs the whole document when updating child publications. The observe handler is also used to send "added" and "removed" updates to the client. It uses observeChanges to send changes to the client, because, as mentioned elsewhere, it is more efficient. The upshot is that transforms are applied to documents when they are "added". The catch is that transforms are not applied when a document is "changed" due to the use of observeChanges. If transforms were applied to changes, it's not clear how that would work since you'd only have the changed fields, not the whole document.

So, to address @steph643's original questions: a. Yes, you can use the transform option of the Collection.find method to add fields to a document before publication. However, the transform will not be called if/when the document changes. This means that if a field used in your transform changes, the transformed fields will not be updated. b. If you want this removed field to still be visible to child publications (which I presume is what you want, otherwise, why bother), then you're currently out of luck. c. See answers to a and b

I've considered adding a transform option to publishComposite, but this would still have the issues described above when a document changes. I'm open to thoughts and ideas.

[maxnowack]

I've created a package for transformations in publications: https://atmospherejs.com/maximum/server-transform a. can be done using my package. b. & c. using my package in combination with the fields option on the cursor.

[Gaelan]

I would like a way to ignore transforms when publishing. I use an ORM-like pattern which uses transform functions to have queries return proper objects instead of the raw mongo data. However, I still want to send plain Mongo data to the client.

[maxnowack]

@Gaelan this behavior should be the meteor default

Transforms are not applied for the callbacks of observeChanges or to cursors returned from publish functions. (http://docs.meteor.com/#/full/mongo_collection)

In your case, the transform function will be executed in the client also. You can instantiate your mongo collection differently on client and server and use different transform functions.

[animesh1987]

Any updates on this issue?

[holmrenser]

A very relevant use of this feature would be the use of the aggregation functionality, especially since MongoDB 3.4. For this option #2 of @steph643's original post would be a requirement to turn the aggregation results back into a publication.

[vbgm]

I am using currently publish composite, and for now it does the a. with the 'transform' function, which is enough in my current use case, but I am afraid I might require the b. and c. and will have to switch to peerlibrary:reactive-publish

Therefore +1 to this.

[vizet]

Any updates on this? Now I have a problem... I have to use transformations + related collections on publishing. +1