Publication indicates it is ready before all data is published

[sahanDissanayake]

Hi @reywood ,

Is there a way to debug why I'm not getting the joined data from the server ?

This is just returning the results

Meteor.publishComposite('jobs', {
    find() {
        return Jobs.find();
    },
    children: [
        {
            find(job) {
                return Properties.find(
                    {
                        _id: job.propertyID
                    },
                    {
                        sort:{createdAt: -1}
                    }
                );
            }
        },
        {
            find(job) {
                return Meteor.users.find(
                    {_id: job.createdBy},
                    {limit: 1}
                );
            }
        }
    ]
});

As this one

Meteor.publish('jobs', {
  return Jobs.find();
});

[sahanDissanayake]

I;m using react on my project too something similar to this https://github.com/micouz/meteor-react-tutorial-next

[mquandalle]

This may be caused by publishComposite returning the ready DDP message before the remaining data. I have a similar problem while trying to subscribe on the server with FastRender https://github.com/wekan/wekan/issues/431.

[sahanDissanayake]

Yep.. This is an issue.. The ready event comes through before the data is fetched.. Any thoughts on a fix @mquandalle ? @reywood ??

getMeteorData() {

        const propertyHandle = Meteor.subscribe('properties');

        const data = {};
        console.log(handle.ready())
        if( propertyHandle.ready()) {

            data.properties = Properties.find().fetch();
        }

        data.currentUser = Meteor.user()

        return data;
    }

Here the propertyHandle.ready() output true even if the subscription is not ready to send the data down.. Because of this I cannot use FastRender, and React SSR too..

[sahanDissanayake]

@reywood here we go, https://github.com/sahanDissanayake/hello-react-meteor

You could see the difference when you change publishComposite to be normal meteor publish

[sahanDissanayake]

https://github.com/sahanDissanayake/hello-react-meteor/blob/c19a0dbf4417f43ed533fa14791a4c0974b4bbbc/server/publications.js

[sahanDissanayake]

Any thoughts on the issue ??

[mquandalle]

I tried to fix this issue, but it may require a rather large refactoring of the internal implementation. I would love to read @reywood on the matter.

We should take inspiration of Meteor.publish implementation here. Especially it seems that under the hood Meteor use observeChanges instead of `observe to publish the cursor.

[sahanDissanayake]

@mquandalle @reywood Should I open a new issue for the handle.ready() functionality issue ??

[TedAvery]

+1

[eXon]

This fix is badly needed or SSR will be impossible with this package. Is there a way I can help?

[sahanDissanayake]

@reywood Do you think there is a fix for this ? Like @eXon mentioned can we help somehow to get this fixed ?

This is a comment found regarding the issue

So yeah, regarding reywood:publish-composite, englue/meteor-publish-composite#67 is an issue for SSR since it currently marks the subscription as ready as soon as the data for the "primary" cursor is sent, and data for the "secondary/joined" cursors get sent after that.

I'm not sure whether it's the only issue though : trying to use the latest meteor-react-router-ssr with reywood:publish-composite, I'm seeing no data at all in the server-side subscription, even on the primary cursor

[TedAvery]

I'm still fairly new to Meteor, but as a temporary workaround, I believe I achieved what this package helps with by manually chaining together two subscriptions in getMeteorData of my React component.

FormContainer = React.createClass({
  mixins: [ReactMeteorData],
  getMeteorData() {
    var data = {};
    var shortname = this.props.shortname;
    var handle = Meteor.subscribe('form_by_shortname', shortname);
    if (handle.ready()) {
      data.form = Forms.findOne({shortname: shortname});
      var handle2 = Meteor.subscribe('template_by_id', data.form.template);
      if (handle2.ready()) {
        data.form.template = FormTemplates.findOne({ _id: data.form.template });
      }
    }
    return data;
  },
  render() { ... }
}

[sahanDissanayake]

@TedAvery this is good for simple structures,

But when it comes to this https://docs.mongodb.org/manual/core/data-model-design/#normalized-data-models

We need the publishComposite to work as it is suppose to

[yched]

@TedAvery the issue with doing that on the client is that it requires more data roundtrips. That's what this package is supposed to avoid :-)

[davegariepy]

+1

[john-osullivan]

++1

[tcastelli]

+1

[pcorey]

+1 Even from a non-SSR perspective, it would be nice (and expected) if a publishComposite didn't respond ready until all children queries are ready.

[lukehollis]

+1

[MaxTwentythree]

+1 Any news on this?

[stolinski]

+1

[achapela]

+1

[rodcope1]

+1

[JWDobken]

+1

[ghost]

+1

[mynameiskyleok]

+1

[reywood]

I've tried repeatedly to reproduce this problem, but I can't get it to happen. If anyone can create a GitHub repo that reproduces this problem, I would be very grateful.

[KoenLav]

We have reproduced this problem with a normal Meteor.publish as well; when publishing the relations of one of our Collections we return an array of cursors in a normal Meteor.publish and when subscribing to this publication subscription.ready() will fire before all data from the relations is actually there.

[KoenLav]

Also, to add: for us this problem only appears when a previously rendered component also subscribes on a subset of one (or more) of the relations.

For instance: when displaying a paginated list of a collection we subscribed on the 'listWithRelations' composite publication which provided the first x (20) results of the collection together with all the related fields (but only those items which are actually necessary to render the current list, not the entire collection).

Afterwards when we go to the create page we subscribe on the 'relations' publication (normal Meteor publication) which is supposed to load data for all the relations, but (I assume) because we were previously subscribed to only a subset of the same collection we are now subscribing to the subscription returns ready too early (because we were previously subscribed, but to a subset).

We can confirm that the create component ONLY and EXACTLY renders exactly those items which were in the 'subset subscription' and that the problem is resolved when the page is refreshed.

So it appears that Meteor does not properly destroy composite subscriptions on the client side when a component is unmounted and still believes the subscriptions to be available.

I hope this sheds some light for some other people.

The solution we have implemented for now is no longer subscribing on the subset of the relations on our list page, but simply subscribing on entire 'relations' publication right away. While not ideal, it is acceptable for our use-case.

[fabyeah]

+1