Any way to get the newest posts without a stream?

[naiveai]

If I want to implement pagination for the newest posts, I think directly accessing the /r/subreddit/new.json and providing convenience before() and after() method of some sort would be easier to work with than a stream. And getting the newest posts for 1 time only is also seemingly impossible without using stream.submissions(pauseAfter: 1).takeWhile((s) => s != null)).toList() or something similar to that, which results in a lot of loading time. Am I missing something in the API or is this the only way to get the newest submissions?

[donovanrost]

    if (instance != null) { //where instance is an instance of DRAW
      Stream stream = instance.subreddit(subreddit).hot();
      _fillSubredditContent(stream: stream);
    }
  }

  void _fillSubredditContent({@required Stream stream}) {
    _apiStream = stream.listen((s) {
      if (!subredditContent.hasValue) {
        subredditContent.add([s]);
      } else {
        List temp = subredditContent.value;
        temp.add(s);
        subredditContent.add(temp);
      }
    });
  }

  pauseStream() => _apiStream.pause();
  resumeStream() => _apiStream.resume();```

This seems to be working pretty great for me. It's not elegant code, but it does the trick

[naiveai]

ALso, how do you get posts that are past the initial 100 posts? The stream only gives you the new posts as they are posted, not the older ones, does it?

[donovanrost]

I don't believe it gives you new posts as they are posted. But perhaps I'm wrong and @bkonyi can correct me on this.

The way I believe this works is that you make your request and it returns a block of 100 posts ( I don't know If there is a way to adjust this size -- I would like to be able to). Then when that block has returned, it sends out a request for another block of 100, and so on until the reddit api says it is out of posts.

DRAW will do this continuously -- it will take a just a few moments to retrieve a few thousand posts. But you can pause and resume the stream by attaching a listener.

[naiveai]

@donovanrost This behaviour might be useful in some cases, but it does not allow enough control for some others, like pagination as I mentioned. As of now, I just decided to use reddit.get('r/{subreddit}/new.json') directly to get the desired response.

[donovanrost]

You might just want to use subreddit('$subreddit').newest() As far as I know, pagination is just done by pausing and resuming the stream.

[bkonyi]

Sorry @naiveai, I've been AFK for a few days, let me take a look and get back to you. I think we do pagination while retrieving the content for the stream in the first place, but it might be worth giving more control as you've suggested.

[bkonyi]

@naiveai which stream are you using exactly and what behavior are you trying to get? Are you looking for a page of new submissions for a given subreddit?

Anyway, here's the implementation that's backing most of the streams in the API. As you can see, we do pagination there and request new data after yielding all the results from a page, so you should be able to avoid doing it yourself.

However, it does seem that there's some bugs in the stream builders where we might be handling the pauseAfter and limit parameters incorrectly, which is why you're seeing the stream pause after 100 items (see the implementation here for context). I'll file an issue and take a look before end of the weekend.

[naiveai]

@bkonyi I'm using the reddit.subreddit(subredditName).stream.submissions. The problem is essentially that you don't have enough control for my particular usecase - you get the first 100 results automatically in the stream, thus wasting network time if the user doesn't actually end up needing it. And there's no way I can see to get posts that are older than past 100. By requesting them manually, I'm able to paginate them using an infinite scroll view or something similar.

[StefanLobbenmeier]

I am a bit late to the party, but I wanted to share my approach as well. This is an excerpt from my code

  Stream<UserContent> getStream(BaseListingMixin listing, {String after}) {
    var params = Map<String, String>();
    params['limit'] = '10';
    params['after'] = after;
    switch (widget.subredditOrderState.getSort()) {
      case Sort.hot:
        return listing.hot(params: params);
      case Sort.top:
        return listing.top(params: params);
      case Sort.newest:
        return listing.newest(params: params);
      default:
        throw DRAWInternalError(
            'Sort ${widget.subredditOrderState.getSort()} is not supported');
    }
  }

I am not that familiar with the reddit API, but from my understanding there is also a "before" parameter, so passing the before parameter in a similar fashion could work for you.

[StefanLobbenmeier]

Excerpt from the Reddit API Doc:

overview

listings

Many endpoints on reddit use the same protocol for controlling pagination and filtering. These endpoints are called Listings and share five common parameters: after / before, limit, count, and show.

Listings do not use page numbers because their content changes so frequently. Instead, they allow you to view slices of the underlying data. Listing JSON responses contain after and before fields which are equivalent to the "next" and "prev" buttons on the site and in combination with count can be used to page through the listing.

The common parameters are as follows:

• after / before - only one should be specified. these indicate the fullname of an item in the listing to use as the anchor point of the slice.
• limit - the maximum number of items to return in this slice of the listing.
• count - the number of items already seen in this listing. on the html site, the builder uses this to determine when to give values for before and after in the response.
• show - optional parameter; if all is passed, filters such as "hide links that I have voted on" will be disabled.

To page through a listing, start by fetching the first page without specifying values for after and count. The response will contain an after value which you can pass in the next request. It is a good idea, but not required, to send an updated value for count which should be the number of items already fetched.

[bkonyi]

@naiveai ah, I'm guessing you're trying to display the newest posts for a subreddit, not listen for new submissions as they come in? In that case, you should be using the Subreddit.newest stream.

Once again, sorry for the delayed response... it's been a crazy month. I'll be able to respond much more frequently from now on.

[bkonyi]

@naiveai I've added a parameter which should make it easier to reduce the number of elements requested by the streams throughout the API here: https://github.com/draw-dev/DRAW/commit/201eab1159b771c4e625d618cd262bf181ddfdca. It should be trivial to add a parameter for after to allow for creating a stream that iterates from a given object ID to the newer objects. If someone has a need for iterating over objects from newest to oldest, that will take a bit more work.