gnip4j icon indicating copy to clipboard operation
gnip4j copied to clipboard

Replay Stream Support

Open jmmk opened this issue 7 years ago • 2 comments

Would it make sense to have a ReplayUriStrategy?

Looking at PowerTrack 2.0 Replay, it would mostly be a copy-paste of PowerTrackV2UriStrategy, but with a few changes:

Instead of

public final class PowerTrackV2UriStrategy implements UriStrategy {
    public static final String PATH_GNIP_STREAM_URI =  "/stream/powertrack/accounts/%s/publishers/%s/%s.json";
    public static final String PATH_GNIP_RULES_URI =  "/rules/powertrack/accounts/%s/publishers/%s/%s.json";

    @Override
    public URI createStreamUri(final String account, final String streamName) {
        ...

        return URI.create(String.format(streamUrlBase + PATH_GNIP_STREAM_URI, account.trim(), publisher.trim(), streamName.trim()));
    }

    private String createRulesBaseUrl(final String account, final String streamName) {
       ...

         return String.format(ruleUrlBase + PATH_GNIP_RULES_URI, account.trim(), publisher.trim(), streamName.trim());
    }
}

it would have something like

public class PowerTrackV2ReplayUriStrategy implements UriStrategy {
    public static final String PATH_GNIP_REPLAY_STREAM_URI
      = "/replay/powertrack/accounts/%s/publishers/%s/%s.json?fromDate=%s&toDate=%s";
    public static final String PATH_GNIP_REPLAY_RULES_URI
      = "/rules/powertrack-replay/accounts/%s/publishers/%s/%s.json";

    DateFormat formatter = new SimpleDateFormat("yyyyMMddHHmm");
    Date from;
    Date to;

    public PowerTrackV2ReplayUriStrategy(final Date from, final Date to) {
        this.from = from;
        this.to = to;
    }

    @Override
    public URI createStreamUri(String account, String streamName) {
        ...

        final String fromString = formatter.format(from);
        final String toString = formatter.format(to);

        return URI.create(String.format(
            this.streamUrlBase + PATH_GNIP_REPLAY_STREAM_URI, account.trim(),
            this.publisher.trim(), streamName.trim(), fromString, toString));
    }

    private String createRulesBaseUrl(final String account, final String streamName) {
        ...

        return String.format(
            this.ruleUrlBase + PATH_GNIP_REPLAY_RULES_URI, account.trim(), this.publisher.trim(),
            streamName.trim());
  }
}

jmmk avatar Sep 28 '16 19:09 jmmk

@jcodagnone You can see the full thing at https://github.com/Traackr/gnip4j/blob/powertrack-v2/core/src/main/java/com/zaubersoftware/gnip4j/api/impl/PowerTrackV2ReplayUriStrategy.java

The empty constructor is for adding/removing Rules, and the one accepting from and to dates for use with a Replay Stream.

A few questions/options:

  • Is this a good strategy, or is there a better way that you see to accomplish this?
  • Would you accept a PR for this?

jmmk avatar Sep 30 '16 18:09 jmmk

One other requirement for Replay Streams - when a replay stream finishes it sends a message that looks like (See the docs for more information):

{
  "info": {
    "message": "Replay Request Completed",
    "sent": "2016-05-27T22:15:50+00:00",
    "activity_count": 8874
  }
}

when successful and

{
  "info": {
    "message": "Replay Request Completed with Errors",
    "sent": "2016-05-27T16:00:02+00:00",
    "activity_count": 56333,
    "minutes_failed": [
      "2013-02-20T00:05:00+00:00",
      "2013-02-20T00:06:00+00:00"
    ]
  }
}

when there was an error

We've used a very naive solution of closing the stream once there is no more content for replay streams, but this could be smarter if it actually deserialized the payload, realized it wasn't an Activity, and took the correct action.

jmmk avatar Oct 14 '16 17:10 jmmk