go-reddit
go-reddit copied to clipboard
Add Subreddit Comment Retrieval and Stream Concurrency
This PR currently sits on top of comment-support as it builds on top of some of it.
Changelog:
- Move the
set{}
struct to its ownhelper
file and refactor it toOrderedMaxSet
- The goal was to give it a maximum size while still checking for duplicates in the latest
ids - Helper since it's more abstract functionality and less
go-reddit
specific
- The goal was to give it a maximum size while still checking for duplicates in the latest
- Refactor streams to a request-callback approach, so requests don't block each other
- The purpose of this is to make requests on pace with the Ticker set for the stream and not have requests block each other
- The
OrderedMaxSet
should handle de-duping request responses
Considerations:
- Perhaps make the streams configurable to either make requests sync or async
This PR should be ready for review / merge
panic happens sometimes with this pr (may exist in original code)
panic: send on closed channel
goroutine 93559 [running]:
github.com/vartanbeno/go-reddit/v2/reddit.(*StreamService).Posts.func2.1(0xc000081800, 0x64, 0x80, 0x0, 0x0)
go-reddit/reddit/stream.go:91 +0xc5
github.com/vartanbeno/go-reddit/v2/reddit.(*StreamService).getPosts(0xc000286288, 0x9bc961, 0x3, 0xc00089c000)
go-reddit/reddit/stream.go:193 +0x106
created by github.com/vartanbeno/go-reddit/v2/reddit.(*StreamService).Posts.func2
go-reddit/reddit/stream.go:60 +0x198
possible fix based on this PR: https://github.com/darren/go-reddit/commit/39267077688fc9cd37974d6420563191894a17b6
@darren Would you mind making a PR against this branch in my fork so I can easily add your code / give you credit for it?
@RGood my fix above is still buggy, panic occasionally:
panic: send on closed channel
goroutine 347827 [running]: github.com/vartanbeno/go-reddit/v2/reddit.(*StreamService).Comments.func2.1(0xc001434800, 0x64, 0x80, 0x0, 0x0) go-reddit/reddit/stream.go:196 +0x1a5 github.com/vartanbeno/go-reddit/v2/reddit.(*StreamService).getComments(0xc0000a0700, 0xe26c10, 0x3, 0xc000138180) go-reddit/reddit/stream.go:218 +0xe3 created by github.com/vartanbeno/go-reddit/v2/reddit.(*StreamService).Comments.func2 go-reddit/reddit/stream.go:161 +0x1a5
So using a quitChan is still tricky and not fixing the problem, I opened another PR https://github.com/RGood/go-reddit/pull/1 by using context