Slack Integration - Some public channels don't pull through as options

[MangoMarcus]

Description We're setting up a Slack integration and the channel we want to use isn't available as an option.

The channel is a new, public channel and is not archived.

As an aside the channels also seem to be ordered from oldest to newest, it might more sense for them to be alphabetical.

Steps to reproduce

1. Add a Slack integration to a form (our workspace has about 300 public unarchived channels)
2. Set the "Channel type" as "Public Channel"
3. Use the refresh button to fetch the channels
4. The list doesn't include all public channels

Form settings

• Multi-page form: No
• Submission Method: Ajax
• Client-side Validation: Yes
• Custom Form Templates: No

Additional info

• Plugin version: 1.5.16
• Craft version: Craft Pro 3.7.4
• Multi-site: No

Additional context It looks like the underlying Slack API call in verbb\formie\integrations\miscellaneous\Slack::fetchFormSettings() has a limit of 50 set which may be the problem.

However the list is actually only pulling through 20 results for us. We suspect that it's our 50 oldest channels, however 30 of those have been archived but are included in the limit for some reason.

[engram-design]

I'll admit I don't have 20+ channels to test with on my end, but does changing that to 100 (or anything up to 1000 which is their upper limit) make a difference?

Strange, that we're supplying exclude_archived which should exclude archived channels. Combined with the limit setting not working, I wonder if something else is going on. But testing changing the limit certainly works for me (changing it to 1)

Happy to look at sorting them alphabetically, but can only do this after channels are fetched.

[MangoMarcus]

@engram-design We've tested increasing the limit which does now pull through the channel that we want. It does seem though that the limit is including archived channels, even though they aren't being included in the API response.

Our workspace now returns 330 channels however setting the limit to 330 only returns 185 - I suppose because the limit is still including archived ones.

I'd submit a PR to increase the limit, however there's also an error when the results are cached in the database. I suspect it's because the maximum SQL query length is being exceeded, the cutoff for us seems to be about 419 results (which I reached by increasing the limit for the members request as well as the channels request).

It's fine for us because the 330 channels we're getting is less than that cutoff of about 419, but it might cause issues for other users if they had a lower proportion of archived channels than we do.

Hope that makes sense, we're going to use a fork with the increased limit for now because it works for us.

Thanks

[engram-design]

Thanks for the investigation, and good call on the database column limit. There's only so much it can cache, and that indeed is a lot of data, some of it unnecessary. Interesting about the limit to the call, that must be on Slack's API which is kind of confusing.

I'll have a further think about how to address this for everyone's sake!

[MangoMarcus]

No problem. Looks like the underlying problem is with Slack and it has been raised before: https://github.com/slackapi/python-slack-sdk/issues/1043#issuecomment-862813806. The exclude_archived seems to be applied after the limit, strangely.

I believe the SQL error I mentioned was actually due to the query length rather than the column size. I suppose you could get round it by splitting it into separate queries and concatenating the field, though the column size could be an issue too I'm not sure.

Thanks!

[engram-design]

Added pagination support for the next release, which should alleviate this issue. To get this early, change your verbb/formie requirement in composer.json to:

"require": {
  "verbb/formie": "dev-craft-4 as 2.0.9",
  "...": "..."
}

Then run composer update.

[engram-design]

Added in 2.0.10