Support for Enterprise slack instances?

[yarikoptic]

Is your feature request related to a problem? Please describe.

Tried to auth against our dandiarchive instance which is on mit's enterprise slack instance.

after telling slackdump to dump list of conversations and going through authentication in a browser just getting

❯ ./slackdump -w dandiarchive
Slackdump dev (commit: placeholder) built on: 2077
authenticated.
2024/02/29 23:41:53 > checking user cache...
2024/02/29 23:41:53 retrieving data...
2024/02/29 23:41:53 application error: callback error: enterprise_is_restricted

Previously successfully fetched a list for another slack workspace on a "regular" slack instance

[rusq]

Hey @yarikoptic, thanks for the suggestion! Right now it would prevent the export from working on the enterprise instances. I suspect, the workaround would be to "observe" the edge api and making a custom call to the edge APIs.

[xiaket]

It looks like we have the code here, but it is not being actively used in the codebase.

[rusq]

I even know who wrote it 😂 It's just a stray one from one of the experimental branches that I did not want to lose. Did not go far with it. It will need to be revised.

[rusq]

Known restrictions:

1. error getting conversation users: callback error: enterprise_is_restricted
2. error getting members for [my test channel]: callback error: enterprise_is_restricted

[rusq]

Experimented with this over the weekend - findings:

Slack does not allow to seek past Page 100, when page > 100 requested, Slack returns the first page (Page=1). Seems to be an internal limitation. The workaround would be to use the Query parameter, to be more specific about the channel names, but to get all channels, this would require iterating through all 65536 runes of unicode give or take the special characters.

I'll take a break from this, maybe some idea comes up.

[rusq]

Hey @yarikoptic , with the great help of a slackdump user in Telegram, we were able to get it going, you can check the "v3" branch.

git checkout v3

The only thing you need to do, is to use the -enteprise flag.

In the examples I will use the "record" format which is a new way to save the Workspace information implemented in v3. It is already quite stable, so the plan is for future versions to support it unchanged. I'll show how to convert to export further, please keep reading.

Exporting

To export full workspace (in the "chunk" format):

go run ./cmd/slackdump archive -enterprise

To export some conversations:

go run ./cmd/slackdump archive -enterprise CHANNEL_ID1 CHANNEL_ID2

This will create "slackdump_" directory. I will refer to it as "slackdump_record_dir" further in this message.

To get the chunk format spec:

go run ./cmd/slackdump help chunk

EXPERIMENTAL: viewing the archive

"View" allows to view the slackdump record archives, Slack and slackdump export ZIP files and directories, and Slackdump dump archives. It will detect the format automatically.

To view the archive, run:

go run ./cmd/slackdump view slackdump_record_dir

Then head to http://localhost:8080

Converting to Slack export format

If you wish for some reason to convert to slack export format, run this command:

go run ./cmd/slackdump convert -o my_export_file.zip slackdump_record_dir

Any feedback is welcomed.

[rusq]

Getting help

Help is still in progress, but most of the commands will provide the flags and their description.

To get the list of commands:

go run ./cmd/slackdump help

To get help for the command or read a help topic:

go run ./cmd/slackdump help <topic or command>

To get help on a subcommand:

go run ./cmd/slackdump help command subcommand

for example

go run ./cmd/slackdump help list users

[xiaket]

Thanks for your hard work on this @rusq ! Really appreciated it! I can confirm that it works on my side as well. I checked out v3 branch, ran make arm_darwin to generate a binary, then used ./arm_darwin workspace to login and ./arm_darwin dump -enterprise C0xxxxxx to dump all messages in a channel. I'd like to report that the record command is not found on my v3 checkout(ffd4532d)

[rusq]

Hey @xiaket, thank you for the feedback! Sorry about that, v3 is in heavy development, I have renamed "record" to "archive" to make the purpose more clear, I'll update my prev msg.

[hutchiko]

@rusq I have also verified this works as advertised off the v3-wiz branch against my work enterprise Slack account. However I did have to make one small change as the Slack URL my workspace uses is not compatible with the current test. Ours is of the format:

https:///company.enterprise.slack.com/...

To get past this I updated the regexp used in url_parse.go to verify the URL to the following:

var slackURLRe = regexp.MustCompile(`^https:\/\/[a-zA-Z0-9]{1}[-\w]+(\.enterprise)?\.slack\.com\/archives\/[A-Z]{1}[A-Z0-9]+(\/p(\d+))?$`)

Thanks for all you work on this.

[rusq]

Hey @hutchiko thanks for letting me know, I'll update the code and tests, when able! Glad that you were able to make it work.

[mahiki]

So is -export broken for enterprise slack workspaces?

I am running export on specific channels, and the messages + attachments are downloaded partially for the first channel in the list, but the process exits with:

$> ./slackdump -no-user-cache -download -export-type standard -export "$EXPORT_DIR_EXP" DXXXXXXXXX
...
2024/07/19 11:01:04 application error: export error: error getting members for DXXXXXXXXX: callback error: enterprise_is_restricted

NOTE: its working just fine on my enterprise workspace getting attachments and json of channel:

./slackdump -no-user-cache -download -base $EXPORT_DIR_PLAIN DXXXXXXXXX

[rusq]

@mahiki make sure you running off the master branch (v3), experimental enterprise support is implemented on v3, v2 does not support enterprise instances.

[wshanks]

I see this:

$ go run ./cmd/slackdump archive -enterprise
archive: 2024/07/24 10:05:22 Error 004 (Authentication Error): auth error: workspace does not exist: ""
Error 004 (Authentication Error): auth error: workspace does not exist: ""
exit status 4

but it's not prompting me to log in. Am I missing something?

[Winniepoom]

Hello,

Can someone confirm if being on the enterprise plan (not the free plan) allows us to use Slack dump to export the entire history data from Slack? Also, does the -enterprise option work on the master branch?

Thank you.

[rusq]

Hey @Winniepoom , -enterprise works on the master branch. master branch contains v3 code in pre-alpha stage.

[rusq]

@wshanks this is fixed now.

You can also use workspace new command, i.e.

go run ./cmd/slackdump workspace new <your_workspace_name>

You can authenticate in more than one workspaces with this command.

List all existing workspaces with: workspace list

To learn about all workspace subcommands, run:

go run ./cmd/slackdump help workspace