Error "failed to dump channel C0XXXXXXX: json: cannot unmarshal number into Go struct field ..."

[yoshimotomasahir]

Slackdump v2.2.0-alpha hangs in the middle.

1. Run slackdump -export archive.zip -export-type standard -download
2. Several channel posts and files are successfully stored in a ZIP file (~500 MB)
3. The program hangs with the following error

...
2022/08/29 12:13:02 file "document.docx" saved to channel_name\attachments: 19000 bytes written
2022/08/29 12:13:02 application error: export error: channels: error: failed to dump "test_channel_name" (C0XXXXXXX): callback error: failed to dump channel C0XXXXXXX: json: cannot unmarshal number into Go struct field Message.messages.blocks of type string

The channel name has Kanji charactors, Katakana charactors, alphabets, numbers, and underscores. This channel has been archived and unarchived several times on Slack.

The ZIP files cannot be viewed with a Windows default ZIP viewer, but I was able to view the ZIP file using 7zip viewer.

• OS: Windows 10

I used slackdump for 30 other Slack workspaces due to the 90 days limits, and they all worked fine. Thanks for the awsome software.

[rusq]

Hey @yoshimotomasahir thanks for the feedback :) It looks like yet another edge case when API returns something that is not expected by slack library. I have prepared the tool to address these type of problems (this issue looks similar to #109 and #115)

The tool is attached to this comment: https://github.com/rusq/slackdump/issues/109#issuecomment-1229155995

would you be able to run this tool like this:

rawoutput.exe C0XXXXXXX 

replacing the C0XXXXXXX with the actual channel ID?

It will generate the slackdump_raw.log file which is a dump of headers and JSON output from the API.

Could you then search for substring:

"blocks": "

I.e. it would look like this: {"message": { ...., "blocks": "xxxxxx", ... } }

and post the enclosed message JSON object { ...., "blocks": "xxxxxx", ... } here? Make sure to replace any sensitive information within that message please.

Thank you )

[yoshimotomasahir]

@rusq Thank you very much for your quick reply.

I have output slackdump_raw.log from rawoutput.exe.

I've removed the sensitive information, but I think I should not to post it on public github, so I will email you the json text file.

If you can not see it, it's my problem, so don't worry about the issue.

[rusq]

@yoshimotomasahir Thanks for the file that you shared. I got some good news! I was able to reproduce the issue, will bee looking into fixing it.

[rusq]

The issue is with this block:

                                {
                                    "type": "date",
                                    "timestamp": 1636961629,
                                    "format": "{date_long} {time_secs}"
                                },

~Block type "date" is not currently supported by upstream library. These are the supported blocks: https://github.com/slack-go/slack/blob/master/block.go#L11~

Scrape that, rushing to conclusions.

[yoshimotomasahir]

Some workflows that can be used with paid subscriptions was set in the channel. It may be related. https://slack.com/intl/en-gb/help/articles/360053571454-Set-up-a-workflow-in-Slack

[rusq]

@yoshimotomasahir It's quite possible that the bot that sent that message with the invalid type of the timestamp is serialising the timestamp incorrectly as an integer. I have raised the issue with slack-go/slack (linked above), would be interesting what they'll say.

The issue happens because the type for the timestamp for the RichText Block Element type Date is defined as string (see https://github.com/slack-go/slack/blob/master/block_rich_text.go#L330) but the bot that sent the message, sends it as a number. When the library tries to load the JSON returned by the API, it fails, because it expects the string.

This is the header of the message:

            "type": "message",
            "subtype": "bot_message",
            "text": "XXXXXXXXX",
            "ts": "1636961664.000300",
            "username": "XXXXXXXXX",
            "bot_id": "B0XXXXXXXXX",

In the original file it would contain the username of the bot - is it one of the paid subscription bots, would you be able to tell?

[yoshimotomasahir]

The bot appears to be a bot made by users with using Workflow Builder. I could not access the details of that bot. It is possible that the users created an inappropriate Workflow.

Workspaces which have the same issues is a paid. So, the issue would not be urgent.

[rusq]

The fix has been created and merged upstream (https://github.com/slack-go/slack/pull/1131), and will make it through in v3.0.0 once it's released!

[dmi3-bu]

Still waiting for the new version, @rusq do you have any plans to release this in near future?

[rusq]

Hey @dmi3-bu, appreciate the wait. I do have a feeling that this upstream change has already made it to v2 branch, but I won't bet my life on it just yet 😂 Will check tonight and let you know.

[rusq]

@dmi3-bu @yoshimotomasahir I've merged the updated lib into master, will release 2.2.8 shortly!

[rusq]

Also @yoshimotomasahir thanks for your donation ❤️