puppet-whatsapp icon indicating copy to clipboard operation
puppet-whatsapp copied to clipboard

Published 20 hours ago • verified publisher icon wechaty

关于 whatsapp 图片下载的逻辑

Open su-chang opened this issue 3 years ago • 3 comments

案例

手机端发送一张图片，从日志来分析会经历以下几个阶段：

  • 创建阶段

    手机端发送图片开始时（可能还未完全上传成功），会收到一条包含 base64 形式缩率图的 message_create 事件。此时 hasMedia 字段的值为 false。

  • 服务端确认阶段

    手机端发送图片成功后，会收到一条包含 base64 形式缩率图的 message_ack 事件。此时 hasMedia 字段的值为 false。

  • 服务端图片上传阶段

    服务端上传图片结束后，会收到 media_uploaded 事件

    • 此时若图片上传成功，则 hasMedia 字段为 true，且 body 字段为空值。
    • 此时若图片上传失败，则 hasMedia 字段为 false，且 body 仍为该图片的缩率图（base64 形式）。

  • 设备端确认阶段

    图片上传成功后，会收到 message_ack 事件，且 hasMedia 字段为 true，且 body 字段为空值。

关于 Message ACK

/** Message ACK */
export enum MessageAck {
  ACK_ERROR = -1,
  ACK_PENDING = 0,
  ACK_SERVER = 1,
  ACK_DEVICE = 2,
  ACK_READ = 3,
  ACK_PLAYED = 4,
}

日志

10: 41: 06 INFO PuppetWhatsApp onMessageCreate({
  "id": {
    "fromMe": true,
    "remote": "[email protected]",
    "id": "3A71F8CA50E0135FAB4E",
    "_serialized": "[email protected]_3A71F8CA50E0135FAB4E"
  },
  "ack": 0,
  "hasMedia": false,
  "body": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDABsSFBcUERsXFhceHBsgKEIrKCUlKFE6PTBCYFVlZF9VXVtqeJmBanGQc1tdhbWGkJ6jq62rZ4C8ybqmx5moq6T/2wBDARweHigjKE4rK06kbl1upKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKT/wgARCABIACkDASIAAhEBAxEB/8QAGQABAAMBAQAAAAAAAAAAAAAAAgABAwQF/8QAFwEBAQEBAAAAAAAAAAAAAAAAAAECA//aAAwDAQACEAMQAAAA9GVXfiOXrzlGmYl7BY1mCs5bFE7Qsw5rMowy9+YzseYMrJsgkAZC7kP/xAAWEQADAAAAAAAAAAAAAAAAAAAQETD/2gAIAQIBAT8ACr//xAAWEQADAAAAAAAAAAAAAAAAAAAQETD/2gAIAQMBAT8ADr//xAAiEAADAAIBAwUBAAAAAAAAAAAAAQIDERIEIVEFEBUiMVP/2gAIAQEAAT8ANmXFGRfZD6e8VcsFuWT6n1OHtmjmvKPmsX869mxsbKe/04x4QxsbKoqjkNlMdFV3HRsbGymUymchvRTKZTGzZTKZTGxmyspWQdjsV7NlWOx2cxWcz//+AAMA/9k=",
  "type": "image",
  "timestamp": 1644547266,
  "from": "[email protected]",
  "to": "[email protected]",
  "deviceType": "ios",
  "isForwarded": false,
  "forwardingScore": 0,
  "isStatus": false,
  "isStarred": false,
  "fromMe": true,
  "hasQuotedMsg": false,
  "vCards": [],
  "mentionedIds": [],
  "isGif": false,
  "isEphemeral": false,
  "links": []
})
10: 41: 11 INFO PuppetWhatsApp onMessageAck({
  "id": {
    "fromMe": true,
    "remote": "[email protected]",
    "id": "3A71F8CA50E0135FAB4E",
    "_serialized": "[email protected]_3A71F8CA50E0135FAB4E"
  },
  "ack": 1,
  "hasMedia": false,
  "body": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDABsSFBcUERsXFhceHBsgKEIrKCUlKFE6PTBCYFVlZF9VXVtqeJmBanGQc1tdhbWGkJ6jq62rZ4C8ybqmx5moq6T/2wBDARweHigjKE4rK06kbl1upKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKT/wgARCABIACkDASIAAhEBAxEB/8QAGQABAAMBAQAAAAAAAAAAAAAAAgABAwQF/8QAFwEBAQEBAAAAAAAAAAAAAAAAAAECA//aAAwDAQACEAMQAAAA9GVXfiOXrzlGmYl7BY1mCs5bFE7Qsw5rMowy9+YzseYMrJsgkAZC7kP/xAAWEQADAAAAAAAAAAAAAAAAAAAQETD/2gAIAQIBAT8ACr//xAAWEQADAAAAAAAAAAAAAAAAAAAQETD/2gAIAQMBAT8ADr//xAAiEAADAAIBAwUBAAAAAAAAAAAAAQIDERIEIVEFEBUiMVP/2gAIAQEAAT8ANmXFGRfZD6e8VcsFuWT6n1OHtmjmvKPmsX869mxsbKe/04x4QxsbKoqjkNlMdFV3HRsbGymUymchvRTKZTGzZTKZTGxmyspWQdjsV7NlWOx2cxWcz//+AAMA/9k=",
  "type": "image",
  "timestamp": 1644547266,
  "from": "[email protected]",
  "to": "[email protected]",
  "deviceType": "ios",
  "isForwarded": false,
  "forwardingScore": 0,
  "isStatus": false,
  "isStarred": false,
  "fromMe": true,
  "hasQuotedMsg": false,
  "vCards": [],
  "mentionedIds": [],
  "isGif": false,
  "isEphemeral": false,
  "links": []
})
10: 41: 11 INFO PuppetWhatsApp onMediaUploaded({
  "id": {
    "fromMe": true,
    "remote": "[email protected]",
    "id": "3A71F8CA50E0135FAB4E",
    "_serialized": "[email protected]_3A71F8CA50E0135FAB4E"
  },
  "ack": 1,
  "hasMedia": false,
  "body": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDABsSFBcUERsXFhceHBsgKEIrKCUlKFE6PTBCYFVlZF9VXVtqeJmBanGQc1tdhbWGkJ6jq62rZ4C8ybqmx5moq6T/2wBDARweHigjKE4rK06kbl1upKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKT/wgARCABIACkDASIAAhEBAxEB/8QAGQABAAMBAQAAAAAAAAAAAAAAAgABAwQF/8QAFwEBAQEBAAAAAAAAAAAAAAAAAAECA//aAAwDAQACEAMQAAAA9GVXfiOXrzlGmYl7BY1mCs5bFE7Qsw5rMowy9+YzseYMrJsgkAZC7kP/xAAWEQADAAAAAAAAAAAAAAAAAAAQETD/2gAIAQIBAT8ACr//xAAWEQADAAAAAAAAAAAAAAAAAAAQETD/2gAIAQMBAT8ADr//xAAiEAADAAIBAwUBAAAAAAAAAAAAAQIDERIEIVEFEBUiMVP/2gAIAQEAAT8ANmXFGRfZD6e8VcsFuWT6n1OHtmjmvKPmsX869mxsbKe/04x4QxsbKoqjkNlMdFV3HRsbGymUymchvRTKZTGzZTKZTGxmyspWQdjsV7NlWOx2cxWcz//+AAMA/9k=",
  "type": "image",
  "timestamp": 1644547266,
  "from": "[email protected]",
  "to": "[email protected]",
  "deviceType": "ios",
  "isForwarded": false,
  "forwardingScore": 0,
  "isStatus": false,
  "isStarred": false,
  "fromMe": true,
  "hasQuotedMsg": false,
  "vCards": [],
  "mentionedIds": [],
  "isGif": false,
  "isEphemeral": false,
  "links": []
})
10: 41: 12 INFO PuppetWhatsApp onMessageAck({
  "mediaKey": "W8CqF3WwSNdrO4gQvRTMWBPaQnB63lhH38/GHUwRZtU=",
  "id": {
    "fromMe": true,
    "remote": "[email protected]",
    "id": "3A71F8CA50E0135FAB4E",
    "_serialized": "[email protected]_3A71F8CA50E0135FAB4E"
  },
  "ack": 2,
  "hasMedia": true,
  "body": "",
  "type": "image",
  "timestamp": 1644547266,
  "from": "[email protected]",
  "to": "[email protected]",
  "deviceType": "ios",
  "isForwarded": false,
  "forwardingScore": 0,
  "isStatus": false,
  "isStarred": false,
  "fromMe": true,
  "hasQuotedMsg": false,
  "vCards": [],
  "mentionedIds": [],
  "isGif": false,
  "isEphemeral": false,
  "links": []
})

方案

  1. 使用 messageImage 方法时 imageType 字段可以恢复使用。

    • 当 PUPPET.ImageType.Thumbnail 时，直接使用 body 字段中的 base64 作为图片
    • 当 PUPPET.ImageType.Artwork 时，需要调用 downloadMedia 方法

  2. 需要控制调用 downloadMedia 方法的时机，必须要在接收到 media_uploaded 事件后，且 hasMedia 字段为 true 时方可执行。

待确认

  1. downloadMedia 方法的耗时
  2. 除了图片消息之外的媒体消息的接收/上传策略

问题

  1. base64的缩略图特别模糊 image

su-chang avatar Feb 11 '22 03:02 su-chang

图片下载建议采用lazy load的方式，如果没有接收到请求，就不下载图片。 另外，通过本地缓存保存下来每个富媒体消息图片上传情况，如果请求下载的时候没有上传完，则做一个循环等待，并设置一个等待最大时间

windmemory avatar Feb 11 '22 03:02 windmemory

bot的手机端发送的图片消息，接收到的日志如下。当 onMediaUploaded 事件中 hasMedia 字段为false的情况下，无法获取到图片信息，只能拿到缩率图。

16:20:32 SILL PuppetWhatsApp Manager onMessageCreate({
  "id":{
    "fromMe":true,
    "remote":"[email protected]",
    "id":"3A1E482D51D4B8BBC35F",
    "_serialized":"[email protected]_3A1E482D51D4B8BBC35F"
  },
  "ack":0,
  "hasMedia":false,
  "body":"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDABsSFBcUERsXFhceHBsgKEIrKCUlKFE6PTBCYFVlZF9VXVtqeJmBanGQc1tdhbWGkJ6jq62rZ4C8ybqmx5moq6T/2wBDARweHigjKE4rK06kbl1upKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKT/wgARCABIACkDASIAAhEBAxEB/8QAGgAAAgMBAQAAAAAAAAAAAAAAAAQCAwUBBv/EABYBAQEBAAAAAAAAAAAAAAAAAAABAv/aAAwDAQACEAMQAAAAW1MHcG84gd5KMqWzh6e5bVCKN8urxfOaSEtPSqlEMwsgZi17NPL6SUWxRIHAVHNCygA//8QAGhEAAwADAQAAAAAAAAAAAAAAARARAAIhMf/aAAgBAgEBPwCZIhw9zY0os+sv/8QAGBEAAgMAAAAAAAAAAAAAAAAAAAEQITD/2gAIAQMBAT8Al2hZ/wD/xAArEAACAQMCBQIGAwAAAAAAAAABAgADBBESIQUTIjFBUWEQFFJxgZEjJHL/2gAIAQEAAT8AoXCMwXBBMtV65cD+u0qnFVUHYCE4+FsxatTGPMJZKZKDJnzNww0mmftASa5DqQ2IylthOWZSRadwmN95UOkADzvKbfzZMqHN43+Zpz2micwc9SPBherVUFKeQPQzNZG1GkwEpOXumJBG3mYAWah6wHLAEZlpfG0pFNGRnMpsLi0WocLqlREW4wjatu8YZHU0wv1RFIcEiV9hFbTw+mvtKD8y4Kn0jUtu85Z9pdKekLtiVerSB5IjUj8uo9hKWhLg6iAcTnJjdh+5zaf1L+5TFS5rhaffEXhVw7rkAAHeGyRU3qMfzLy2o0GD5DBvUw1rYHq0z5i09pa39rSqMxGD2GBF4xb+Ax/E4lxE1nXlMyqBuJWqOydTk/cwmajP//4AAwD/2Q==",
  "type":"image",
  "timestamp":1644913232,
  "from":"[email protected]",
  "to":"[email protected]",
  "deviceType":"ios",
  "isForwarded":false,
  "forwardingScore":0,
  "isStatus":false,
  "isStarred":false,
  "fromMe":true,
  "hasQuotedMsg":false,
  "vCards":[],
  "mentionedIds":[],
  "isGif":false,
  "isEphemeral":false,
  "links":[]
})
16:20:57 SILL PuppetWhatsApp Manager onMessageAck({
  "id":{
    "fromMe":true,
    "remote":"[email protected]",
    "id":"3A1E482D51D4B8BBC35F",
    "_serialized":"[email protected]_3A1E482D51D4B8BBC35F"
  },
  "ack":1,
  "hasMedia":false,
  "body":"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDABsSFBcUERsXFhceHBsgKEIrKCUlKFE6PTBCYFVlZF9VXVtqeJmBanGQc1tdhbWGkJ6jq62rZ4C8ybqmx5moq6T/2wBDARweHigjKE4rK06kbl1upKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKT/wgARCABIACkDASIAAhEBAxEB/8QAGgAAAgMBAQAAAAAAAAAAAAAAAAQCAwUBBv/EABYBAQEBAAAAAAAAAAAAAAAAAAABAv/aAAwDAQACEAMQAAAAW1MHcG84gd5KMqWzh6e5bVCKN8urxfOaSEtPSqlEMwsgZi17NPL6SUWxRIHAVHNCygA//8QAGhEAAwADAQAAAAAAAAAAAAAAARARAAIhMf/aAAgBAgEBPwCZIhw9zY0os+sv/8QAGBEAAgMAAAAAAAAAAAAAAAAAAAEQITD/2gAIAQMBAT8Al2hZ/wD/xAArEAACAQMCBQIGAwAAAAAAAAABAgADBBESIQUTIjFBUWEQFFJxgZEjJHL/2gAIAQEAAT8AoXCMwXBBMtV65cD+u0qnFVUHYCE4+FsxatTGPMJZKZKDJnzNww0mmftASa5DqQ2IylthOWZSRadwmN95UOkADzvKbfzZMqHN43+Zpz2micwc9SPBherVUFKeQPQzNZG1GkwEpOXumJBG3mYAWah6wHLAEZlpfG0pFNGRnMpsLi0WocLqlREW4wjatu8YZHU0wv1RFIcEiV9hFbTw+mvtKD8y4Kn0jUtu85Z9pdKekLtiVerSB5IjUj8uo9hKWhLg6iAcTnJjdh+5zaf1L+5TFS5rhaffEXhVw7rkAAHeGyRU3qMfzLy2o0GD5DBvUw1rYHq0z5i09pa39rSqMxGD2GBF4xb+Ax/E4lxE1nXlMyqBuJWqOydTk/cwmajP//4AAwD/2Q==",
  "type":"image",
  "timestamp":1644913232,
  "from":"[email protected]",
  "to":"[email protected]",
  "deviceType":"ios",
  "isForwarded":false,
  "forwardingScore":0,
  "isStatus":false,
  "isStarred":false,
  "fromMe":true,
  "hasQuotedMsg":false,
  "vCards":[],
  "mentionedIds":[],
  "isGif":false,
  "isEphemeral":false,
  "links":[]
})
16:20:57 SILL PuppetWhatsApp Manager onMediaUploaded({
  "id":{
    "fromMe":true,
    "remote":"[email protected]",
    "id":"3A1E482D51D4B8BBC35F",
    "_serialized":"[email protected]_3A1E482D51D4B8BBC35F"
  },
  "ack":1,
  "hasMedia":false,
  "body":"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDABsSFBcUERsXFhceHBsgKEIrKCUlKFE6PTBCYFVlZF9VXVtqeJmBanGQc1tdhbWGkJ6jq62rZ4C8ybqmx5moq6T/2wBDARweHigjKE4rK06kbl1upKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKT/wgARCABIACkDASIAAhEBAxEB/8QAGgAAAgMBAQAAAAAAAAAAAAAAAAQCAwUBBv/EABYBAQEBAAAAAAAAAAAAAAAAAAABAv/aAAwDAQACEAMQAAAAW1MHcG84gd5KMqWzh6e5bVCKN8urxfOaSEtPSqlEMwsgZi17NPL6SUWxRIHAVHNCygA//8QAGhEAAwADAQAAAAAAAAAAAAAAARARAAIhMf/aAAgBAgEBPwCZIhw9zY0os+sv/8QAGBEAAgMAAAAAAAAAAAAAAAAAAAEQITD/2gAIAQMBAT8Al2hZ/wD/xAArEAACAQMCBQIGAwAAAAAAAAABAgADBBESIQUTIjFBUWEQFFJxgZEjJHL/2gAIAQEAAT8AoXCMwXBBMtV65cD+u0qnFVUHYCE4+FsxatTGPMJZKZKDJnzNww0mmftASa5DqQ2IylthOWZSRadwmN95UOkADzvKbfzZMqHN43+Zpz2micwc9SPBherVUFKeQPQzNZG1GkwEpOXumJBG3mYAWah6wHLAEZlpfG0pFNGRnMpsLi0WocLqlREW4wjatu8YZHU0wv1RFIcEiV9hFbTw+mvtKD8y4Kn0jUtu85Z9pdKekLtiVerSB5IjUj8uo9hKWhLg6iAcTnJjdh+5zaf1L+5TFS5rhaffEXhVw7rkAAHeGyRU3qMfzLy2o0GD5DBvUw1rYHq0z5i09pa39rSqMxGD2GBF4xb+Ax/E4lxE1nXlMyqBuJWqOydTk/cwmajP//4AAwD/2Q==",
  "type":"image",
  "timestamp":1644913232,
  "from":"[email protected]",
  "to":"[email protected]",
  "deviceType":"ios",
  "isForwarded":false,
  "forwardingScore":0,
  "isStatus":false,
  "isStarred":false,
  "fromMe":true,
  "hasQuotedMsg":false,
  "vCards":[],
  "mentionedIds":[],
  "isGif":false,
  "isEphemeral":false,
  "links":[]
})

su-chang avatar Feb 15 '22 08:02 su-chang

Related issue: https://github.com/pedroslopez/whatsapp-web.js/issues/1200

su-chang avatar Feb 15 '22 08:02 su-chang