NanmiCoder
[问题] 抖音创作者爬取相关问题
当爬取的创作者主页视频过多时,会报错,请问是被平台风控了么,爬取的创作者id : "MS4wLjABAAAAli__17ML_EVQuB-WpOFYRfHq8tz1nsYTSHR3QXprkXJIgLG93YwHbrWYqaQZmYEv" 报错内容截图:
拉取代码更新了仍然一样, 能确认不是账号被风控了,我用我朋友的抖音账号爬的,IP代理开和关我都试了一样,response的响应为空,应该是没有请求到数据,我看了一下每个请求都有uifid,fp,verifyFp这三个参数,于是从我打开我的浏览器找到每个请求的查询参数(成功请求了的),拷贝下来这三个参数加到通用参数里,报错还是一样的,四个链接的查询参数如下: 创作者信息请求键值 device_platform:webapp aid:6383 channel:channel_pc_web publish_video_strategy_type:2 source:channel_pc_web sec_user_id:MS4wLjABAAAA9ywvgzJlK7FzC3VYFxE-Wvp477YcuKawIUV2soRYo41V0c4xz5uDmVQvFt0I2xyn personal_center_strategy:1 profile_other_record_enable:1 land_to:1 update_version_code:170400 pc_client_type:1 pc_libra_divert:Windows support_h265:1 support_dash:1 version_code:170400 version_name:17.4.0 cookie_enabled:true screen_width:2560 screen_height:1080 browser_language:zh-CN browser_platform:Win32 browser_name:Chrome browser_version:136.0.0.0 browser_online:true engine_name:Blink engine_version:136.0.0.0 os_name:Windows os_version:10 cpu_core_num:16 device_memory:8 platform:PC downlink:10 effective_type:4g round_trip_time:100 webid:7469015755414259234 uifid:fd93a02f3f52f94d15e514b5060909b7718dbdccf5e652d020e98a2368c5c67f2e9049f95e1ec5edb1f0af7f9564287923977119e4ccefe905755ef98e601ea12e9c9418512e9dde63931d66536673c41849ce02f383283008177ccde027ba6acc4652d1c1ac9901a3b96fcf9c9e4f7efbdba2d9834798018c4252c402883d43a1f899d2a71f56432a227f494276ef2a938b1d24a722a3080cf1537b8850e402 msToken:mnPWEeofjpD86TtSlM8aKFe4HJvfnPglZ43EnWIvi0wOrOm3yLZJbi7V74M5yNWy_20cpv_eba2LlTt6Z0VDPnKxevi9a7gWJBr7xtVog3T3xTMKcY5G0dbwQoSHCKcTYZXMlM_vKUTV0thsLArvgqPs3T7LP48-FyayhIcd9lQ4C6yjJJiN1A== a_bogus:OJ4nkwSiE2/nKdFt8ObT9XAlu7yANBWycBTObTCTtNPmTwtOSbPRoPbNjxL/sggKIYpTh93HSEz/bDnc/zXiZe9pLmkfSkwWyT/n9z0ogHHXaPvgXr6LSRGioivSUA4YmAIWEZvRWGMNZxcWVr9TABQHq/4iBbEZBq37VZYnE9K4UAWjho/Aa-7kiX7qKf== verifyFp:verify_m9tv78iq_u7BHDcnO_9T0g_4vdd_BolI_8AHyenCeuJUO fp:verify_m9tv78iq_u7BHDcnO_9T0g_4vdd_BolI_8AHyenCeuJUO
创作者帖子请求参数 device_platform:webapp aid:6383 channel:channel_pc_web sec_user_id:MS4wLjABAAAA9ywvgzJlK7FzC3VYFxE-Wvp477YcuKawIUV2soRYo41V0c4xz5uDmVQvFt0I2xyn max_cursor:0 locate_item_id:7496487512597826876 locate_query:false show_live_replay_strategy:1 need_time_list:1 time_list_query:0 whale_cut_token: cut_version:1 count:18 publish_video_strategy_type:2 from_user_page:1 update_version_code:170400 pc_client_type:1 pc_libra_divert:Windows support_h265:1 support_dash:1 version_code:290100 version_name:29.1.0 cookie_enabled:true screen_width:2560 screen_height:1080 browser_language:zh-CN browser_platform:Win32 browser_name:Chrome browser_version:136.0.0.0 browser_online:true engine_name:Blink engine_version:136.0.0.0 os_name:Windows os_version:10 cpu_core_num:16 device_memory:8 platform:PC downlink:10 effective_type:4g round_trip_time:100 webid:7469015755414259234 uifid:fd93a02f3f52f94d15e514b5060909b7718dbdccf5e652d020e98a2368c5c67f2e9049f95e1ec5edb1f0af7f9564287923977119e4ccefe905755ef98e601ea12e9c9418512e9dde63931d66536673c41849ce02f383283008177ccde027ba6acc4652d1c1ac9901a3b96fcf9c9e4f7efbdba2d9834798018c4252c402883d43a1f899d2a71f56432a227f494276ef2a938b1d24a722a3080cf1537b8850e402 msToken:mnPWEeofjpD86TtSlM8aKFe4HJvfnPglZ43EnWIvi0wOrOm3yLZJbi7V74M5yNWy_20cpv_eba2LlTt6Z0VDPnKxevi9a7gWJBr7xtVog3T3xTMKcY5G0dbwQoSHCKcTYZXMlM_vKUTV0thsLArvgqPs3T7LP48-FyayhIcd9lQ4C6yjJJiN1A== a_bogus:EJ45gwS7Yp8jKVFGYCbz9X5UNXy/NsuyBlT/bbKTCNKUTqMaSYN6oPSijxLMs26K9Ypwhe17HEzlYdncKTUwZH9kompvSk0ffG/I9L8LgHHhPskgJr6gS4tiLivzUA4Yu5IjEM6R6G0x1E5W9NCsApQHu/4iBbfZPH3UVZznO9umUSWjwn2Ia5JDYXwqyj== verifyFp:verify_m9tv78iq_u7BHDcnO_9T0g_4vdd_BolI_8AHyenCeuJUO fp:verify_m9tv78iq_u7BHDcnO_9T0g_4vdd_BolI_8AHyenCeuJUO
二级评论 device_platform:webapp aid:6383 channel:channel_pc_web item_id:7496487512597826876 comment_id:7496851468504089404 cut_version:1 cursor:0 count:3 item_type:0 update_version_code:170400 pc_client_type:1 pc_libra_divert:Windows support_h265:1 support_dash:1 version_code:170400 version_name:17.4.0 cookie_enabled:true screen_width:2560 screen_height:1080 browser_language:zh-CN browser_platform:Win32 browser_name:Chrome browser_version:136.0.0.0 browser_online:true engine_name:Blink engine_version:136.0.0.0 os_name:Windows os_version:10 cpu_core_num:16 device_memory:8 platform:PC downlink:10 effective_type:4g round_trip_time:200 webid:7469015755414259234 uifid:fd93a02f3f52f94d15e514b5060909b7718dbdccf5e652d020e98a2368c5c67f2e9049f95e1ec5edb1f0af7f9564287923977119e4ccefe905755ef98e601ea12e9c9418512e9dde63931d66536673c41849ce02f383283008177ccde027ba6acc4652d1c1ac9901a3b96fcf9c9e4f7efbdba2d9834798018c4252c402883d43a1f899d2a71f56432a227f494276ef2a938b1d24a722a3080cf1537b8850e402 verifyFp:verify_m9tv78iq_u7BHDcnO_9T0g_4vdd_BolI_8AHyenCeuJUO fp:verify_m9tv78iq_u7BHDcnO_9T0g_4vdd_BolI_8AHyenCeuJUO msToken:mnPWEeofjpD86TtSlM8aKFe4HJvfnPglZ43EnWIvi0wOrOm3yLZJbi7V74M5yNWy_20cpv_eba2LlTt6Z0VDPnKxevi9a7gWJBr7xtVog3T3xTMKcY5G0dbwQoSHCKcTYZXMlM_vKUTV0thsLArvgqPs3T7LP48-FyayhIcd9lQ4C6yjJJiN1A== a_bogus:EvsRhH6iEqWcadMGYKPcyVNUE6DMNBWySeToSJcT9NPkGqMaN8PbxabKnxw6O6xJnYBzhC57Rj4MTfdcOtXzZC9pzmpvuktW6TV99g8ogqwhGzwsDHbBCzYFoXMC8bTq-QV4ilgIZUG7Zjd-wqdD/d-nH/uK5QShQ1Oyk/ucP9B610yAgpnrPptkOXiT0IcJ
抖音评论 device_platform:webapp aid:6383 channel:channel_pc_web aweme_id:7496487512597826876 cursor:0 count:10 item_type:0 insert_ids: whale_cut_token: cut_version:1 rcFT: update_version_code:170400 pc_client_type:1 pc_libra_divert:Windows support_h265:1 support_dash:1 version_code:170400 version_name:17.4.0 cookie_enabled:true screen_width:2560 screen_height:1080 browser_language:zh-CN browser_platform:Win32 browser_name:Chrome browser_version:136.0.0.0 browser_online:true engine_name:Blink engine_version:136.0.0.0 os_name:Windows os_version:10 cpu_core_num:16 device_memory:8 platform:PC downlink:10 effective_type:4g round_trip_time:200 webid:7469015755414259234 uifid:fd93a02f3f52f94d15e514b5060909b7718dbdccf5e652d020e98a2368c5c67f2e9049f95e1ec5edb1f0af7f9564287923977119e4ccefe905755ef98e601ea12e9c9418512e9dde63931d66536673c41849ce02f383283008177ccde027ba6acc4652d1c1ac9901a3b96fcf9c9e4f7efbdba2d9834798018c4252c402883d43a1f899d2a71f56432a227f494276ef2a938b1d24a722a3080cf1537b8850e402 verifyFp:verify_m9tv78iq_u7BHDcnO_9T0g_4vdd_BolI_8AHyenCeuJUO fp:verify_m9tv78iq_u7BHDcnO_9T0g_4vdd_BolI_8AHyenCeuJUO msToken:mnPWEeofjpD86TtSlM8aKFe4HJvfnPglZ43EnWIvi0wOrOm3yLZJbi7V74M5yNWy_20cpv_eba2LlTt6Z0VDPnKxevi9a7gWJBr7xtVog3T3xTMKcY5G0dbwQoSHCKcTYZXMlM_vKUTV0thsLArvgqPs3T7LP48-FyayhIcd9lQ4C6yjJJiN1A== a_bogus:O74nk7y7Dp/fPdKtucPnyAQlC2IMrTWyRPT/bY1P7NKcGqzPvbNfxPbucxqh8Lrr9SpkhHIHVj-MbDdbOGXwZCrkKmkkuoJbyz5C9gvLMqNXGFhsENjFCuuzqXMKWRsq-A5JiIh62UGLZfd-kNQ8/d-n7/KFQm8hQ1Obk/zcP9Bh1MyAgZnrPQSpxhTqKj==