PlatonAI issues

Results 30 issues of


                                            PlatonAI

Pulsar Driver should supports canConnect method

Add a new method canConnect or equivent to Pulsar Driver.

Handle error ERR_EMPTY_RESPONSE from browser.

A typical error message: ``` This page isn’t working wrd.walmart.com didn’t send any data. ERR_EMPTY_RESPONSE ``` It seems that the proxy ip has been blocked by the website.

Connection timed out when closing staging repository

Connection timed out when closing staging repository: ``` Uploaded to ossrh: https://oss.sonatype.org:443/service/local/staging/deployByRepositoryId/aiplatonpulsar-1066/ai/platon/pulsar/pulsar-protocol/1.10.16/pulsar-protocol-1.10.16.pom.asc (659 B at 805 B/s) [INFO] * Upload of locally staged artifacts finished. [INFO] * Closing staging repository...

Failed to load proxy.providers.txt by multiple threads in parallel. The file should be locked.

Failed to load proxy.providers.txt by multiple threads in parallel. The file should be locked. 17:02:17.125 [-worker-33] WARN a.p.p.c.proxy.ProxyLoader - Failed to load - /home/platonai/.pulsar/proxy/providers-enabled/proxy.providers.txt 17:02:17.125 [-worker-30] WARN a.p.p.c.proxy.ProxyLoader - Failed...

Too many warning logs after MongoDB crashes

Too many warning logs after MongoDB crashes: 10:56:03.617 [r-worker-5] WARN a.p.p.c.i.StreamingCrawler - [Unexpected] ai.platon.shaded.com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches ai.platon.shaded.com.mongodb.client.internal.MongoClientDeleg ate$1@7335a5ec. Client view...

Could not parse query '--x--': unexpected token at '--x--'

18:56:56.985 [main] WARN a.p.pulsar.dom.select.PowerSelector - Failed to parse css query | #productDescription, h2:contains(Product Description) --x-- div | https://www.amazon.com/dp/B07V2CLJLV | Could not parse query '--x--': unexpected token at '--x--'

Properly handle 404 Not Found pages

For example, we construct a page URL does not exist: https://www.amazon.com/dp/006323047_404. PulsarR has to properly handle such pages: 1. properly handle the status code 2. do not retry

The privacy context was not closed when the proxy IP expired.

Original report: The proxy is expired (xxx), context reset will be triggered https://github.com/platonai/exotic-amazon/issues/19 This bug has already be fixed in 1.10.x.

InaccessibleObjectException: Unable to make field private final long java.time.Duration.seconds accessible

bin/build-run.sh // OK 2023-02-16 22:12:09.736 INFO [main] a.p.p.a.m.PulsarMasterKt - Starting PulsarMasterKt v1.10.10-SNAPSHOT using Java 17.0.5 on regulus with PID 21576 (/home/vincent/workspace/pulsar-1.10.x/pulsar-app/pulsar-master/target/pulsar-master-1.10.10-SNAPSHOT.jar started by vincent in /home/vincent/workspace/pulsar-1.10.x) And then we issue...

Headless chrome has detectable user-agent

使用最新的Google Chrome时：使用正常标题的google-chrome浏览器时： Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36 使用google-chrome-headless浏览器时： Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/95.0.4638.69 Safari/537.36