Jan-Petter Gundersen

HTML title encoding issue

3

There's an encoding issue in the HTML provider. `$embed->getProviders()['html']->getTitle()` Returns: `Tilbehør for brytere =` Expected: `Tilbehør for brytere =

[Misc::fix_protocol](https://github.com/simplepie/simplepie/blob/master/library/SimplePie/Misc.php#L194) is returning an invalid encoded URL. The root issue is probably that [IRI::parse_iri](https://github.com/simplepie/simplepie/blob/master/library/SimplePie/IRI.php#L361) is trying to parse an protocol-less URL, witch is expecting perfectly valid URLs only. Example: (links...

Better phone browser support

2

Hi I love Beyondpad, but I really miss better browser support on phones (or even the app for Android). Is it possible to set default / fallback page width (or...

Parsing performance issue

6

When parsing large robots.txt files, the process can take several minutes to finish, even with 100% CPU power dedicated. This is a problem in general, and is not related to...

``` $t1gor = new t1gor\RobotsTxtParser\RobotsTxtParser(

Invalid encodings are not ignored

2

No errors/warnings should be generated when parsing, still I get these: ``` mb_internal_encoding(): Unknown encoding "OSF10020402" // valid, but not installed mb_internal_encoding(): Unknown encoding "UTF9" // invalid mb_internal_encoding(): Unknown encoding...

Byte limit

1

**Feature request:** Limit the maximum number of bytes to parse. > A maximum file size may be enforced per crawler. Content which is after the maximum file size may be...

Relative paths is always allowed

1

``` User-agent: * Disallow: / ``` ``` $this->assertTrue($parser->isDisallowed("&&1@|")); $this->assertFalse($parser->isAllowed('+£€@@1¤')); ``` The two tests above fails, paths allowed according to the parser, the robots.txt says otherwise... At least the second one...

Thanks for this great addon. One of my automations failed a few days ago, and I ended up using way more power at the cheapest hour than intended. My target...