haxelib submit freezes on "Processing file...."

[jgranick]

Recently, I have not been able to submit haxelibs (or if I am, I'm lucky for it to work before a very long time out)

This appears to be related to including a "haxedoc.xml", as this occurs for OpenFL and Lime, but not for smaller libraries that don't include a generated doc XML file

[back2dos]

Could you provide a sample archive to reproduce the problem?

[jgranick]

Try submitting a recent version of Lime or OpenFL, or if I understand the issue, an archive with one of their "haxedoc.xml" files

[ibilon]

Having a haxedoc.xml will trigger this https://github.com/HaxeFoundation/haxelib/blob/d3593aeb315cef8be6aef1a53e4fc5e932181e1f/src/haxelib/server/Repo.hx#L235-L262

but I don't think there's anything heavy going on, unless the xml is heavy? @jgranick

Also the Data function returns the first haxedoc.xml, regardless of how deep it is, maybe we would only look at it if it's next to haxelib.json?

[ibilon]

Alright so I tested it with the zip from openfl 3.0.1, had to remove all but haxelib.json and a third of haxedoc.xml (nekotools server as a very low post size limit) and the server took 6 minutes at 100% cpu.

I looked deeper at the code I linked, and it actually sounds quite heavy if you have a lot of types. Processing, sorting, copying and serializing a 2Mb xml file isn't quick.

Do we really need to parse the documentation? I can't even find where to access it from the website.

[nadako]

If anything, this should be done in a separate process, not in critical upload handler.

[ibilon]

I checked for .documentation in the entire repository: in the current website: nothing in the current server/client: only two mention in server when processing an upload the legacy server had a page to show the documentation.

So either we should show it, wonder why it didn't make it into the new website (and offload the parsing after the submit) or stop parsing it.