registry: submodule is not handled correctly

[kt3k]

deno_dom module has ./wpt submodule (which points to https://github.com/web-platform-tests/wpt/). When I click wpt folder in https://deno.land/x/[email protected], it shows empty page https://deno.land/x/[email protected]/wpt

[wperron]

We should add a submodule option to the git clone operation in the publishing process to support this

[Im-Beast]

I'm really looking forward to fix this issue, as I got very dissapointed just now figuring out that submodules are not handled correctly 😞

[wperron]

Well, submodules simply aren't handled at all. I've added the feat label so we can start looking into it soon. There's a couple of things to take into consideration, namely the size on disk as submodules can recurse pretty deep.

[Im-Beast]

The thing is repo itself can go very deeply also without using submodules, so they're not a big problem itself. Therefore, there should just be a size limit for a module, however for bigger than desired modules it would be cool if someone could've just "link" their hosting to the system. I don't really know how big performance gap is, so I can't really talk about that tho.

[wperron]

There's a logical limit of 20mb by default that we check but there's also an implicit limit because of Lambda's 512mb filesystem limit that we can't do anything about. I imagine that repos using submodules risk triggering that limit more often, but I guess we'll just have to monitor it and get some statistics.