Show Dash user contrib docsets in Zeal

[jboadas]

Just trying to see the user contrib docsets but can't see it on the list

[sirajulm]

Which docset creates the problem? Have you checked with some other user contributed docsets? Are the config files proper as specified here: Generating Dash Docsets

[josephfinlayson]

You do it this way:

https://twitter.com/J4lf/status/553489605958594560

[wosc]

Thanks @josephfinlayson for pointing out that "feed" can also be a Tarball!

However, most Tarballs are removed from the User Contributions repository and uploaded to the @Kapeli CDN, which is not publicly accessible AFAICT, so what you need to do is trawl through the Pull Requests and look for the Tarballs there.

This is a bit of annoying manual work, but I don't know what a more convenient solution could look like, since I don't think it would be fair to ask @Kapeli to offer these docsets for us non-paying Zeal users via bandwidth/resources that @Kapeli is paying for.

[Kapeli]

Sorry for not seeing this issue earlier!

Actually I'd be glad if Zeal would use these docsets as well. I already allow access to the other (main) docsets, so I don't see a reason why not to do the same with the user contributed ones.

You can get a json of all the available user contributed docsets at http://sanfrancisco.kapeli.com/feeds/zzz/user_contributed/build/index.json

Once you have that, you can get to the docset archives like this:

http://<mirror>.kapeli.com/feeds/zzz/user_contributed/build/<key name inside json>/<archive name>

Examples:

http://newyork.kapeli.com/feeds/zzz/user_contributed/build/Atom/Atom.tgz
http://london.kapeli.com/feeds/zzz/user_contributed/build/SDL/org.libsdl.sdl20.docset.tgz

The icons in the json are base64 encoded.

Regarding the <mirror> part, you're supposed to choose the closest/best mirror. I've given details about how to do that to @trollixx.

[trollixx]

Thank you, @Kapeli!

I am working on a revamped docset manager for Zeal, which will include user contributed docsets.

[lowjoel]

Would RDoc/Yard documentation support (like Dash) belong to this ticket also?

[dkniffin]

:+1: to rdoc support in zeal.

[NoAnyLove]

Since I was looking for docset for matplotlib, it's very happy to find this and finally soveld my problem. Thank you @Kapeli !

[lowjoel]

@Kapeli The instructions on the Dash site only show how to get docsets for published gems Yardoc/Rdoc, but doesn't document how to generate them. Any way to document that in the meanwhile?

[Kapeli]

Nope, sorry. The tool Dash uses to generate docs from Ruby Gems is not really usable if it's not run by Dash. I'll open-source it and package it into something everyone can use if more users ask.

The tool is written in Objective-C, so it won't be usable outside of OS X even if I release it.

[lowjoel]

@Kapeli I'm looking forward to it, if it's released. Even if it's in Obj-C I think people would write ports.

In the absence of that, does your docset license allow Zeal to use the Gem documentation from your site?

[Kapeli]

In the absence of that, does your docset license allow Zeal to use the Gem documentation from your site?

No idea what you mean. What gem documentation?

[lowjoel]

@Kapeli Dash allows users to download Yardoc/Rdoc documentation for gems, running through your generator on your end. Since your docsets are licensed out for Zeal, does that allow Zeal to request for Yardoc/Rdoc docsets through your service too?

(Maybe I'm missing something)

[Kapeli]

The Ruby Gem docsets are not generated on my servers. Dash (i.e. the client) downloads the Ruby Gem from rubygems.org, then generates the documentation using yard and then generates the docset. This all happens locally. My servers are not involved.

[lowjoel]

I see, so then for that we would need that tool to generate the proper docsets... Let's see what demand is like.

I was really looking forward to that since I'm doing Rails apps quite a lot nowadays...

[trollixx]

I've created #285 to track progress on Ruby documentation generation. This issue is about supporting user-contributed docsets from Dash.

[prusswan]

It would be "good to have"

[kidd]

Trying to implement the ability to get custom docsets, I find that by building the string as explained in this thread, I get things like "http://newyork.kapeli.com/feeds/zzz/user_contributed/build/Google App Engine - Python/GAE-Python.tgz" which have spaces, and they don't seem to exist. tried urlencoding and didn't work either... @Kapeli , any hints?

[trollixx]

It should be http://newyork.kapeli.com/feeds/zzz/user_contributed/build/Google_App_Engine-Python/GAE-Python.tgz.

[Kapeli]

You're supposed to add the "key" of the dictionary entry (first line in the sample below), not the value under name. That would be Google_App_Engine-Python instead of Google App Engine - Python.

    "Google_App_Engine-Python" : {     // <--- use this
      "author" : {
        "name" : "TZ Martin",
        "link" : "https:\/\/twitter.com\/tzmartin"
      },
      "aliases" : [
        "Google App Engine Python",
        "AppEngine",
        "Google Cloud"
      ],
      "archive" : "GAE-Python.tgz",
      "specific_versions" : [
        {
          "version" : "1.6.9.445",
          "archive" : "versions\/1.6.9.445\/GAE-Python.tgz"
        }
      ],
      "version" : "1.6.9.445",
      "name" : "Google App Engine - Python"
    },

[despairblue]

@Kapeli do the user contrib docs also have xml feeds?

At the moment if one wants to use one of the user contrib docs one has to download the archive and extract it manually into zeals docset folder, since zeal's add feed option only accepts xml feads not the tgz links directly.

[Kapeli]

@Kapeli do the user contrib docs also have xml feeds?

No, sorry.

[despairblue]

Ok, but still, thanks for making them available anyhow :smiley:

[kidd]

I've been dabbling with this issue, and created a little service (hosted in heroku) to retrieve a list of all the contrib docsets in a straight format (parsing xml and json is not the funniest thing in emacs lisp). It's still unfinished, and subject to changes, and in fact, I'd like to make it retrieve also the official ones.

https://dashes-to-dashes.herokuapp.com/docsets/contrib

@Kapeli, Is it ok to follow this path? or you prefer we don't re-distribute the list in any other form appart from the official one? (completely understandable due to adding an extra part in the game is not always the best solution and changes in the api will have to be synched by both).

[despairblue]

@kidd cool stuff, but I'm lacking some background here as I don't see the purpose. This is still json and not parsable by zeal. Or am I mistaken?

[kidd]

I was doing it for https://github.com/areina/helm-dash, and my idea was to end with only json, and directly parseable like [{"name": "foo", "archive": "http://lala.com/foo.tgz"}...] . My plans were to create this intermediate layer for both official and user-created docsets.

As I can't create the xml feeds myself... So it's an alternative, and if we all agree on one way of showing this info, I can adapt both helm-dash and dashes-to-dashes.

[trollixx]

Wow, that's the most commented issue so far. :)

I do have plans to add the support for the community contributed docsets in the nearest time. Importing the list to our database is a really simple task. We already have a working process for official docsets, so Zeal users see updates almost immediately as they get released for Dash.

The problem is our docset management UI. It's primitive and adding another hundred of docsets to it would make it hard to use. So, Zeal definitely needs a better docset manager, and that will take some time to implement.

I think I'll add importing of user docsets on the server-side some time this or next week, but no any estimates on that landing as a stable release yet.

[Kapeli]

@kidd It's ok with me. Feel free to redistribute and reformat the json in any way you want for helm-dash.

[kidd]

Hi. the PR finally landed, and in https://github.com/kidd/dashes-to-dashes there's the code for this reformatting of the docsets info. For now we're only using it for user-contribs ( https://dashes-to-dashes.herokuapp.com/docsets/contrib ) but probably we'll use it also for the official ones.

Just FYI

[crmne]

I created a little program in Python 3 and easygui (pip3 install easygui) to download and extract the user docsets in Zeal's docsets folder. Hope it helps!

https://gist.github.com/crmne/3fe84c05013fa87d74a8