emacs-ycmd in Emacs Ipython Notebook (EIN)?

[DGalt]

trafficstars

This may be more of a ycmd thing than emacs-ycmd, but I figured I'd ask here first.

Is there any way to get emacs-ycmd completions working within EIN (If not familiar with EIN, it basically allows you to edit jupyter notebooks within emacs. repository: https://github.com/millejoh/emacs-ipython-notebook/)

I have completions working just fine for standard python files. I'm guessing the issue is that jupyter (ipython) notebooks use a different file extension than typical python files (since, at least as I understand, ycmd identifies which completer to use based on the filetype - although I could very well be wrong about that).

[abingham]

(My "short response" has turned into a bit of a rambling thinking-out-loud...)

It might be possible, but I'm not sure how much work it would take. The first issue I can think of is that ipython notebooks are not pure Python code, so the completion mechanism would need to know how to tell the difference. When we communicate with ycmd, we ship off the full text of the file being completed, so perhaps emacs-ycmd could figure that out at the client end. If EIN has facilities for extracting code blocks, it might be very easy. As part of the request to ycmd we can also specify the file-type, i.e. python in this case, so these seems like good steps along the way. File-type and contents might be enough for ycmd to work with.

We also send the path to the file as part of the request, and I'm not entirely sure what ycmd does with that. If we send the full text of a code block plus a filename which actually contains a superset of that codeblock, I don't know what will happen. Presumably it will actually work since, in general, the contents we send across are from modified buffers which don't match the file contents, i.e. ycmd evidently works fine in this case.

So I guess what I'm saying is that it doesn't seem unreasonable! And it would be super-cool if it worked. Do you have any capacity to try implementing it? I don't know when I'll get a chance.

[abingham]

Also, does EIN work well for you? I've tried it a few times, and I always run into strange hangs and stuff. I'd love to be able to work with notebooks entirely within emacs, but the state of ein has always been a bit lacking for me.

[DGalt]

That sounds like a promising avenue to at least look in to. I'm still a bit of an eLisp novice unfortunately, but I'm not adverse to looking in to this more. I have to get up to speed on how emacs-ycmd and ycmd are working, though. The dev who has taken over EIN (@millejoh) may also be able to shed some light on this.

As for your other question, I haven't run into any major hiccups with EIN, although I'm primarily just using it for testing code snippets / exploratory data analysis (I primarily work with scientific python stuff), so I can't speak to a very wide array of use cases. Generally, though, it works quite well ("it just works", as the expression goes).

[abingham]

I thought about this a bit more, and one complication is how we'll report our position to ycmd. To get completions, we send to ycmd the code itself as well as our position in that code (line, column). Currently we derive that simply from the position in the emacs buffer, but of course that seems like it won't work so well for notebooks.

We'd need to first decide whether we send across all of the code in the notebook for each completion request, or just the current snippet. Since there are almost always dependencies between the snippets, and because the notebook authors often expect them to be treated as continuous (e.g. imports from one snippet are used in another), I think we'll need to send it all.

With that decided, we'd need to figure out how to determine our position in the conglomerate, independent of our position in the emacs buffer. I don't have any brilliant ideas on that right now, but perhaps some sort of narrowing scheme could work.

If you feel like taking a swing at this, I'll be more than happy to help. I'm no expert in elisp, but I learned what I have by jumping into the deep end (this project being part of that deep end!) If you want to see how emacs talks to ycmd, you can look at ycmd--request. This is where we initiate the HTTP message to the ycmd server (all communication is JSON in HTTP). Also look at ycmd--standard-content which is where we gather up the components which are part of almost every message we send to the server: file path, line number, etc. See ycmd-get-completions for a high-level function that puts all of the parts together...this is probably your best guide to getting started. You can see how this function is used if you look at company-ycmd--get-candidates.

If I find the time and inspiration I might take a swing at this as well, but I can't promise. I'm working on a book and that's taking up pretty much all of my spare time :-/

[DGalt]

I agree, I think that the most logical thing is to send the entire set of code blocks. As you said, there are almost always dependencies between the different code blocks, and jupyter notebooks (at least, in the most basic sense) treats the whole workspace as one big namespace even though you can separate the snippets of code out into different executable segments.

Part of the question about how to approach it may be dependent on how EIN treats those blocks of code. I'm a bit bogged down with work stuff for the next few days, but I'll talk with the EIN dev and start reading through the docs you recommend this week. I'm sure I'll be back with questions soon ;)