IOCaml doesn't work with latest Jupyter?

[dsblank]

I finally got IOCaml started with Jupyter, but then it fails with a variety of issues, as if it doesn't really work with the latest Jupyter. Is this correct?

[andrewray]

A patch was provided by @signalpillar a couple of months ago that got it working with the latest Jupyter at that time.

Not sure how things have changed since then though.

[dsblank]

Ok, thanks for the info. Do you know where I can find this patch?

[andrewray]

The patch was in #36, and is included in this repo and was released with opam.

[dsblank]

Oh, ok. Then things aren't good. The first error I had was that the notebook didn't pass the test for the signed signature. I commented out that check, but it was in a bad state afterwards.

[Phyronnaz]

Try jupyter notebook --Session.key=''
Works for me on Jupyter 4.0.6

[signalpillar]

I also created this docker image https://hub.docker.com/r/signalpillar/ocaml-playground/
It contains the fixes.

[DhashS]

@signalpillar That Dockerfile no longer builds, this is the closest I got to one that does, but it still hits a zmq error.
https://gist.github.com/Aeolus0/9621119c62f48373bc2d

[andrewray]

I had a bit of a hack around with docker/jupyter and this Dockerfile works

https://gist.github.com/andrewray/a6e44c0ab497e8737dc2

[c-cube]

In a project of mine, I took some code from IOCaml to get a jupyter interface (for something that is not OCaml). I replaced the local bindings to zmq by libraries available on opam, used lwt, and nocrypto+hex to implement the support for signatures. If @andrewray is interested, it could be a good idea to move what I have into a separate repo/opam package ("jupyter-kernel", say) and use it in both our projects.

[andrewray]

Hey this is cool. I like the signature stuff very much.

Regarding zmq I deliberatly moved from the standard library in order to support v3 and v4 simultaneously, but if things have changed then perhaps its now the better solution.

Sharing code would be great, but the iocaml-server component (which to be fair I have not updated for a long while now) should be considered. It does not work with the newer jupyter specs.

[Naereen]

For Jupyter on Python3, the hack jupyter notebook --Session.key='' does not work any more, as '' is seen as an empty string and not an empty bytes, one should use jupyter notebook --Session.key='b""' instead (empty bytes b"").

[c-cube]

Indeed, I faced the same minor issue, and it's cleaner to implement hmac anyway.
@andrewray there are incompatible versions of zmq? or is v3/v4 referring to jupyter spec? I'm mostly interested in recent versions since that my distribution moves fast.

About iocaml-server, isn't it specific to having OCaml as a language? At least I think the generic kernel component ought to live in its own repository. I haven't updated everything yet (need to check mime data in particular). Lwt helps a lot though.

[c-cube]

Can't resist posting it: https://pbs.twimg.com/media/CzMC1PhW8AAHne9.jpg:large ^^
@andrewray the code is now isolated relatively cleanly in https://github.com/c-cube/stimsym/tree/master/src/ipython if you want to take a look.

[c-cube]

update: as of d7e4620e65d890f90c878449e06eb75033dafa3b , stimsym now supports version 5 of the protocol (mostly), including completion and the signature handling. It seems that jupyter-notebook can backward translate from protocol v5 to v4 if needed, so it should cover many use cases.

I propose that we create a jupyter-kernel library that would be standalone, relying on external libraries (lwt-zmq, lwt, nocrypto, hex, uuidm, etc.) for interacting with jupyter frontends. It should be useful both for iocaml and for other projects (including mine). If you (@andrewray) want, I can create it myself.

[andrewray]

My apologies, I have been extremely unhelpful here.

there are incompatible versions of zmq? or is v3/v4 referring to jupyter spec?

I was referring to the zmq protocol version. In the early days ocaml-zmq was broken quite a lot, and I was also fighting different distros with either v3 or v4 installed, so I knocked up a simple zmq interface which has worked well for me so far.

I can well believe the ocaml-zmq package is in better shape now, and zmq a bit more stable on distros, so moving to it shouldn't be a problem.

About iocaml-server, isn't it specific to having OCaml as a language?

Not really. It lacks the complete ipython server feature set (HMAC being one example), so was not really a good replacement for all other kernels out there, but for projects built in OCaml, and potentially using a library form of the iocaml-kernel, as you are doing, it should work.

Or at least it should work with the ancient protocol I implemented.

I propose that we create a jupyter-kernel library that would be standalone, relying on external libraries (lwt-zmq, lwt, nocrypto, hex, uuidm, etc.)

This is certainly looking like a good idea now.

If you (@andrewray) want, I can create it myself.

You've certainly put a good bit more effort into this than I have recently...so you are welcome to do it yourself if you prefer. I am willing to set this up myself though, and if you prefer this option, I will open a separate issue so we can discuss the best way to structure it (looking towards moving iocaml+kernel+js+server to ijupyter+kernel+lib+js+server+external projects).

[c-cube]

Ok, I think I will create a repo for the kernel library (relying on 3rd party libs and my own work, + what I copied and modified from yours). Then, on this repo, we can discuss things in more details. Do you want to create an organization as well, or create the repo under your own name, since most of the code (especially the json specs) is yours?

[andrewray]

Ok cool. Doing it under a new organisation where we could keep related projects (ie an ocaml kernel and, hopefully, a new server) seems a good idea.

[c-cube]

ok. How about ocaml-jupyter for the org name?

[andrewray]

sounds good.

[c-cube]

Here we go: https://github.com/ocaml-jupyter/jupyter-kernel . We can stop hijacking this issue!