broken kernel... pty issue?

[hkarl]

Hi,

I've seen the message and experienced the problem: java_kernel does not work. However, not sure this is related to an update of ipython.

I tried the following setup on both a Linux debian (+/- uptodate) and an OsX 10.11.6

(jupyter) ~> pip freeze | grep jupyter
jupyter==1.0.0
jupyter-client==4.3.0
jupyter-console==4.1.1
jupyter-core==4.1.0
jupyterhub==0.6.1
(jupyter) ~> pip freeze | grep ipython
ipython==4.2.1
ipython-genutils==0.1.0

kulla, java-9 +/- up-to-date.

It works on OSX, but jshell does not start up. I dug down to the pexcept pseudo-terminal startup code. In bpyhton, this call blocks:

ptyprocess.PtyProcess.spawn(["/home/jupyterhub/java/jdk-9/bin/java", "-jar", "/home/jupyterhub/java/kulla/kulla.jar"])

and when looking at the user's processes, there is only a python3 child process, but no java process. Looks like the exec call in the PtyProcess does not take place?

Do you think that is the right direction to look for problems? (I understand you don't have time currently to do that yourself.)

Thanks a lot,

Holger

[Bachmann1234]

That seems very likely. If you wanna take a stab at it I would appreciate it.

[Bachmann1234]

And yeah. Life has gotten particularly busy recently and I have not had time for side projects. Good things mostly :-)

[hkarl]

ok - but not getting any progress. I asked here:

http://stackoverflow.com/questions/40937420/drive-java-kulla-from-pexpect-context-jupyter

Let's see... thanks!

[nilsga]

Is this still an issue? I'm running within the docker image, and it seems to work just fine?

[dsblank]

Appears to be an issue with later versions of jdk-9. Downgrading to jdk-9.ea+100 works for me.

[hkarl]

Yes, still an issue for me; with jdk build 9-ea+147

Good to know there is one version known to work ...

In case you are looking: this is available here http://download.java.net/java/jdk9/archive/100/binaries/jre-9-ea+100_linux-x64_bin.tar.gz

(Oracle makes you edit download links by hand to protect you against yourself ... 👎 )

[nilsga]

Yeah, build 132 works, but not 151.

Error:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.5/site-packages/pexpect/spawnbase.py", line 150, in read_nonblocking
    s = os.read(self.child_fd, size)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.5/site-packages/pexpect/expect.py", line 99, in expect_loop
    incoming = spawn.read_nonblocking(spawn.maxread, timeout)
  File "/opt/conda/lib/python3.5/site-packages/pexpect/pty_spawn.py", line 465, in read_nonblocking
    return super(spawn, self).read_nonblocking(size)
  File "/opt/conda/lib/python3.5/site-packages/pexpect/spawnbase.py", line 155, in read_nonblocking
    raise EOF('End Of File (EOF). Exception style platform.')
pexpect.exceptions.EOF: End Of File (EOF). Exception style platform.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/conda/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jovyan/javakernel/__main__.py", line 9, in <module>
    IPKernelApp.launch_instance(kernel_class=JavaKernel)
  File "/opt/conda/lib/python3.5/site-packages/traitlets/config/application.py", line 652, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-120>", line 2, in initialize
  File "/opt/conda/lib/python3.5/site-packages/traitlets/config/application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/opt/conda/lib/python3.5/site-packages/ipykernel/kernelapp.py", line 454, in initialize
    self.init_kernel()
  File "/opt/conda/lib/python3.5/site-packages/ipykernel/kernelapp.py", line 365, in init_kernel
    user_ns=self.user_ns,
  File "/opt/conda/lib/python3.5/site-packages/traitlets/config/configurable.py", line 412, in instance
    inst = cls(*args, **kwargs)
  File "/home/jovyan/javakernel/kernel.py", line 34, in __init__
    self._start_java_repl()
  File "/home/jovyan/javakernel/kernel.py", line 52, in _start_java_repl
    continuation_prompt=u'   ...> '
  File "/opt/conda/lib/python3.5/site-packages/pexpect/replwrap.py", line 55, in __init__
    self._expect_prompt()
  File "/opt/conda/lib/python3.5/site-packages/pexpect/replwrap.py", line 66, in _expect_prompt
    timeout=timeout)
  File "/opt/conda/lib/python3.5/site-packages/pexpect/spawnbase.py", line 390, in expect_exact
    return exp.expect_loop(timeout)
  File "/opt/conda/lib/python3.5/site-packages/pexpect/expect.py", line 105, in expect_loop
    return self.eof(e)
  File "/opt/conda/lib/python3.5/site-packages/pexpect/expect.py", line 50, in eof
    raise EOF(msg)
pexpect.exceptions.EOF: End Of File (EOF). Exception style platform.
<pexpect.pty_spawn.spawn object at 0x7f781850beb8>
command: /home/jovyan/jdk-9/bin/java
args: [b'/home/jovyan/jdk-9/bin/java', b'-jar', b'/home/jovyan/kulla.jar']
buffer (last 100 chars): ''
before (last 100 chars): 'void main(String[] args)\r\nor a JavaFX application class must extend javafx.application.Application\r\n'
after: <class 'pexpect.exceptions.EOF'>
match: None
match_index: None
exitstatus: None
flag_eof: True
pid: 21
child_fd: 34
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_string:
    0: "jshell> "
    1: "   ...> "
```

This might have something to do with it: `void main(String[] args) or a JavaFX application class must extend javafx.application.Application`

[nilsga]

This might have something to do with it: http://hg.openjdk.java.net/jdk9/jdk9/langtools/rev/e5a42ddaf633

Seems like the main method in JShellTool is gone.

[nilsga]

If the launcher script changes java -jar kulla.jar to java -cp kulla.jar jdk.internal.jshell.tool.JShellToolProvider, it might work. Giving it a try.

[nilsga]

It gets rid of the kernel crash, but executing statements never terminates, so there's still something else that is wrong...

[nilsga]

I'm pretty sure this has something to do with ansi escape sequences to do. After digging deeper, it hangs when the jshell process outputs the sequence \x1b[6n

[dsblank]

@nilsga I agree, exactly. I also found this same issue when running jshell inside an Emacs shell.

But reverting back to an earlier jdk-9 avoids this problem.

[nilsga]

It would be very interesting to find out exact which version broke it, to pinpoint the exact change that caused it. For me, it works with build 132 but not 151. Have not tested any builds in between.

[nilsga]

Also, I don't think the kulla.jar is actually required anymore, since the downloadable JDK eap contains the jshell binary.

[dsblank]

@nilsga I believe that you are correct: using jdk-9ea+100/bin/jshell should work (although I did not try yet). I did try using pexpect with jdk-9ea+150/bin/jshelland that did NOT work because of the same control character issue described above.