Unexpected behavior using arrow keys on the terminal

[George Prekas]

Using Cygwin 1.7.32, mintty 1.1.3 and OpenSSH_6.7p1 I am getting unexpected behavior regarding the use of arrow keys on the terminal. You can reproduce the behavior by doing the following:

ssh linux
cd /usr/src/linux/tools/perf
make
cd ~
/usr/src/linux/tools/perf/perf record echo 42
/usr/src/linux/tools/perf/perf report

Pressing UP or DOWN should highlight one of the rows displayed. You can verify expected behavior by using either PuTTY or native Linux.

Observation #1: You can fix perf's behavior by applying perf.patch (attached).

Observation #2: Using Wireshark, I've observed that when I ssh to a host and press UP or DOWN on my terminal 3 packets are transmitted from the client. PuTTY on the other hand transmits only 1 packet (larger in size).

Observation #3: I wrote the program test.c (attached). If I run it and press UP or DOWN:
* on Windows from cmd.exe it says "Read 3 bytes. First is 27."
* on Linux it says "Read 3 bytes. First is 27."
* on Linux via PuTTY it says "Read 3 bytes. First is 27."
* on Windows from mintty.exe it says "Read 1 bytes. First is 27. Read 1 bytes. First is 91. Read 1 bytes. First is 65."

My understanding is that the unexpected behavior occurs because Cygwin sends the UP/DOWN sequence one byte at a time. Specifically:

* winsup\cygwin\fhandler_tty.cc @ fhandler_pty_master::write
This is the function called by the write system call invoked by mintty. Here len = 3. line_edit is invoked 3 times.
* winsup\cygwin\fhandler_termios.cc @ fhandler_termios::line_edit
This is called by the previous and it calls accept_input.
* winsup\cygwin\fhandler_tty.cc @ fhandler_pty_master::accept_input
This does the actual WriteFile to the pipe.

I would have provided a patch to fix the problem, but I am not sure I completely understand the semantics of the above mentioned methods.
Regards,
George Prekas

Dipl. Electrical and Computer Engineer, National Technical University of Athens

P.S. To be honest, I am not sure how much unexpected is this behavior. I just thought it would be good to have a bug report for it.

[Corinna Vinschen]

Hi George,

Post by George Prekas
Using Cygwin 1.7.32, mintty 1.1.3 and OpenSSH_6.7p1 I am getting
unexpected behavior regarding the use of arrow keys on the terminal.
ssh linux
cd /usr/src/linux/tools/perf
make
cd ~
/usr/src/linux/tools/perf/perf record echo 42
/usr/src/linux/tools/perf/perf report
Pressing UP or DOWN should highlight one of the rows displayed. You
can verify expected behavior by using either PuTTY or native Linux.
Observation #1: You can fix perf's behavior by applying perf.patch (attached).
Observation #2: Using Wireshark, I've observed that when I ssh to a
host and press UP or DOWN on my terminal 3 packets are transmitted
from the client. PuTTY on the other hand transmits only 1 packet
(larger in size).
* on Windows from cmd.exe it says "Read 3 bytes. First is 27."
* on Linux it says "Read 3 bytes. First is 27."
* on Linux via PuTTY it says "Read 3 bytes. First is 27."
* on Windows from mintty.exe it says "Read 1 bytes. First is 27. Read
1 bytes. First is 91. Read 1 bytes. First is 65."
My understanding is that the unexpected behavior occurs because Cygwin
This is the function called by the write system call invoked by
mintty. Here len = 3. line_edit is invoked 3 times.
This is called by the previous and it calls accept_input.
This does the actual WriteFile to the pipe.
I would have provided a patch to fix the problem, but I am not sure I
completely understand the semantics of the above mentioned methods.

I have to admit that I'm not quite sure either. In theory I'd say that
the perf tool is making some invalid assumption here. You can't rely on
a set of bytes sent from any input source to be always sent or received
as a single package, unless you're working with a transport guaranteing
this.

Your analyzes of the underlying mechanism in Cygwin is correct, though.
Despite what I said above, I take a look into this and perhaps I can fix
the code to send more than 1 byte at a time.


Thanks,
Corinna

[Corinna Vinschen]

Post by Corinna Vinschen
Hi George,

Post by George Prekas
Using Cygwin 1.7.32, mintty 1.1.3 and OpenSSH_6.7p1 I am getting
unexpected behavior regarding the use of arrow keys on the terminal.
ssh linux
cd /usr/src/linux/tools/perf
make
cd ~
/usr/src/linux/tools/perf/perf record echo 42
/usr/src/linux/tools/perf/perf report
Pressing UP or DOWN should highlight one of the rows displayed. You
can verify expected behavior by using either PuTTY or native Linux.
Observation #1: You can fix perf's behavior by applying perf.patch (attached).
Observation #2: Using Wireshark, I've observed that when I ssh to a
host and press UP or DOWN on my terminal 3 packets are transmitted
from the client. PuTTY on the other hand transmits only 1 packet
(larger in size).
Observation #3: I wrote the program test.c (attached). If I run it and
* on Windows from cmd.exe it says "Read 3 bytes. First is 27."
* on Linux it says "Read 3 bytes. First is 27."
* on Linux via PuTTY it says "Read 3 bytes. First is 27."
* on Windows from mintty.exe it says "Read 1 bytes. First is 27. Read
1 bytes. First is 91. Read 1 bytes. First is 65."
My understanding is that the unexpected behavior occurs because Cygwin
This is the function called by the write system call invoked by
mintty. Here len = 3. line_edit is invoked 3 times.
This is called by the previous and it calls accept_input.
This does the actual WriteFile to the pipe.
I would have provided a patch to fix the problem, but I am not sure I
completely understand the semantics of the above mentioned methods.

I have to admit that I'm not quite sure either. In theory I'd say that
the perf tool is making some invalid assumption here. You can't rely on
a set of bytes sent from any input source to be always sent or received
as a single package, unless you're working with a transport guaranteing
this.
Your analyzes of the underlying mechanism in Cygwin is correct, though.
Despite what I said above, I take a look into this and perhaps I can fix
the code to send more than 1 byte at a time.

In fact, both sides of the pipe were writing/reading in single byte
chunks.

The slave read side did something weird: Before reading, it always
checked if the number of bytes available in the pipe is > VMIN. If so,
it only read VMIN bytes. Since VMIN is 1 by default, it always reads
only single bytes, since changing VMIN is only done rather seldomly by
applications.
This code snippet is actually 12 years old. Much of the code
surrounding it changed considerably, just this snippet persisted. I
disabled it now since it really looks like a remnant from the past, and
it certainly doesn't reflect how it works on other OSes.

On the master write side I applied a possible, but definitely
experimental fix, which solves this issue by writing in 32 byte chunks
in non-canonical mode. This should (mostly) circumvent the above effect,
but it doesn't quite guarantee it.

With luck, this change also speeds up tty output by reducing the # of
calls to WriteFile/ReadFile.

I uploaded a new snapshot to https://cygwin.com/snapshots, the one
from today. Please give it a try.


Corinna

[George Prekas]

Hi Corinna,

Your snapshot works! I verified that perf utility works. I also verified
through Wireshark that only one packet is sent when I press UP or DOWN
(the old behavior was that 3 packets were sent).

Thank you very much for your effort and prompt reply!

George

Post by Corinna Vinschen
Hi George,

Post by George Prekas
Using Cygwin 1.7.32, mintty 1.1.3 and OpenSSH_6.7p1 I am getting
unexpected behavior regarding the use of arrow keys on the terminal.
ssh linux
cd /usr/src/linux/tools/perf
make
cd ~
/usr/src/linux/tools/perf/perf record echo 42
/usr/src/linux/tools/perf/perf report
Pressing UP or DOWN should highlight one of the rows displayed. You
can verify expected behavior by using either PuTTY or native Linux.
Observation #1: You can fix perf's behavior by applying perf.patch (attached).
Observation #2: Using Wireshark, I've observed that when I ssh to a
host and press UP or DOWN on my terminal 3 packets are transmitted
from the client. PuTTY on the other hand transmits only 1 packet
(larger in size).
Observation #3: I wrote the program test.c (attached). If I run it and
* on Windows from cmd.exe it says "Read 3 bytes. First is 27."
* on Linux it says "Read 3 bytes. First is 27."
* on Linux via PuTTY it says "Read 3 bytes. First is 27."
* on Windows from mintty.exe it says "Read 1 bytes. First is 27. Read
1 bytes. First is 91. Read 1 bytes. First is 65."
My understanding is that the unexpected behavior occurs because Cygwin
This is the function called by the write system call invoked by
mintty. Here len = 3. line_edit is invoked 3 times.
This is called by the previous and it calls accept_input.
This does the actual WriteFile to the pipe.
I would have provided a patch to fix the problem, but I am not sure I
completely understand the semantics of the above mentioned methods.

I have to admit that I'm not quite sure either. In theory I'd say that
the perf tool is making some invalid assumption here. You can't rely on
a set of bytes sent from any input source to be always sent or received
as a single package, unless you're working with a transport guaranteing
this.
Your analyzes of the underlying mechanism in Cygwin is correct, though.
Despite what I said above, I take a look into this and perhaps I can fix
the code to send more than 1 byte at a time.

In fact, both sides of the pipe were writing/reading in single byte
chunks.

The slave read side did something weird: Before reading, it always
checked if the number of bytes available in the pipe is > VMIN. If so,
it only read VMIN bytes. Since VMIN is 1 by default, it always reads
only single bytes, since changing VMIN is only done rather seldomly by
applications.
This code snippet is actually 12 years old. Much of the code
surrounding it changed considerably, just this snippet persisted. I
disabled it now since it really looks like a remnant from the past, and
it certainly doesn't reflect how it works on other OSes.

On the master write side I applied a possible, but definitely
experimental fix, which solves this issue by writing in 32 byte chunks
in non-canonical mode. This should (mostly) circumvent the above effect,
but it doesn't quite guarantee it.

With luck, this change also speeds up tty output by reducing the # of
calls to WriteFile/ReadFile.

I uploaded a new snapshot to https://cygwin.com/snapshots, the one
from today. Please give it a try.


Corinna

[Corinna Vinschen]

Post by George Prekas
Hi Corinna,
Your snapshot works! I verified that perf utility works. I also verified
through Wireshark that only one packet is sent when I press UP or DOWN
(the old behavior was that 3 packets were sent).
Thank you very much for your effort and prompt reply!

Thanks for testing!


Corinna