pagination issue in verbose mode

[nbanb]

Dear Vi

Is there a way that verbose mode ('-vv') send log in a more readable maner ?

(terminator + xterm-256colors)

Or (gnome terminal + xterm-256colors)

I would expect something more readable like :

  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes                        
 l[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes
 o[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes
 g[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes
 i[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes                                     
 n[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes
 :[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes
  [INFO  websocat::ws_peer] Sending WebSocket ping
  [INFO  websocat::ws_peer] Received a pong from websocket; RTT = 763.972µs
  [INFO  websocat::trivial_peer] Special byte detected. Triggering EOF.
  [DEBUG websocat::my_copy] zero len
  [DEBUG websocat::my_copy] read_done
  [DEBUG websocat::my_copy] done
  [INFO  websocat::sessionserve] Forward finished
  [DEBUG websocat::sessionserve] Forward shutdown finished
  [DEBUG websocat::ws_peer] drop WsWriteWrapper
  [INFO  websocat::sessionserve] One of directions finished

$ websocat --version
websocat 1.13.0

Thanks for help
Kind regards
nabnba

[vi]

Maybe redirect the logs away from the terminal? The terminal seems to be in raw mode and not converting \ns to \r\ns.

Maybe you can do something like stty cooked to turn off raw mode.

The problem is not with the terminal emulator (such as gnome-terminal or terminator), but with tty Line discipline.

[nbanb]

Dear @vi

Thanks for answer.
Yes, terminal is in raw mode due to #152

So I could not use stty cooked as I must do stty raw -echo before invocating websocat to avoid character interpretation failiure during shell interractive session and program like vi in websocat session.

I was asking if something like exit_on_specific_byte was existing to handle this case when verbose debug mode is enable and terminal is in raw mode.

I want to say an option like :

--log-raw-ldisc

and when set, \n s and \r\n s of loglines sent by -vv would be interpreted as new line or return

Thanks again for help
Kind regards
nbanba

[vi]

Maybe just redirect logs away from the terminal, to a separate file/socket; then maybe view that file/socket in a separate terminal?

For example, use tmux or other terminal multiplexor to create two panes: one is set to raw mode and is interfaced by Websocat, the other is in cooked mode and shows Websocat's logs (and maybe that would be the pane Websocat starts from).

[nbanb]

Dear @vi
Thanks for answer, you're right I think there is something to do with file descriptors.
Go back here when I did find how to suites my need
Kind regards
nbanba

[nbanb]

Dear @vi

Thanks again for answer and help !

Here are multiple points to concider in this issue :
- terminal raw mode is mandatory due to #152
- For end users, I cannot add a new underlying required tools to use the library I maintain, ex: tmux (which would require to install an external program)
- As the library for which I open this issue is used on industrial terminals (on sort of automatas with only 1 serial bash terminal AND no way to install external tools) debug output MUST be in the current terminal and not in another terminal or another bash session.

Since I open this case and according to your suggestions , I made a lot of tests to try to redirect the debug outut to a new file descriptor (to be able to interpret correctly \r\n from this fd in terminal).
I did things like :

mkfifo /dev/shm/fbxfifo
exec 3>&-
exec 3<>/dev/shm/fbxfifo
CR=$(echo -en "\r")
while read <&3; do echo -e "\r${REPLY//$CR/}"; done &
tput init; stty raw -echo; SSL_CERT_FILE=/dev/shm/fbx-cacert-ws websocat -H "X-Fbx-App-Auth: $_SESSION_TOKEN" --origin https://fbx.fbx.lan --protocol "chat, superchat" -v -E --binary --ping-interval 10 --byte-to-exit-on 11 exit_on_specific_byte:stdio: exit_on_specific_byte:wss://fbx.fbx.lan/api/v12/vm/2/console 2>/dev/shm/fbxfifo; stty sane cooked; tput init
kill $!

but a cat /dev/shm/fbxfifo have a good output form (but from another terminal) and often missed some information, for example when ping keepalive is enable, some 'ping' are seen in websocat terminal but 'pong' reply is in the fifo. It's not always the case and I wasn't able to reproduce it exactly everytime during several continuous tests and I was not able to find all log messages in the fifo (none of the test have a 100% log messages in the fifo, there are always some (eparse) printed in websocat terminal, and for the moment I cannot explane this behavior).

Maybe I missed something but with file descriptors I wasn't able to have this debug output in the current terminal which is (and must be) in raw mode

[DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes                        
 l[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes
 o[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes
 g[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes
 i[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes                                     
 n[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes
 :[DEBUG websocat::ws_peer] incoming binary
  [DEBUG websocat::readdebt] Fulfilling the debt of 1 bytes
  [INFO  websocat::ws_peer] Sending WebSocket ping
  [INFO  websocat::ws_peer] Received a pong from websocket; RTT = 763.972µs
  [INFO  websocat::trivial_peer] Special byte detected. Triggering EOF.
  [DEBUG websocat::my_copy] zero len
  [DEBUG websocat::my_copy] read_done
  [DEBUG websocat::my_copy] done
  [INFO  websocat::sessionserve] Forward finished
  [DEBUG websocat::sessionserve] Forward shutdown finished
  [DEBUG websocat::ws_peer] drop WsWriteWrapper
  [INFO  websocat::sessionserve] One of directions finished
...

I think I will have a deeper look at stty, maybe I will find the way to stay in raw mode but only interpret \r\n.

Thanks again for help
Kind regards
nbanba

[vi]

Is Websocat logs only a part of debugging suite or those logs are expected to be visible to actual users of those "industrial terminals"?

Maybe it is worth to promote some specific log messages into e.g. script runs, so you can fine tune how exactly things are printed? Mixing content and logs is a poor idea in general and doubly poor when direct control of terminal is assumed.

Is there a list of relevant log messages you are actually looking for; or you are setting up flexible debuggability to be able to solve new issues later?

---

Maybe in the end it should be a dedicated connector tool that supports features you need the way you need? Websocat is fine for prototyping and debugging, but may be non-ideal fit for production.

[nbanb]

Dear @vi

Thanks again for answer and help.
Users of "industrials terminal" asked for a debug mode for troubleshooting where they could see all "transactions" while communicating with remote systems to be able to trace nearly everything (debug mode will not be used in production). As the connection and the terminal cannot be multiplexed, I need to mix logs on stderr and current stdin-stdout on the same terminal (know that's generaly not the best idea, but there are not many possibility).

I think stty was the good answer to this use case, as I finally achieve to get the required output :
In replacement of :

stty raw -echo ; websocat ... ; stty sane cooked

This expression of stty make the job :

stty raw inlcr -igncr opost -echo ; websocat ... ; stty sane cooked

Arrows in vi and autocomplete in bash is still working #152 and debug logs are now easily readable :

[INFO  websocat::stdio_threaded_peer] get_stdio_peer (threaded)
[INFO  websocat::ws_client_peer] get_ws_client_peer
[INFO  websocat::net_peer] Connected to TCP 10.0.100.10:443
[INFO  websocat::ws_client_peer] Connected to ws
[INFO  websocat::ws_peer] Sending WebSocket ping
[INFO  websocat::ws_peer] Received a pong from websocket; RTT = 742.603µs
[INFO  websocat::ws_peer] Sending WebSocket ping
[INFO  websocat::ws_peer] Received a pong from websocket; RTT = 723.372µs
...
[INFO  websocat::trivial_peer] Special byte detected. Triggering EOF.
[INFO  websocat::sessionserve] Forward finished
[INFO  websocat::sessionserve] One of directions finished

Last thing but it can be due to the websocket server itself (or maybe websocat), after connecting a remote qemu VM console over websocket (with terminal in raw mode) and using vi in the VM shell, I have to send a tput reset or better a tput init to get back the terminal to a normal behavior as after closing vi, I saw that \r\n is interpreted as \r only and a cat of a multiline file will be printed (and will scroll) only on the last line of the terminal.
(This issue was present before I open the present case)
A dirty fix consist to add to ~/.bashrc for each user connecting the qemu VM console over websocket the following

v () { /usr/bin/vim $*;tput init; } && alias vi="v $*"

not very clean but usefull

Thanks again for time and help
Kind regards
nbanba