Uploaded image for project: 'Guacamole'
  1. Guacamole
  2. GUACAMOLE-1505

Self built guacd, Release 1.4.0 segfaults on RDP-Disconnect

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.4.0
    • 1.5.2
    • guacamole, guacd, RDP
    • None

    Description

      Setup

      Component Release
      guacamole server, guacd 1.4.0
      OS, guacamole server CentOS 7
      FreeRDP library version freerdp-libs-2.1.1-5.el7_9.x86_64
      RDP server Windows 10 Pro, version 21H1, OS build 19043.1415

      Context

      System was updated from Guacamole version 1.2.0, self built to version 1.4.0, self built likewise. Self built Guacamole version 1.2.0 ran for more than a year before the update to self built to version 1.4.0. We have two Guacamole instances on two different hosts, all with the same setup as specified in the table above.

      Build

      Component Release
      RPM package builder Mock, version 1.3.2-2
      build destination OS CentOS 7
      build from Git commit 
      commit be9041fefd9c1f7c647845aa0709caedcb54e812 (tag: refs/tags/1.4.0-RC1, tag: refs/tags/1.4.0)
Merge: 1f6f45e6 09bd4af7
Author: Virtually Nick <necouchman@users.noreply.github.com>
Date:   Sat Dec 25 10:18:39 2021 -0500

    GUACAMOLE-478: Merge add clipboard line ending normalization option for RDP.
      FreeRDP library freerdp-devel.x86_64 0:2.1.1-5.el7_9
      Mock RPM build logs mock_guacamole-server_build-1.4.0_logs.tar.xz

      Reproducibility

      Description Reproducibility
      with version 1.4.0, self built always
      with version 1.3.0, from Epel repository not reproducible
      with version 1.2.0, self built not reproducible, ran for more than a year on two hosts

      Steps to Reproduce

      Case 1: Disconnect on Timeout
      1. Click on Guacamole web frontend on a connection (Windows 10 system)
      2. desktop with authentication dialogue is shown
      3. do nothing, wait until automatic disconnect timeout kicks in on inactivity
      4. system is disconnected
      Case 2: Active User Disconnect
      1. Click on Guacamole web frontend on a connection (Windows 10 system)
      2. desktop with authentication dialogue is shown
      3. authenticate by given Windows active directory credentials
      4. RDP server let's you login
      5. work on desktop, clicking thins, starting applications, e.g. web browser Word, other editors, all works as usual
      6. goto Windows menu in left lower corner >> Power >> Disconnect
      7. system is disconnected

      guacd Logs

      In both use cases we see these log lines from the sessions ran under self built version 1.4.0:

      Jan 17 13:58:17 aguacate-srv-us1 kernel: guacd[10109]: segfault at 10 ip 00007f3d38114972 sp 00007f3d1b7fdba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 14:00:14 aguacate-srv-us1 kernel: guacd[10170]: segfault at 10 ip 00007f3d38114972 sp 00007f3d1b7fdba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 14:02:41 aguacate-srv-us1 kernel: guacd[10232]: segfault at 10 ip 00007f3d38114972 sp 00007f3d1a7fbba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 15:08:24 aguacate-srv-us1 kernel: guacd[10736]: segfault at 10 ip 00007f3d38114972 sp 00007f3d317f9ba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 17:29:36 aguacate-srv-us1 kernel: guacd[11724]: segfault at 10 ip 00007f3d38114972 sp 00007f3d00ff8ba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 17:30:37 aguacate-srv-us1 kernel: guacd[11751]: segfault at 10 ip 00007f3d38114972 sp 00007f3d00ff8ba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 17:31:13 aguacate-srv-us1 kernel: guacd[9981]: segfault at 10 ip 00007f3d38915972 sp 00007f3d2f8d7ba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d388f5000+35000]
Jan 17 17:32:03 aguacate-srv-us1 kernel: guacd[11766]: segfault at 10 ip 00007f3d38114972 sp 00007f3d00ff8ba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 17:32:57 aguacate-srv-us1 kernel: guacd[11802]: segfault at 10 ip 00007f3d38114972 sp 00007f3cfb7fdba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 17:32:57 aguacate-srv-us1 kernel: guacd[11823]: segfault at 10 ip 00007f3d38114972 sp 00007f3d33ffeba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 17:35:30 aguacate-srv-us1 kernel: guacd[11893]: segfault at 10 ip 00007f3d38114972 sp 00007f3cfa7fbba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 17:36:23 aguacate-srv-us1 kernel: guacd[11915]: segfault at 10 ip 00007f3d38114972 sp 00007f3cfa7fbba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 17:39:51 aguacate-srv-us1 kernel: guacd[11960]: segfault at 10 ip 00007f3d38114972 sp 00007f3cfa7fbba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 17:40:41 aguacate-srv-us1 kernel: guacd[11981]: segfault at 10 ip 00007f3d38114972 sp 00007f3cfa7fbba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 22:10:31 aguacate-srv-us1 kernel: guacd[10249]: segfault at 10 ip 00007f3d38114972 sp 00007f3d03ffeba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 17 22:13:47 aguacate-srv-us1 kernel: guacd[14007]: segfault at 10 ip 00007f3d38114972 sp 00007f3cf37fdba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 18 01:35:03 aguacate-srv-us1 kernel: guacd[15548]: segfault at 10 ip 00007f3d38114972 sp 00007f3cf37fdba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d380f4000+35000]
Jan 18 06:03:36 aguacate-srv-us1 kernel: guacd[16088]: segfault at 10 ip 00007f3d38915972 sp 00007f3cf37fdba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d388f5000+35000]
Jan 18 09:45:40 aguacate-srv-us1 kernel: guacd[19101]: segfault at 10 ip 00007f3d39116972 sp 00007f3d1affcba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d390f6000+35000]
Jan 18 09:55:23 aguacate-srv-us1 kernel: guacd[19159]: segfault at 10 ip 00007f3d39116972 sp 00007f3d1bffeba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d390f6000+35000]
Jan 18 10:03:37 aguacate-srv-us1 kernel: guacd[19290]: segfault at 10 ip 00007f3d39116972 sp 00007f3d1b7fdba0 error 4 in libguac-client-rdp.so.0.0.0[7f3d390f6000+35000]
Jan 18 10:10:22 aguacate-srv-us1 kernel: guacd[19390]: segfault at 10 ip 00007fc7ec9a5972 sp 00007fc7e39c9ba0 error 4 in libguac-client-rdp.so.0.0.0[7fc7ec985000+35000]
Jan 18 10:11:33 aguacate-srv-us1 kernel: guacd[19420]: segfault at 10 ip 00007fd027dd0972 sp 00007fd022e71ba0 error 4 in libguac-client-rdp.so.0.0.0[7fd027db0000+35000]

      Case 1: Logs, disconnect on Timeout

      With additional log lines from guacd daemon start :

      Jan 18 10:09:42 aguacate-srv-us1 guacd[19384]: Guacamole proxy daemon (guacd) version 1.4.0 started
Jan 18 10:09:42 aguacate-srv-us1 guacd[19384]: Communication will require SSL/TLS.
Jan 18 10:09:42 aguacate-srv-us1 guacd[19384]: Using PEM keyfile /etc/guacamole/ssl/aguacate-srv-us1.server.lan.key
Jan 18 10:09:42 aguacate-srv-us1 guacd[19384]: Using certificate file /etc/guacamole/ssl/aguacate-srv-us1.server.lan.pem
Jan 18 10:09:42 aguacate-srv-us1 guacd[19384]: Listening on host 172.19.209.153, port 4822
Jan 18 10:09:49 aguacate-srv-us1 guacd[19384]: Creating new client for protocol "rdp"
Jan 18 10:09:49 aguacate-srv-us1 guacd[19384]: Connection ID is "$d5a18cb3-2596-4a0c-9ab6-b03b019c196f"
Jan 18 10:09:49 aguacate-srv-us1 guacd[19386]: Security mode: Negotiate (ANY)
Jan 18 10:09:49 aguacate-srv-us1 guacd[19386]: Resize method: none
Jan 18 10:09:49 aguacate-srv-us1 guacd[19386]: No clipboard line-ending normalization specified. Defaulting to preserving the format of all line endings.
Jan 18 10:09:49 aguacate-srv-us1 guacd[19386]: User "@cdc6023e-109a-48a8-a443-ce92d41c72e3" joined connection "$d5a18cb3-2596-4a0c-9ab6-b03b019c196f" (1 users now present)
Jan 18 10:09:49 aguacate-srv-us1 guacd[19386]: Loading keymap "base"
Jan 18 10:09:49 aguacate-srv-us1 guacd[19386]: Loading keymap "en-us-qwerty"
Jan 18 10:09:51 aguacate-srv-us1 guacd[19386]: Connected to RDPDR 1.13 as client 0x0004
Jan 18 10:10:22 aguacate-srv-us1 guacd[19386]: RDP server closed/refused connection: Manually logged off.
Jan 18 10:10:22 aguacate-srv-us1 guacd[19386]: Internal RDP client disconnected
Jan 18 10:10:22 aguacate-srv-us1 kernel: guacd[19390]: segfault at 10 ip 00007fc7ec9a5972 sp 00007fc7e39c9ba0 error 4 in libguac-client-rdp.so.0.0.0[7fc7ec985000+35000]
Jan 18 10:10:22 aguacate-srv-us1 guacd[19384]: Connection "$d5a18cb3-2596-4a0c-9ab6-b03b019c196f" removed.

      Case 2: Logs, active User Disconnect

      With additional log lines from guacd daemon start :

      Jan 18 10:11:01 aguacate-srv-us1 systemd: Removed slice User Slice of root.
Jan 18 10:11:03 aguacate-srv-us1 guacd[19414]: Guacamole proxy daemon (guacd) version 1.4.0 started
Jan 18 10:11:03 aguacate-srv-us1 guacd[19414]: Communication will require SSL/TLS.
Jan 18 10:11:03 aguacate-srv-us1 guacd[19414]: Using PEM keyfile /etc/guacamole/ssl/aguacate-srv-us1.server.lan.key
Jan 18 10:11:03 aguacate-srv-us1 guacd[19414]: Using certificate file /etc/guacamole/ssl/aguacate-srv-us1.server.lan.pem
Jan 18 10:11:03 aguacate-srv-us1 guacd[19414]: Listening on host 172.19.209.153, port 4822
Jan 18 10:11:06 aguacate-srv-us1 guacd[19414]: Creating new client for protocol "rdp"
Jan 18 10:11:06 aguacate-srv-us1 guacd[19414]: Connection ID is "$79aa04b7-698e-49a1-ba13-75114dc85456"
Jan 18 10:11:06 aguacate-srv-us1 guacd[19416]: Security mode: Negotiate (ANY)
Jan 18 10:11:06 aguacate-srv-us1 guacd[19416]: Resize method: none
Jan 18 10:11:06 aguacate-srv-us1 guacd[19416]: No clipboard line-ending normalization specified. Defaulting to preserving the format of all line endings.
Jan 18 10:11:06 aguacate-srv-us1 guacd[19416]: User "@51e12e7d-9ff7-41c0-ac9c-0a5fa422817e" joined connection "$79aa04b7-698e-49a1-ba13-75114dc85456" (1 users now present)
Jan 18 10:11:06 aguacate-srv-us1 guacd[19416]: Loading keymap "base"
Jan 18 10:11:06 aguacate-srv-us1 guacd[19416]: Loading keymap "en-us-qwerty"
Jan 18 10:11:08 aguacate-srv-us1 guacd[19416]: Connected to RDPDR 1.13 as client 0x0004
Jan 18 10:11:21 aguacate-srv-us1 guacd[19416]: Connected to RDPDR 1.13 as client 0x0003
Jan 18 10:11:21 aguacate-srv-us1 guacd[19416]: RDPDR user logged on
Jan 18 10:11:33 aguacate-srv-us1 guacd[19416]: RDP server closed/refused connection: Manually disconnected.
Jan 18 10:11:33 aguacate-srv-us1 guacd[19416]: Internal RDP client disconnected
Jan 18 10:11:33 aguacate-srv-us1 kernel: guacd[19420]: segfault at 10 ip 00007fd027dd0972 sp 00007fd022e71ba0 error 4 in libguac-client-rdp.so.0.0.0[7fd027db0000+35000]
Jan 18 10:11:33 aguacate-srv-us1 guacd[19414]: Connection "$79aa04b7-698e-49a1-ba13-75114dc85456" removed.

      Core Dumps from Signal SIGSEGV

      Description Core Dump
      Case 1: Disconnect on Timeout core-guacd-11-0-0-19386-1642497022_disconnect_timeout.xz
      Case 2: Active User Disconnect core-guacd-11-0-0-19416-1642497093_disconnect_active.xz

      In both use cases it seems to be the same issue with the same backtrace.

      It seems to be always the same first thread, the first one which segfaults, in this example the one with PID 20440:

      guacd(19654)─┬─guacd(20437)─┬─{guacd}(20440)
             │              ├─{guacd}(20441)
             │              ├─{guacd}(20442)
             │              ├─{guacd}(20443)
             │              ├─{guacd}(20444)
             │              ├─{guacd}(20445)
             │              ├─{guacd}(20446)
             │              ├─{guacd}(20447)
             │              ├─{guacd}(20448)
             │              └─{guacd}(20449)

      Case 1: Backtrace, disconnect on Timeout

      gdb /sbin/guacd core-guacd-11-0-0-19386-1642497022_disconnect_timeout
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/guacd...Reading symbols from /usr/lib/debug/usr/sbin/guacd.debug...done.
done.
[New LWP 19390]
[New LWP 19386]
[New LWP 19389]
[New LWP 19393]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/sbin/guacd -f'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007fc7ec9a5972 in guac_rdp_user_leave_handler (user=0x2985e10) at user.c:169
169         guac_common_cursor_remove_user(rdp_client->display->cursor, user);
Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.x86_64 cairo-1.15.12-4.el7.x86_64 elfutils-libelf-0.176-5.el7.x86_64 elfutils-libs-0.176-5.el7.x86_64 expat-2.1.0-12.el7.x86_64 fontconfig-2.13.0-4.3.el7.x86_64 freerdp-libs-2.1.1-5.el7_9.x86_64 freetype-2.8-14.el7_9.1.x86_64 glibc-2.17-325.el7_9.x86_64 gsm-1.0.13-11.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-51.el7_9.x86_64 libX11-1.6.7-4.el7_9.x86_64 libXau-1.0.8-2.1.el7.x86_64 libXext-1.3.3-3.el7.x86_64 libXrender-0.9.10-1.el7.x86_64 libattr-2.4.46-13.el7.x86_64 libcap-2.22-11.el7.x86_64 libcom_err-1.42.9-19.el7.x86_64 libgcc-4.8.5-44.el7.x86_64 libgcrypt-1.5.3-14.el7.x86_64 libglvnd-1.0.1-0.8.git5baa1e5.el7.x86_64 libglvnd-egl-1.0.1-0.8.git5baa1e5.el7.x86_64 libglvnd-glx-1.0.1-0.8.git5baa1e5.el7.x86_64 libgpg-error-1.12-3.el7.x86_64 libicu-50.2-4.el7_7.x86_64 libjpeg-turbo-1.2.90-8.el7.x86_64 libogg-1.3.0-7.el7.x86_64 libpng-1.5.13-8.el7.x86_64 libselinux-2.5-15.el7.x86_64 libssh2-1.8.0-4.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64 libuuid-2.23.2-65.el7_9.1.x86_64 libvorbis-1.3.3-8.el7.1.x86_64 libwebp-0.3.0-10.el7_9.x86_64 libwinpr-2.1.1-5.el7_9.x86_64 libxcb-1.13-1.el7.x86_64 libxkbfile-1.0.9-3.el7.x86_64 lz4-1.8.3-1.el7.x86_64 openssl-libs-1.0.2k-22.el7_9.x86_64 pcre-8.32-17.el7.x86_64 pixman-0.34.0-1.el7.x86_64 systemd-libs-219-78.el7_9.5.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-19.el7_9.x86_64
(gdb) backtrace
#0  0x00007fc7ec9a5972 in guac_rdp_user_leave_handler (user=0x2985e10) at user.c:169
#1  0x00007fc7f3c1b580 in guac_user_handle_connection (user=user@entry=0x2985e10, usec_timeout=usec_timeout@entry=15000000) at user-handshake.c:362
#2  0x00000000004048c6 in guacd_user_thread (data=0x7fc7e80249e0) at proc.c:98
#3  0x00007fc7f303bea5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007fc7f1937b0d in clone () from /lib64/libc.so.6

      Case 2: Backtrace, active User Disconnect

      gdb /sbin/guacd core-guacd-11-0-0-19416-1642497093_disconnect_active
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/guacd...Reading symbols from /usr/lib/debug/usr/sbin/guacd.debug...done.
done.
[New LWP 19420]
[New LWP 19423]
[New LWP 19419]
[New LWP 19416]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/sbin/guacd -f'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007fd027dd0972 in guac_rdp_user_leave_handler (user=0x20a2e10) at user.c:169
169         guac_common_cursor_remove_user(rdp_client->display->cursor, user);
Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.x86_64 cairo-1.15.12-4.el7.x86_64 elfutils-libelf-0.176-5.el7.x86_64 elfutils-libs-0.176-5.el7.x86_64 expat-2.1.0-12.el7.x86_64 fontconfig-2.13.0-4.3.el7.x86_64 freerdp-libs-2.1.1-5.el7_9.x86_64 freetype-2.8-14.el7_9.1.x86_64 glibc-2.17-325.el7_9.x86_64 gsm-1.0.13-11.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-51.el7_9.x86_64 libX11-1.6.7-4.el7_9.x86_64 libXau-1.0.8-2.1.el7.x86_64 libXext-1.3.3-3.el7.x86_64 libXrender-0.9.10-1.el7.x86_64 libattr-2.4.46-13.el7.x86_64 libcap-2.22-11.el7.x86_64 libcom_err-1.42.9-19.el7.x86_64 libgcc-4.8.5-44.el7.x86_64 libgcrypt-1.5.3-14.el7.x86_64 libglvnd-1.0.1-0.8.git5baa1e5.el7.x86_64 libglvnd-egl-1.0.1-0.8.git5baa1e5.el7.x86_64 libglvnd-glx-1.0.1-0.8.git5baa1e5.el7.x86_64 libgpg-error-1.12-3.el7.x86_64 libicu-50.2-4.el7_7.x86_64 libjpeg-turbo-1.2.90-8.el7.x86_64 libogg-1.3.0-7.el7.x86_64 libpng-1.5.13-8.el7.x86_64 libselinux-2.5-15.el7.x86_64 libssh2-1.8.0-4.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64 libuuid-2.23.2-65.el7_9.1.x86_64 libvorbis-1.3.3-8.el7.1.x86_64 libwebp-0.3.0-10.el7_9.x86_64 libwinpr-2.1.1-5.el7_9.x86_64 libxcb-1.13-1.el7.x86_64 libxkbfile-1.0.9-3.el7.x86_64 lz4-1.8.3-1.el7.x86_64 openssl-libs-1.0.2k-22.el7_9.x86_64 pcre-8.32-17.el7.x86_64 pixman-0.34.0-1.el7.x86_64 systemd-libs-219-78.el7_9.5.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-19.el7_9.x86_64
(gdb) backtrace
#0  0x00007fd027dd0972 in guac_rdp_user_leave_handler (user=0x20a2e10) at user.c:169
#1  0x00007fd033159580 in guac_user_handle_connection (user=user@entry=0x20a2e10, usec_timeout=usec_timeout@entry=15000000) at user-handshake.c:362
#2  0x00000000004048c6 in guacd_user_thread (data=0x7fd0280249c0) at proc.c:98
#3  0x00007fd032579ea5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007fd030e75b0d in clone () from /lib64/libc.so.6

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. GUACAMOLE-1496 error 4 in libguac-client-rdp.so.0.0.0

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. GUACAMOLE-1203 guacd, Release 1.2.0 segfaults in connection with RDP-Sessions

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              mjumper Mike Jumper
              jkuri Juergen Kuri
              Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: