attach: Gracefully handle file permission errors

When `memray attach` uses a debugger to attach to the target process,
it instructs the process to `dlopen()` a shared library that's part of
Memray. If the Memray install is owned by a different user than the
process being attached to, it's possible the target process won't have
the privileges necessary to load the library. In GDB's case this leads
to an internal error, taking the target process down with the debugger.

Work around this by adding an explicit permission check to the GDB
script and bailing out with a clear error message if the shared library
can't be read. Add a matching error message to the LLDB script, though
for LLDB we can let the `dlopen` run and gracefully handle the failure.

Signed-off-by: Andrew Pan <andrew.pan@trailofbits.com>
Co-authored-by: Matt Wozniski <mwozniski@bloomberg.net>

Author: tnytown

news/940.bugfix.rst

@@ -0,0 +1 @@
+Fix a bug that crashed target processes when ``memray attach`` fails. The underlying permissions issue is now accounted for and a troubleshooting message prints when it arises.

src/memray/commands/_attach.gdb

@@ -18,6 +18,13 @@ p "MEMRAY: Process is Python 3.7+."
 set scheduler-locking on
 call (int)Py_AddPendingCall(&PyCallable_Check, (void*)0)
 
+p "Checking if we can access the library"
+set $result = (int)access($libpath, 5)
+if $result != 0
+    p "cannot open shared object file"
+    quit 1
+end
+
 # When updating this list, also update the "commands" call below,
 # and the breakpoints hardcoded for lldb in attach.py
 b malloc

src/memray/commands/_attach.lldb

@@ -9,7 +9,10 @@ p ((void(*)(void*))PyMem_Free)
 
 p "MEMRAY: Process is Python 3.7+."
 
-# When adding new breakpoints, also update _attach.gdb
+# When adding new breakpoints, also update _attach.gdb - but note that
+# PyErr_CheckSignals and PyCallable_Check are intentionally excluded,
+# as for some reason including them caused lldb on Linux to stop at
+# breakpoints that had already been deleted.
 breakpoint set -b malloc -b calloc -b realloc -b free -b PyMem_Malloc -b PyMem_Calloc -b PyMem_Realloc -b PyMem_Free
 
 # Set commands to execute when breakpoint is reached
@@ -21,7 +24,8 @@ expr auto $dlerror = $dlsym($rtld_default, "dlerror")
 expr auto $dll = ((void*(*)(const char*, int))$dlopen)($libpath, $rtld_now)
 p ((char*(*)(void))$dlerror)()
 expr auto $spawn = $dlsym($dll, "memray_spawn_client")
-p ((int(*)(int))$spawn)($port)?"FAILURE":"SUCCESS"
+p ((char*(*)(void))$dlerror)()
+p (!$dll || !$spawn || ((int(*)(int))$spawn)($port)) ? "FAILURE" : "SUCCESS"
 DONE
 
 continue

src/memray/commands/attach.py

@@ -231,6 +231,13 @@ def debugger_inject(debugger: str, pid: int, port: int, verbose: bool) -> str |
         if "MEMRAY: Process is Python 3.7+." not in output:
             return "The process does not seem to be running Python 3.7 or newer."
 
+    if "cannot open shared object file" in output:
+        return (
+            "The target process couldn't open the memray shared library.\n"
+            "This can occur if the memray installation is owned by a different user "
+            "and is inaccessible to the target process."
+        )
+
     return "An unexpected error occurred. Run with --verbose to debug the failure."