3.5.0 viewer crashes on startup when scalable rendering is set to always

[cyrush]

Describe the bug

Reproduce on LC using visit -v 3.5.0 with the following minimal ~.visit/config file:

<?xml version="1.0"?>
<Object name="VisIt">
    <Field name="Version" type="string">3.4.2</Field>
    <Object name="VIEWER">
        <Object name="DEFAULT_VALUES">
            <Object name="RenderingAttributes">
                <Field name="scalableActivationMode" type="string">Always</Field>
            </Object>
    </Object>
</Object>

[cyrush@rzwhippet58:visit]$ visit -v 3.5.0 
Running: gui3.5.0 -forceversion 3.5.0
Running: viewer3.5.0 -forceversion 3.5.0 -geometry 1503x999+417+54 -borders 29,5,5,5 -shift 17,27 -preshift -12,2 -defer -host 127.0.0.1 -port 5600
Running: mdserver3.5.0 -forceversion 3.5.0 -host 127.0.0.1 -port 5601
VisIt's viewer exited abnormally! Aborting the Graphical User Interface. VisIt's developers may be reached via our GitHub discussions, https://github.com/visit-dav/visit/discussions

VisIt 3.4.2 runs fine.

[cyrush]

This also happens with the macOS release :-(

[markcmiller86]

Yep, just went looking and see that too. Lemme see if I can get a stack dump.

[markcmiller86]

@cyrush just an FYI if you run into this on your mac...

VisIt's debugger launching logic starts lldb in an xterm. However, on my M1 mac, all the XQuartz stuff is x86_64 and so running under Rosetta. If you naively launch lldb in sucn an xterm, the debugger is set to debug x86_64 code and will fail to launch/start any VisIt component. The solution is to ensure that VisIt's lldb launching logic does something like arch -arm64 /usr/bin/lldb from inside the xterm.

[markcmiller86]

Looks like I don't have things set to compile with debugging info...So, here is a partially useful stack dump...

  * frame #0: 0x0000000000000000
    frame #1: 0x0000000100676c70 libavtviswindow_ser.dylib`VisWinRendering::MSAAAvailable() + 92
    frame #2: 0x000000010055df10 libviewercore_ser.dylib`ViewerWindow::GetWindowAttributes() const + 1004
    frame #3: 0x000000010055e2fc libviewercore_ser.dylib`ViewerWindow::SendWindowEnvironmentToEngine(EngineKey const&) + 76
    frame #4: 0x000000010055f65c libviewercore_ser.dylib`ViewerWindow::ChangeScalableRenderingMode(bool) + 128
    frame #5: 0x000000010050f3ec libviewercore_ser.dylib`ViewerMessaging::ProcessCommands() + 124
    frame #6: 0x0000000106f3a3fc QtCore`void doActivate<false>(QObject*, int, void**) + 1412
    frame #7: 0x0000000107176bdc QtCore`QSingleShotTimer::timerEvent(QTimerEvent*) + 148
    frame #8: 0x0000000106f33040 QtCore`QObject::event(QEvent*) + 648
    frame #9: 0x0000000106825d38 QtWidgets`QApplicationPrivate::notify_helper(QObject*, QEvent*) + 272
    frame #10: 0x0000000106826dec QtWidgets`QApplication::notify(QObject*, QEvent*) + 452
    frame #11: 0x0000000106ef180c QtCore`QCoreApplication::notifyInternal2(QObject*, QEvent*) + 200
    frame #12: 0x00000001070570a8 QtCore`QTimerInfoList::activateTimers() + 968
    frame #13: 0x000000010c75b0f4 libqcocoa.dylib`QCocoaEventDispatcherPrivate::activateTimersSourceCallback(void*) + 28
    frame #14: 0x000000018ea66a64 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 28
    frame #15: 0x000000018ea669f8 CoreFoundation`__CFRunLoopDoSource0 + 172
    frame #16: 0x000000018ea66764 CoreFoundation`__CFRunLoopDoSources0 + 232
    frame #17: 0x000000018ea653b8 CoreFoundation`__CFRunLoopRun + 840
    frame #18: 0x000000018ea649e8 CoreFoundation`CFRunLoopRunSpecific + 572
    frame #19: 0x000000019a50627c HIToolbox`RunCurrentEventLoopInMode + 324
    frame #20: 0x000000019a5094e8 HIToolbox`ReceiveNextEventCommon + 676
    frame #21: 0x000000019a694484 HIToolbox`_BlockUntilNextEventMatchingListInModeWithFilter + 76
    frame #22: 0x0000000192985a34 AppKit`_DPSNextEvent + 684
    frame #23: 0x00000001933245cc AppKit`-[NSApplication(NSEventRouting) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 688
    frame #24: 0x0000000192978be4 AppKit`-[NSApplication run] + 480
    frame #25: 0x000000010c75ba54 libqcocoa.dylib`QCocoaEventDispatcher::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) + 1528
    frame #26: 0x0000000106ef98cc QtCore`QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) + 632
    frame #27: 0x0000000106ef1f1c QtCore`QCoreApplication::exec() + 200
    frame #28: 0x000000010001b3a0 viewer`ViewerMain(int, char**) + 744
    frame #29: 0x000000018e5dab98 dyld`start + 6076

[markcmiller86]

Ok, here is a better dump...

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000000000000
    frame #1: 0x0000000100ce0d88 libavtviswindow_ser.dylib`VisWinRendering::MSAAAvailable(this=0x000000013c6ba940) at VisWinRendering.C:2737:29
    frame #2: 0x0000000100d2c0c8 libavtviswindow_ser.dylib`VisWindow::MSAAAvailable(this=0x000000013c8ae400) const at VisWindow.C:6213:23
    frame #3: 0x0000000100702304 libviewercore_ser.dylib`ViewerWindow::MSAAAvailable(this=0x000000013c8b4000) const at ViewerWindow.C:7473:23
    frame #4: 0x0000000100701d4c libviewercore_ser.dylib`ViewerWindow::GetWindowAttributes(this=0x000000013c8b4000) const at ViewerWindow.C:6574:33
    frame #5: 0x0000000100702360 libviewercore_ser.dylib`ViewerWindow::SendWindowEnvironmentToEngine(this=0x000000013c8b4000, ek=0x000000013c6b73d0) at ViewerWindow.C:6680:30
    frame #6: 0x0000000100703910 libviewercore_ser.dylib`ViewerWindow::ChangeScalableRenderingMode(this=0x000000013c8b4000, newMode=true) at ViewerWindow.C:8349:9
    frame #7: 0x0000000100694e44 libviewercore_ser.dylib`ViewerCommandSetScalableRenderingMode::Execute(this=0x0000600001eb63a0) at ViewerInternalCommands.C:128:17
    frame #8: 0x0000000100695d0c libviewercore_ser.dylib`ViewerMessaging::ProcessCommands(this=0x0000600003d3cb90) at ViewerMessaging.C:201:18
    frame #9: 0x0000000100480934 libviewer.dylib`ViewerSubject::ProcessInternalCommands(this=0x000000013d904080) at ViewerSubject.C:3718:31
    frame #10: 0x00000001003fcad0 libviewer.dylib`ViewerSubject::qt_static_metacall(_o=0x000000013d904080, _c=InvokeMetaMethod, _id=18, _a=0x000000016fdfcbf8) at moc_ViewerSubject.cpp:356:22
    frame #11: 0x000000010829a3fc QtCore`void doActivate<false>(QObject*, int, void**) + 1412
    frame #12: 0x00000001084d6bdc QtCore`QSingleShotTimer::timerEvent(QTimerEvent*) + 148
    frame #13: 0x0000000108293040 QtCore`QObject::event(QEvent*) + 648
    frame #14: 0x0000000107b85d20 QtWidgets`QApplicationPrivate::notify_helper(QObject*, QEvent*) + 272
    frame #15: 0x0000000107b86dd4 QtWidgets`QApplication::notify(QObject*, QEvent*) + 452
    frame #16: 0x000000010825180c QtCore`QCoreApplication::notifyInternal2(QObject*, QEvent*) + 200
    frame #17: 0x00000001083b70a8 QtCore`QTimerInfoList::activateTimers() + 968
    frame #18: 0x000000010dabb0dc libqcocoa.dylib`QCocoaEventDispatcherPrivate::activateTimersSourceCallback(void*) + 28
    frame #19: 0x000000018ea66a64 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 28
    frame #20: 0x000000018ea669f8 CoreFoundation`__CFRunLoopDoSource0 + 172
    frame #21: 0x000000018ea66764 CoreFoundation`__CFRunLoopDoSources0 + 232
    frame #22: 0x000000018ea653b8 CoreFoundation`__CFRunLoopRun + 840
    frame #23: 0x000000018ea649e8 CoreFoundation`CFRunLoopRunSpecific + 572
    frame #24: 0x000000019a50627c HIToolbox`RunCurrentEventLoopInMode + 324
    frame #25: 0x000000019a5094e8 HIToolbox`ReceiveNextEventCommon + 676
    frame #26: 0x000000019a694484 HIToolbox`_BlockUntilNextEventMatchingListInModeWithFilter + 76
    frame #27: 0x0000000192985a34 AppKit`_DPSNextEvent + 684
    frame #28: 0x00000001933245cc AppKit`-[NSApplication(NSEventRouting) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 688
    frame #29: 0x0000000192978be4 AppKit`-[NSApplication run] + 480
    frame #30: 0x000000010dabba3c libqcocoa.dylib`QCocoaEventDispatcher::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) + 1528
    frame #31: 0x00000001082598cc QtCore`QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) + 632
    frame #32: 0x0000000108251f1c QtCore`QCoreApplication::exec() + 200
    frame #33: 0x000000010001b11c viewer`ViewerMain(argc=11, argv=0x000000016fdfef08) at viewer.C:274:26
    frame #34: 0x000000010001bc94 viewer`main(argc=17, argv=0x000000016fdfef08) at viewer.C:348:12
    frame #35: 0x000000018e5dab98 dyld`start + 6076
(lldb) up
frame #1: 0x0000000100ce0d88 libavtviswindow_ser.dylib`VisWinRendering::MSAAAvailable(this=0x000000013c6ba940) at VisWinRendering.C:2737:29
   2734     vtkOpenGLRenderWindow* oglWin = vtkOpenGLRenderWindow::SafeDownCast(GetRenderWindow());
   2735     int msamples = 0;
   2736     if(oglWin)
-> 2737         oglWin->GetState()->vtkglGetIntegerv(GL_MAX_SAMPLES, &msamples);
   2738     return (msamples > 1);
   2739 #endif
   2740     return false;
(lldb) print oglWin
(vtkGenericOpenGLRenderWindow *) 0x000000013c6b87e0
(lldb) print oglWin->GetState()
(vtkOpenGLState *) 0x000000013c8a1000

[cyrush]

I created a debug build on LC -- it's related to MSAA.

VisWindow::MSAAAvailable,                                                                      FP=7fffffff6ce0
ViewerWindow::MSAAAvailable,                                                                   FP=7fffffff6d00
ViewerWindow::GetWindowAttributes,                                                             FP=7fffffff7380
ViewerWindow::SendWindowEnvironmentToEngine,                                                   FP=7fffffff9e20
ViewerWindow::ChangeScalableRenderingMode,                                                     FP=7fffffff9e50
ViewerCommandSetScalableRenderingMode::Execute,                                                FP=7fffffff9e70
ViewerMessaging::ProcessCommands,                                                              FP=7fffffff9eb0
ViewerSubject::ProcessInternalCommands,                                                        FP=7fffffff9ee0
ViewerSubject::qt_static_metacall,                                                             FP=7fffffff9f30
     _Z10doActivateILb0EEvP7QObjectiPPv,                                                            FP=7fffffff9ff0
     _ZN16QSingleShotTimer10timerEventEP11QTimerEvent,                                              FP=7fffffffa010
     _ZN7QObject5eventEP6QEvent,                                                                    FP=7fffffffa080
     _ZN19QApplicationPrivate13notify_helperEP7QObjectP6QEvent,                                     FP=7fffffffa0b0
     _ZN16QCoreApplication15notifyInternal2EP7QObjectP6QEvent,                                      FP=7fffffffa120
     _ZN14QTimerInfoList14activateTimersEv,                                                         FP=7fffffffa1a0
     _ZL19timerSourceDispatchP8_GSourcePFiPvES1_,                                                   FP=7fffffffa1b0
     g_main_context_dispatch,                                                                       FP=7fffffffa240
     g_main_context_dispatch,                                                                       FP=7fffffffa2a0
     g_main_context_iteration,                                                                      FP=7fffffffa2c0
     _ZN20QEventDispatcherGlib13processEventsE6QFlagsIN10QEventLoop17ProcessEventsFlagEE,           FP=7fffffffa310
     _ZN10QEventLoop4execE6QFlagsINS_17ProcessEventsFlagEE,                                         FP=7fffffffa390
     _ZN16QCoreApplication4execEv,                                                                  FP=7fffffffa3e0
ViewerMain,                                                                                    FP=7fffffffa510
main,                                                                                          FP=7fffffffa530
     __libc_start_main,                                                                             FP=7fffffffa5e0
     _start,                                                                                        FP=7fffffffa5e8

[cyrush]

@biagas we are going to need your help with this one.

[markcmiller86]

What I suspect is happening is that when Scalable Rendering is Always in Settings, the GL context is not really ready during this point in VisIt startup to ask this question about MSAA features.

This change prevents the crash for me...

diff --git a/src/avt/VisWindow/Colleagues/VisWinRendering.C b/src/avt/VisWindow/Colleagues/VisWinRendering.C
index b717360f7c..6595fd4900 100644
--- a/src/avt/VisWindow/Colleagues/VisWinRendering.C
+++ b/src/avt/VisWindow/Colleagues/VisWinRendering.C
@@ -2733,7 +2733,7 @@ VisWinRendering::MSAAAvailable()
 #ifdef GL_MAX_SAMPLES
     vtkOpenGLRenderWindow* oglWin = vtkOpenGLRenderWindow::SafeDownCast(GetRenderWindow());
     int msamples = 0;
-    if(oglWin)
+    if (oglWin && oglWin->IsCurrent())
         oglWin->GetState()->vtkglGetIntegerv(GL_MAX_SAMPLES, &msamples);
     return (msamples > 1);
 #endif

What I don't know is whether it winds up being overzelous about disabling MSAA. I mean, does VisIt ever call this again, maybe after GL is initialized? If not, this fix is not appropriate.

But, it is very likely that GL has yet to have been sufficiently initialized at this point in VisIt startup to ask this question about GL.

[markcmiller86]

Another option might be to ignore any SR mode setting in settings file until VisIt has fully initialized and then if SR mode is set in the settings file, set SR mode to whatever the settings file says it should be at the end of initialization and before anything else (e.g. plots, etc.) runs.

[biagas]

@markcmiller86 thanks for pointing out place for fix, I will run some tests to see if I can answer your questions.

[biagas]

I tried testing this on rzwhippet, to see if Mark's suggested change would permanently disable MSAA.
However, cannot test that as MSAA is not available regardless of the config setting for scalable rendering, not sure why.
I know I have a note that MSAA doesn't work in our testing infrastructure, but I thought it worked otherwise on LC.
the call get get GL_MAX_SAMPLES returns '1'. Strange. I will test on Windows.