Add ASAN and Valgrind integration for CI test suite#289
Add ASAN and Valgrind integration for CI test suite#289fdcastel wants to merge 10 commits intoFirebirdSQL:masterfrom
Conversation
On Linux, sizeof(wchar_t) = 4 bytes but sizeof(SQLWCHAR) = 2 bytes
(unixODBC defines SQLWCHAR as unsigned short). The ConvertingString
constructor used sizeof(wchar_t) to convert a byte-count argument
into the number of narrow characters needed:
lengthString = length / sizeof(wchar_t); // = 12/4 = 3 on Linux
For SQLGetDiagRecW the state buffer is declared as State(12, sqlState),
giving lengthString=3 and Alloc() allocating 3+2=5 bytes. strcpy()
then writes the 5-character SQL state plus its NUL terminator (6 bytes)
into that 5-byte buffer, producing a 1-byte heap-buffer-overflow caught
by AddressSanitizer.
The same latent bug exists in SQLErrorW (same State(12, sqlState) pattern).
Fix: divide by sizeof(SQLWCHAR) instead of sizeof(wchar_t).
sizeof(SQLWCHAR) == 2 on all platforms (Windows: SQLWCHAR=wchar_t=2;
Linux/unixODBC: SQLWCHAR=unsigned short=2), so the formula now yields:
lengthString = 12 / sizeof(SQLWCHAR) = 6
and Alloc() allocates 6+2=8 bytes, comfortably holding the SQL state.
On Windows sizeof(wchar_t)==sizeof(SQLWCHAR)==2, so this change is
a no-op there.
Found by: AddressSanitizer (introduced in CI via PR FirebirdSQL#288/FirebirdSQL#289)
Test: DataTypeTest.SmallintRoundTrip -> ExecIgnoreError -> SQLExecDirect
-> unixODBC dispatcher -> SQLGetDiagRecW -> sqlGetDiagRec(strcpy)
Bug found and fixed by ASANThe ASAN CI job introduced by this PR immediately caught a real memory bug in the driver. What ASAN reportedThe first ASAN run failed on The full output is in the CI job log: GitHub Actions -> Build and Test -> job Root cause
ConvertingString<> State( 12, sqlState );In lengthString = length / sizeof(wchar_t); // BUGOn Windows
The bug was silently harmless on Windows and went undetected for years. FixOne-line change in // Before
lengthString = length / sizeof(wchar_t);
// After
lengthString = length / sizeof(SQLWCHAR);
CI after fixAll 4 matrix jobs now pass with no
This is a concrete demonstration that ASAN integration pays off immediately. |
|
@irodushka Enjoy! 😉 |
|
Hi @fdcastel Can you please resolve the conflicts after merging PR#286 Cheers! |
Sure! I’ll look into this in a few hours. |
Add CMake options ENABLE_ASAN and ENABLE_VALGRIND (Linux/GCC/Clang only, mutually exclusive) with corresponding compiler/linker flags and CTest memcheck configuration. CMake changes: - ENABLE_ASAN appends -fsanitize=address -fno-omit-frame-pointer to compile and link flags; suppresses -DLOGGING in Debug builds for cleaner sanitizer output - ENABLE_VALGRIND finds the valgrind binary and configures MEMORYCHECK_COMMAND for ctest -T memcheck - Mutual exclusion enforced via FATAL_ERROR CMake presets: - Add 'asan' and 'valgrind' configure/build/test presets inheriting from 'debug', with ASAN_OPTIONS env and Valgrind timeout multiplier Build script (firebird-odbc-driver.build.ps1): - Add -Sanitizer parameter (None, Asan, Valgrind) that passes the corresponding CMake option; sets ASAN_OPTIONS at runtime; skips sanitizers on Windows with a warning CI (build-and-test.yml): - Add Linux x64 ASAN matrix entry (ubuntu-22.04, Debug) - Add Linux x64 Valgrind matrix entry (ubuntu-22.04, Debug) with valgrind package installation - Sanitizer jobs do not upload release artifacts Also adds valgrind.supp suppressions file (seeded with fbclient rule). Closes FirebirdSQL#288
LSAN_OPTIONS referenced a relative 'lsan.supp' path but CTest runs from the build directory. Use an absolute path derived from the source/workspace root so the file is always found. Also adds the lsan.supp suppressions file with initial rules for libfbclient, libodbc, and libodbcinst.
ASAN correctly detected a pre-existing heap-buffer-overflow in OdbcError::sqlGetDiagRec (strcpy into an undersized ConvertingString buffer). This proves the sanitizer integration works, but the existing bug should not block PRs. Mark the ASAN matrix entry continue-on-error: true until the underlying memory bugs are fixed in a separate PR.
On Linux, sizeof(wchar_t) = 4 bytes but sizeof(SQLWCHAR) = 2 bytes
(unixODBC defines SQLWCHAR as unsigned short). The ConvertingString
constructor used sizeof(wchar_t) to convert a byte-count argument
into the number of narrow characters needed:
lengthString = length / sizeof(wchar_t); // = 12/4 = 3 on Linux
For SQLGetDiagRecW the state buffer is declared as State(12, sqlState),
giving lengthString=3 and Alloc() allocating 3+2=5 bytes. strcpy()
then writes the 5-character SQL state plus its NUL terminator (6 bytes)
into that 5-byte buffer, producing a 1-byte heap-buffer-overflow caught
by AddressSanitizer.
The same latent bug exists in SQLErrorW (same State(12, sqlState) pattern).
Fix: divide by sizeof(SQLWCHAR) instead of sizeof(wchar_t).
sizeof(SQLWCHAR) == 2 on all platforms (Windows: SQLWCHAR=wchar_t=2;
Linux/unixODBC: SQLWCHAR=unsigned short=2), so the formula now yields:
lengthString = 12 / sizeof(SQLWCHAR) = 6
and Alloc() allocates 6+2=8 bytes, comfortably holding the SQL state.
On Windows sizeof(wchar_t)==sizeof(SQLWCHAR)==2, so this change is
a no-op there.
Found by: AddressSanitizer (introduced in CI via PR FirebirdSQL#288/FirebirdSQL#289)
Test: DataTypeTest.SmallintRoundTrip -> ExecIgnoreError -> SQLExecDirect
-> unixODBC dispatcher -> SQLGetDiagRecW -> sqlGetDiagRec(strcpy)
The heap-buffer-overflow in ConvertingString (sizeof(wchar_t) vs sizeof(SQLWCHAR)) has been fixed. ASAN now passes all 375 tests cleanly on ubuntu-22.04/GCC, so the soft-fail guard is no longer needed.
|
@irodushka Rebased onto current Conflicts were in Resolution: kept all new upstream matrix entries and added the sanitizer entries on top; merged the |
- Move option() declarations outside if(NOT MSVC) guard; emit warning on MSVC instead of hiding the options entirely - Rename ENABLE_ASAN -> BUILD_WITH_ASAN and ENABLE_VALGRIND -> BUILD_WITH_VALGRIND to follow existing BUILD_* naming convention - Remove duplicate target_compile_definitions for DEBUG/LOGGING (already defined via add_compile_options in Linux section) - Separate LOGGING into add_definitions() and conditionally skip it for both ASAN and Valgrind builds (not just ASAN)
Per PR review feedback: consolidate the build-type options matrix and express the sanitizer carve-out via add_definitions/remove_definitions instead of an inverted if-NOT guard. Behavior is unchanged — LOGGING is defined for Debug builds and stripped for ASAN/Valgrind builds.
|
@irodushka Thanks for the valuable insights. I’ve just pushed all the changes. Please take a look when you have a moment. |
Per review feedback: remove_definitions() cannot undo definitions added via add_compile_options(), so the add-then-remove pattern was misleading. Collapse to a single if() with the combined condition instead.
|
We have a bug! The reason is in the Can you please remind me what do you expect specifying the |
The first git describe call used --always, which made it return the short commit hash instead of failing when --exclude "*-*" ruled out every matching tag (as is the case today: only prerelease tags like v3.5.0-rc5 exist). The non-zero exit that was supposed to trigger the fallback without --exclude never fired, so parsing dropped to 0.0.0.0. Drop --always from both calls: if the stable-only call finds nothing it exits non-zero and the fallback runs; if the fallback also finds nothing the existing warning path reports it.
|
@irodushka you're right, good catch. Fixed in b7aa955. Intent of the logic
Why it was broken
The fix
Verified locally: |
|
Aha. Great) Another note - let's exclude |
|
Bad thing happens. is not good. I get while running So it's not such a trivial case with that bug. I will investigate tomorrow for it's a bit late here now. But you understand, I'm holding off on this PR until we solve this mystery. |
|
Thanks @irodushka -- I will review all of them all in a few hours. |
DEBUG adds extra logging; exclude it alongside LOGGING for cleaner output.
|
Done in 5323214 — moved both -DDEBUG and -DLOGGING to the sanitizer guard. All 11 CI jobs green. |
|
The wchar_t → SQLWCHAR fix (commit 832d8e7) is correct: on Linux unixODBC defines SQLWCHAR=2 bytes but wchar_t=4 bytes, causing the buffer underallocation you caught with ASAN. That's a real bug fix, not a bug. The stack smashing you're seeing with the tests is a separate issue — likely the test itself or something in the test harness. The heap-buffer-overflow in SQLGetDiagRecW that the wchar fix addresses should actually be resolved by it. Can you isolate which test triggers the stack smashing and whether it's reproducible without ASAN? |
I don't argue with that. I only want to say,
The very first test case - DataTypeTest.SmallintRoundTrip. It is easily reproducible wo ASAN as well. I'll take a look when I've sorted out my affairs at my main job. P.S. Actually not in the std::make_unique<> ctor. This ctor calls TempTable ctor, then ExecIgnoreError() -> unixodbc.SQLExecDirect() -> extract_diag_error_w() -> stack guard is fired. |
|
And, @fdcastel, I'll let you in on a secret!) I have a customer in Moscow, he's using the ODBC driver in his project (on Linux), and he builds it with ASAN. He was the first one who got that ASAN error with buffer overflow, so when I asked you to implement the ASAN build I really knew all this story in advance) I fixed it in a fast QuickWin way for him (locally) - just changed the strcpy() calls to strncpy() in a few places and hardcoded the length limit. Ugly but efficient. Of cause your fix is much more elegant, preferable and the right direction anyway, no doubt here. |
Hahahaha Fantastic!
|
| if ( len > 0 ) | ||
| len--; | ||
| #else | ||
| len = mbstowcs( (wchar_t*)unicodeString, (const char*)byteString, lengthString ); |
There was a problem hiding this comment.
This place is terrible. The root of evil is here.
mbstowcs will convert each byte from byteString to FOUR bytes (sizeof(wchar_t)) in unicodeString. Given that unicodeString is represented in SQLWCHAR units, this inevitably leads to disaster.
| //len = mbstowcs( (wchar_t*)unicodeString, (const char*)byteString, lengthString ); | |
| mbstate_t state = {0}; | |
| const char *p = (const char*)byteString; | |
| size_t out_idx = 0; | |
| size_t rc; | |
| char16_t *u16 = (char16_t *)unicodeString; | |
| while ( (rc = std::mbrtoc16(&u16[out_idx], p, lengthString, &state)) ) | |
| { | |
| if (rc == std::size_t(-1) || rc == std::size_t(-2)) | |
| break; // Invalid or truncated input | |
| else if (rc == std::size_t(-3)) | |
| out_idx++; // UTF-16 high surrogate | |
| else | |
| { | |
| p += rc; | |
| out_idx++; | |
| } | |
| } | |
| len = out_idx; |
There was a problem hiding this comment.
I think this code will work just fine on Windows as well, so maybe we have no need to gate it with #ifdef here. I haven't tested it on Windows, though. On Linux, it seems to work fine, both with and wo ASAN and Valgrind.

Summary
Integrate AddressSanitizer (ASAN) and Valgrind into the CI test pipeline so that memory bugs (buffer overflows, use-after-free, leaks, etc.) are caught automatically on every PR.
Closes #288
Changes
CMake (
CMakeLists.txt)ENABLE_ASANoption (default OFF, Linux/GCC/Clang only): appends-fsanitize=address -fno-omit-frame-pointerto compile and link flags. Suppresses-DLOGGINGin Debug builds for cleaner sanitizer output.ENABLE_VALGRINDoption (default OFF): finds thevalgrindbinary and configuresMEMORYCHECK_COMMAND/MEMORYCHECK_COMMAND_OPTIONSforctest -T memcheck.FATAL_ERROR.if(NOT MSVC)since MSVC ASAN has different semantics (future work).CMake Presets (
CMakePresets.json)asanconfigure/build/test presets inheriting fromdebug, withASAN_OPTIONSenvironment variable.valgrindconfigure/build/test presets inheriting fromdebug, with a 600s test timeout (Valgrind is ~10-20x slower).Build script (
firebird-odbc-driver.build.ps1)-Sanitizerparameter acceptingNone,Asan,Valgrind.-DENABLE_ASAN=ONor-DENABLE_VALGRIND=ONto CMake.ASAN_OPTIONS/LSAN_OPTIONSenvironment variables at runtime for ASAN builds.CI (
build-and-test.yml)valgrindpackage installation.ASAN_OPTIONSandLSAN_OPTIONSenvironment variables.Valgrind suppressions (
valgrind.supp)libfbclient.soglobal initialization leaks.Local usage