Enable and package the C# language extension on Linux - #110
Draft
SicongLiu2000 wants to merge 37 commits into
Draft
Enable and package the C# language extension on Linux#110SicongLiu2000 wants to merge 37 commits into
SicongLiu2000 wants to merge 37 commits into
Conversation
SQL Server on Linux expects a .tar.gz file instead of a .zip when using CREATE EXTERNAL LANGUAGE.
We will use the nethost to locate the system's installed hostfxr, as using a local libhostfxr.so breaks the .NET runtime lookup. nethost contains the logic needed to find the .NET root based on the DOTNET_ROOT environment variable or the install_location configuration file, which we will require for .NET runtime lookup on diverging Linux distributions (e.g. RHEL and Debian packages not installing .NET in the same base directories).
- Replace PowerShell build scripts with bash (build, archive, restore) - Remove checked-in libnethost.a; fetch at build time via restore-packages.sh - Add libnethost.a existence guard in build script - Add convert_string() abstraction in DotnetEnvironment - Conditional nethost.h include (Linux only), HOSTFXR_PATH_BUFFER_SIZE constant - Add LOG_ERROR on hostfxr/dlopen/dlsym failures (release-safe, not just assert) - Guard S_OK/E_FAIL macros (#ifndef) on non-Windows - Add Linux test build/run scripts and CMake platform support - Add -fshort-wchar and replace std::wstring with vector<wchar_t>/raw wchar_t* in test code to avoid ABI mismatch under -fshort-wchar on Linux - Add platform-conditional HintPath in test csproj - Add Linux dlopen/dlsym equivalents in test API code; throw on readlink failure (cherry picked from commit 2497aa2)
Bundle the entire .NET runtime (hostfxr, coreclr, shared framework) in the extension tar.gz so it works in containers without a system .NET installation. Changes: - build-dotnet-core-CSharp-extension.sh: dotnet build -> dotnet publish --self-contained -r linux-x64 - restore-packages.sh: add -r linux-x64 to dotnet restore for runtime pack download during network-enabled phase - DotnetEnvironment.cpp: pass assembly_path to get_hostfxr_path() for self-contained hostfxr discovery; default dotnet_root to extension directory when DOTNET_ROOT env var is not set Fixes: Extension Init() failed with -8223344 (hostfxr not found)
Load libhostfxr.so from the extension directory (self-contained deployment) rather than using get_hostfxr_path(), which may find an incompatible older hostfxr (e.g. .NET Core 3.x shipped with SQL Server). This mirrors the Windows approach of loading hostfxr.dll from the extension root. Falls back to nethost discovery for framework-dependent deployments where the bundled libhostfxr.so is not present. Fixes: hostfxr_initialize_for_runtime_config failed with 0x80008093 (Initialization for self-contained components is not supported)
Adds [DIAG] cout messages to load_hostfxr() and get_dotnet() to trace: - m_root_path value - Whether bundled libhostfxr.so is found via access() - Which hostfxr path is actually loaded (bundled vs nethost fallback) - config_path and dotnet_root values passed to hostfxr - hostfxr_initialize_for_runtime_config return code
…d Linux deployment hostfxr_initialize_for_runtime_config rejects self-contained runtimeconfig.json files (with 'includedFrameworks') returning error 0x80008093 (HostIncompatibleConfig). This is by design in .NET - that API only supports framework-dependent components. For self-contained deployments on Linux, use hostfxr_initialize_for_dotnet_command_line instead, which supports both framework-dependent and self-contained apps (.NET 5+). The hdt_load_assembly_and_get_function_pointer delegate works from command-line initialized contexts since .NET 5. Validated locally in WSL with bundled .NET 8.0.25 hostfxr - returns 0x00000000.
…ned .NET hosting - Use hostfxr_initialize_for_dotnet_command_line instead of hostfxr_initialize_for_runtime_config for self-contained deployments - Use hdt_get_function_pointer (type 6, Default ALC) instead of hdt_load_assembly_and_get_function_pointer (type 5, Isolated ALC) to resolve AssemblyLoadContext type cast failures - Add get_function_pointer_fn typedef and hdt_get_function_pointer enum - Dual-path call_managed_method: tries Default ALC first, falls back to Isolated ALC for backward compatibility
…nt runtimeconfig hostfxr_initialize_for_dotnet_command_line returns 0x80008093 (InvalidConfigFile) because .NET does not support self-contained component initialization. Instead: - Post-process runtimeconfig.json: replace includedFrameworks with framework - Create shared/Microsoft.NETCore.App/<version>/ symlink to extension root - Always use hostfxr_initialize_for_runtime_config (works for both modes) - Include shared/ directory in tar.gz - Fix double 0x prefix in to_hex_string
…ction SQL Server's tar extraction does not preserve symbolic links when extracting CREATE EXTERNAL LANGUAGE payloads. The shared/Microsoft.NETCore.App/<version>/ symlink (pointing to ../../../) was silently dropped, causing hostfxr to fail with 0x80008096 (FrameworkMissingFailure) - 'No frameworks were found'. Replace the directory symlink with hard links for each .dll and .so file. Hard links are stored efficiently in tar (as link entries with zero additional tar size) and are reliably extracted by all tar implementations - either as real hard links or as file copies.
SQL Server's CREATE EXTERNAL LANGUAGE tar extraction silently drops both symbolic links AND hard links. Use actual file copies (cp) to ensure framework DLLs/SOs are always present after extraction. Previous attempts: - Symlinks (ln -sfn): dropped by tar extraction -> 0x80008096 - Hard links (ln): also dropped by tar extraction -> 0x80008096 - File copies (cp): regular file entries, always extracted correctly
…debugging Log dotnet_root path, shared/ directory existence, framework version directories, runtimeconfig.json content, and relevant environment variables before calling hostfxr_initialize_for_runtime_config. This helps diagnose the 0x80008096 (FrameworkMissingFailure) error.
hostfxr requires Microsoft.NETCore.App.deps.json to recognize a framework version directory as valid. The build script was only copying *.dll and *.so, missing the .deps.json and .version files that hostfxr needs for framework resolution. This caused error 0x80008096 (FrameworkMissingFailure).
The .deps.json file only exists in the .NET SDK's shared/Microsoft.NETCore.App/ directory, NOT in the self-contained publish output. hostfxr requires this file to recognize a framework version as valid. Without it, hostfxr reports FrameworkMissingFailure (0x80008096) even when all DLLs/SOs are present.
…aged_method The get_dotnet_load_assembly function previously returned immediately after obtaining hdt_get_function_pointer without also obtaining hdt_load_assembly_and_get_function_pointer. This meant that when the Default ALC failed to resolve Microsoft.SqlServer.CSharpExtension by name (as happens in component-host scenarios on Linux), the fallback path in call_managed_method was never available because m_load_assembly_and_get_function_pointer was null. Fix: Always obtain both delegates. The explicit-path delegate (hdt_load_assembly_and_get_function_pointer) serves as a reliable fallback when name-based resolution via hdt_get_function_pointer fails. Also: switch diagnostic logging from LOG() (debug-only) to LOG_ERROR() (always on) for key initialization checkpoints so failures are visible in Release builds.
…xx-init diagnostics
…nd at dlopen on Ubuntu 22.04-based mssql containers)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
DsMainDev C# Linux E2E was enabled on six x64 distros, but every board failed in
Setupbecause the officialdata-sql-language-extensions-linux 1.0.0-CI-master-20260813.1package contains R/Python/Java/ONNX only. Consequently mssql-server has no/opt/mssql-extensibility/lib/dotnet-core-CSharp-lang-extension-linux.zipto install.This branch rebases the Linux implementation from #64 onto current
mainand carries forward the proven runtime/package fixes fromdev/sicongliu/linuxTestwithout that branch's obsolete v2-host reset.What
nethost/hostfxrdiscovery and framework-dependent runtime layout that survives SQL archive extraction..zipoutput required byCREATE EXTERNAL LANGUAGE, while retaining.tar.gzcompatibility output.SetHostCallbacksimplementation.Validation
main(9a897b70).7992ab3.SetHostCallbacksnative/managed code remains present.dotnet-core-CSharp-lang-extension-linux.zip.Related: #64