[regression] spandsp crashes when receiving fax in FreeSWITCH

[rlaager]

After upgrading libspandsp3 from 3.0.0-12951482879-0a9a1bd2da~bookworm to 3.0.0-16125392215-7977601689~bookworm, FreeSWITCH is immediately crashing upon receiving a fax.

Here is an example stack trace:

                Stack trace of thread 3333317:
                #0  0x00007f632d3be70b __memset_sse2_unaligned_erms (libc.so.6 + 0xa370b)
                #1  0x00007f632ce86470 fax_init (libspandsp.so.3 + 0x33470)
                #2  0x00007f6329cd0e1e spanfax_init (mod_spandsp.so + 0xfe1e)
                #3  0x00007f6329cd1c4c mod_spandsp_fax_process_fax (mod_spandsp.so + 0x10c4c)
                #4  0x00007f632d5a35ce switch_core_session_exec (libfreeswitch.so.1 + 0x935ce)
                #5  0x00007f632d5a3df1 switch_core_session_execute_application_get_flags (libfreeswitch.so.1 + 0x93df1)
                #6  0x00007f632d6abf2e _ZN11CoreSession7executeEPKcS1_ (libfreeswitch.so.1 + 0x19bf2e)
                #7  0x00007f63290ebffc _wrap_CoreSession_execute (mod_python3.so + 0x12ffc)
                #8  0x00007f6328afdbc1 n/a (libpython3.11.so.1.0 + 0x1c9bc1)
                #9  0x00007f6328aafe8d _PyObject_MakeTpCall (libpython3.11.so.1.0 + 0x17be8d)
                #10 0x00007f6328a3dc63 _PyEval_EvalFrameDefault (libpython3.11.so.1.0 + 0x109c63)
                #11 0x00007f6328b9eb6a n/a (libpython3.11.so.1.0 + 0x26ab6a)
                #12 0x00007f63290e3b2c eval_some_python (mod_python3.so + 0xab2c)
                #13 0x00007f63290e41cd python_function (mod_python3.so + 0xb1cd)
                #14 0x00007f632d5a35ce switch_core_session_exec (libfreeswitch.so.1 + 0x935ce)
                #15 0x00007f632d5a3df1 switch_core_session_execute_application_get_flags (libfreeswitch.so.1 + 0x93df1)
                #16 0x00007f632d5a7ab2 switch_core_standard_on_execute (libfreeswitch.so.1 + 0x97ab2)
                #17 0x00007f632d5a1150 switch_core_session_thread (libfreeswitch.so.1 + 0x91150)
                #18 0x00007f632d59c827 switch_core_session_thread_pool_worker (libfreeswitch.so.1 + 0x8c827)
                #19 0x00007f632d3a41f5 start_thread (libc.so.6 + 0x891f5)
                #20 0x00007f632d42489c __clone3 (libc.so.6 + 0x10989c)

The Python is because, in our case, we have some Python involved before rxfax is invoked.

I didn't get much out of the stack trace other than that spandsp was involved. I did a git bisect and found that commit d9681c3 is the first bad commit.

That commit is very large. I picked out the changes to autogen.sh, configure.ac, and m4 (as they seemed easy to separate). With those changes, it is still good. So the issue relates to actual source code changes made in d9681c3.

With my local build of d9681c3, here is a bt full:

#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0)
    at ./nptl/pthread_kill.c:44
        tid = <optimized out>
        ret = 0
        pd = <optimized out>
        old_mask = {__val = {4597140017428724033}}
        ret = <optimized out>
#1  0x00007fbfd44eff4f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
No locals.
#2  0x00007fbfd44a0fb2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
        ret = <optimized out>
#3  0x00007fbfd448b472 in __GI_abort () at ./stdlib/abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x20, sa_sigaction = 0x20}, sa_mask = {__val = {2314885530818453536, 0, 0, 0, 
              0, 4412756336094033712, 9083887384008815727, 365640440525450612, 579296435960222468, 9223372036854775808, 0, 
              13827552055903212896, 0, 0, 0, 0}}, sa_flags = 0, sa_restorer = 0x0}
#4  0x00007fbfd44e442f in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7fbfd45fe459 "%s\n")
    at ../sysdeps/posix/libc_fatal.c:156
        ap = {{gp_offset = 24, fp_offset = 0, overflow_arg_area = 0x7fbfd24b9b50, reg_save_area = 0x7fbfd24b9ae0}}
        fd = <optimized out>
        list = <optimized out>
        nlist = <optimized out>
        cp = <optimized out>
#5  0x00007fbfd44f986a in malloc_printerr (str=str@entry=0x7fbfd4601568 "malloc(): invalid size (unsorted)")
    at ./malloc/malloc.c:5660
No locals.
#6  0x00007fbfd44fc95c in _int_malloc (av=av@entry=0x7fbf38000030, bytes=bytes@entry=1041) at ./malloc/malloc.c:3998
        next = <optimized out>
        iters = <optimized out>
        nb = <optimized out>
        idx = <optimized out>
        bin = <optimized out>
        victim = <optimized out>
        size = <optimized out>
        victim_index = <optimized out>
        remainder = <optimized out>
        remainder_size = <optimized out>
        block = <optimized out>
        bit = <optimized out>
        map = <optimized out>
        fwd = <optimized out>
        bck = <optimized out>
        tcache_unsorted_count = 0
        tcache_nb = <optimized out>
        tc_idx = 64
        return_cached = <optimized out>
        __PRETTY_FUNCTION__ = "_int_malloc"
#7  0x00007fbfd44fd9b9 in __GI___libc_malloc (bytes=1041) at ./malloc/malloc.c:3323
        ar_ptr = 0x7fbf38000030
        victim = <optimized out>
        tbytes = <optimized out>
        tc_idx = <optimized out>
        __PRETTY_FUNCTION__ = "__libc_malloc"
#8  0x00007fbfd3ff0aec in queue_init () from /lib/x86_64-linux-gnu/libspandsp.so.3
No symbol table info available.
#9  0x00007fbfd402f6ee in v8_restart () from /lib/x86_64-linux-gnu/libspandsp.so.3
No symbol table info available.
#10 0x00007fbfd402f7a9 in v8_init () from /lib/x86_64-linux-gnu/libspandsp.so.3
No symbol table info available.
#11 0x00007fbfd3fddd4b in fax_init () from /lib/x86_64-linux-gnu/libspandsp.so.3
No symbol table info available.
--Type <RET> for more, q to quit, c to continue without paging--
#12 0x00007fbfd0eefe1e in spanfax_init (pvt=pvt@entry=0x7fbf38224a40, trans_mode=trans_mode@entry=AUDIO_MODE)
    at mod_spandsp_fax.c:814
        session = 0x7fbfb003e728
        channel = 0x7fbfb003c800
        fax = 0x7fbf38285588
        t38 = <optimized out>
        t30 = 0x7fbf38285588
        tmp = <optimized out>
        tz = <optimized out>
        fec_entries = <optimized out>
        fec_span = <optimized out>
        compressions = <optimized out>
        __PRETTY_FUNCTION__ = "spanfax_init"
        __func__ = "spanfax_init"
#13 0x00007fbfd0ef0c4c in mod_spandsp_fax_process_fax (session=0x7fbfb003e728, data=<optimized out>, app_mode=<optimized out>)
    at mod_spandsp_fax.c:1573
        pvt = 0x7fbf38224a40
        channel = 0x7fbfb003c800
        read_codec = {codec_interface = 0x0, implementation = 0x0, fmtp_in = 0x0, fmtp_out = 0x0, flags = 0, memory_pool = 0x0, 
          private_info = 0x0, agreed_pt = 0 '\000', mutex = 0x0, next = 0x0, session = 0x0, cur_frame = 0x0}
        write_codec = {codec_interface = 0x0, implementation = 0x0, fmtp_in = 0x0, fmtp_out = 0x0, flags = 0, memory_pool = 0x0, 
          private_info = 0x0, agreed_pt = 0 '\000', mutex = 0x0, next = 0x0, session = 0x0, cur_frame = 0x0}
        read_frame = 0x0
        write_frame = {codec = 0x0, source = 0x0, packet = 0x0, packetlen = 0, extra_data = 0x0, data = 0x0, datalen = 0, 
          buflen = 0, samples = 0, rate = 0, channels = 0, payload = 0 '\000', timestamp = 0, seq = 0, ssrc = 0, 
          m = SWITCH_FALSE, flags = 0, user_data = 0x0, pmap = 0x0, img = 0x0, geometry = {w = 0, h = 0, x = 0, y = 0, z = 0, 
            M = 0, X = 0}}
        read_impl = {codec_type = SWITCH_CODEC_TYPE_AUDIO, ianacode = 0 '\000', iananame = 0x55ebae50ee38 "PCMU", fmtp = 0x0, 
          samples_per_second = 8000, actual_samples_per_second = 8000, bits_per_second = 64000, microseconds_per_packet = 20000, 
          samples_per_packet = 160, decoded_bytes_per_packet = 320, encoded_bytes_per_packet = 160, 
          number_of_channels = 1 '\001', codec_frames_per_packet = 160, init = 0x7fbfd47d6a70 <switch_g711u_init>, 
          encode = 0x7fbfd47d6410 <switch_g711u_encode>, decode = 0x7fbfd47d6a90 <switch_g711u_decode>, encode_video = 0x0, 
          decode_video = 0x0, codec_control = 0x0, destroy = 0x7fbfd47d6a60 <switch_g711u_destroy>, codec_id = 123, 
          impl_id = 138, modname = 0x55ebae50e530 "CORE_PCM_MODULE", next = 0x55ebae50ed10}
        buf = 0x7fbf38226528
        req_counter = 0
        __func__ = "mod_spandsp_fax_process_fax"
        __PRETTY_FUNCTION__ = "mod_spandsp_fax_process_fax"

Note that malloc is giving the error malloc(): invalid size (unsorted) when called for a 1041 byte allocation from queue_init. I'm not certain, but this sure feels to me like corruption that occurred from something else, with this allocation being the thing that is triggering malloc to notice.

I'm not sure where to go next.

[SGaitskell]

Updated one of our servers to 3.0.0-16125392215-7977601689~bookworm this morning, and FreeSWITCH started crashing on every fax call.

Rolling back to 3.0.0-12951482879-0a9a1bd2da~bookworm fixed the issue.

I've tested both rxfax and txfax and get a crash with either.

<extension name="Fax_Rx">
  <condition field="destination_number" expression="^(100)$">
    <action application="answer"/>
    <action application="rxfax" data="/tmp/test.tif"/>
    <action application="hangup"/>
  </condition>
</extension>

<extension name="Fax_Tx">
  <condition field="destination_number" expression="^(101)$">
    <action application="answer"/>
    <action application="txfax" data="/tmp/test.tif"/>
    <action application="hangup"/>
  </condition>
</extension>

In the commit you linked to, some of the header files have been changed.

As FreeSWITCH is compiled against the old version library, it's likely allocating incorrect struct sizes, which are then getting clobbered.

[andywolk]

@coppice-git We are seeing issues with using the latest SpanDSP code.
To be precise
Working Debian package 3.0.0-12951482879-0a9a1bd2da~bookworm is based on an outdated SpanDSP branch (last code change is from Jun 17, 2023):
https://github.com/freeswitch/spandsp/tree/packages

While broken Debian package 3.0.0-16125392215-7977601689~bookworm is based on the latest SpanDSP master branch:
https://github.com/freeswitch/spandsp/tree/master

[andywolk]

Another example with a backtrace attached:

• Segfault with libspandsp3:3.0.0-16125392215-7977601689~bookworm signalwire/freeswitch#2859
• https://github.com/user-attachments/files/21253833/gdb.txt

[SGaitskell]

@andywolk With ClueCon around the corner, I assume a new FreeSWITCH release is imminent which will fix the issue?

[andywolk]

@andywolk With ClueCon around the corner, I assume a new FreeSWITCH release is imminent which will fix the issue?

Downgrading spandsp package fixes the issue. Going to remove the broken one from our repos.

[andywolk]

Broken spandsp debian packages have been removed from our repos.

[andywolk]

@coppice-git We need to version bump SpanDSP and ship the latest in a new package.

[andywolk]

Closing this issue as there are no changes to the code required so far.

[andywolk]

• Bump version to 3.1.1, bump ABI to 4.0.0 #126