Can DLT be used in safe systrems?

[jhnlmn]

Hi,
Embedded, real time, safe and automotive SW is not allowed to call malloc, mutex and other non-deterministic APIs at run time.
I tried to call DLT_LOG in a loop and see that it calls calloc() and free() for every DLT_LOG call.
I see language in dlt_for_developers.md under "Send log message with given buffer" that dynamic allocation can be avoided
by using dlt_user_log_write_start_w_given_buffer.
But I find very little documentation about how to use dlt_user_log_write_start_w_given_buffer and other DLT APIs safely.
I see that dlt_user_log_write_start_w_given_buffer calls dlt_user_log_init which may call dlt_init, which calls dlt_init_common,
which calls many unsafe functions like malloc, getenv, mutex creation and others.
So, in its current form it cannot be used in safe SW.

I wonder whether it may be possible to perform some minimal refactoring of DLT,
may be place some code (such as dlt_init) under #ifndef DLT_SAFE (like some vendors are doing these days)
and then document clearly which functions can be called at run time and which only during initialization (again this is a common practice now).

Thank you

[minminlittleshrimp]

Hi @jhnlmn
Yes, thanks.
Finally someone has to mention about a safe, real time log framework for automotive, at least ASIL B and ISO26262 😀

To me, dlt-daemon has reach it time, and hard to say a huge refactor works here.

Fortunately, at Bosch we are developing and working on the same philosophy: a safe realtime dlt.

I cannot tell further, and cannot promise anything now, but when the prototype done, it shall be made open source upstream, so that code can be reviewed under more eyes.

Thanks and hope to see any collab from you in the future

[jhnlmn]

I see that the biggest problem with current implementation is not only calloc (which can be avoided), but mutex in dlt_user_log_send_log.
afaik it is better to avoid mutexes in safe deterministic code.
That's why many loggers use lock free queues.
But lock free queues may not be very reliable - log messages may be lost or even corrupted.
In other words, if we continue using mutex, then a (hypothetical) plane may crash intermittently, but, at least, every log is preserved.
If we switch to lock free queue, then a log message may be lost/corrupted intermittently, but, at least, the plane will be safe.

Question is whether DLT protocol requires some kind of delivery guarantee for log messages or is some intermittent loss/corruption
may be acceptable?
I see there is a requirement for Transmission Retry.
But another requirement is to have Message Counter to identify lost messages.

So, can mutex be replaced with lock-free mechanism in dlt_user_log_send_log?

[SebastianZieglerSZ]

Hi @jhnlmn and @minminlittleshrimp,

I only know of commercial DLT (Log & Trace) implementations, which you could find online, that reach ASIL-D according to ISO26262, but even the COVESA DLT implementation would adopt to not use exceptions and other unsafe APIs, i would not know
how to guarantee that this software has also been developed according to the process defined by ISO26262 and A-Spice.

fyi: Eclipse SCORE is currently trying to do exactly that (https://eclipse-score.github.io/score/main/features/baselibs/index.html).

Kind regards,
Sebastian

[minminlittleshrimp]

@SebastianZieglerSZ
Thanks for the info, this is also new to me. Usually I only work with COVESA DLT under QM, no ASIL or ISO here.
Afaik, "commercial DLT (Log & Trace) implementations" for ASIL B and ISO26262 is in fact AUTOSAR DLT.
DLT-Daemon "not use exceptions and other unsafe APIs" this never be defined as first stage of the design, honestly I believe.
Let see how far our team can go with this approach, ISO26262 and A-Spice -> Team has to check this since the implementation totally based on AUTOSAR Spec.

[minminlittleshrimp]

I see that the biggest problem with current implementation is not only calloc (which can be avoided), but mutex in dlt_user_log_send_log. afaik it is better to avoid mutexes in safe deterministic code. That's why many loggers use lock free queues. But lock free queues may not be very reliable - log messages may be lost or even corrupted. In other words, if we continue using mutex, then a (hypothetical) plane may crash intermittently, but, at least, every log is preserved. If we switch to lock free queue, then a log message may be lost/corrupted intermittently, but, at least, the plane will be safe.

Question is whether DLT protocol requires some kind of delivery guarantee for log messages or is some intermittent loss/corruption may be acceptable? I see there is a requirement for Transmission Retry. But another requirement is to have Message Counter to identify lost messages.

So, can mutex be replaced with lock-free mechanism in dlt_user_log_send_log?

@jhnlmn
All the time, lock-free or lock-based under discussion. E.g. check libltt-ng lock-free - RCU oriented per-ring buffer approach, look back spdlog - lock-based.
So which approach the best? Honestly I am not sure, but yes, for ASIL-B and ISO26262 then our team can plan for this and we are working on that. Cannot tell further now.
Thanks

[SebastianZieglerSZ]

Hi @minminlittleshrimp,
thanks for your feedback! I am not exactly sure what you mean with "commercial DLT (Log & Trace) implementations" for ASIL B and ISO26262 is in fact AUTOSAR DLT"?
In my opinion the DLT protocol in version 1 is the same for COVESA and AUTOSAR. You are right if you mean that AUTOSAR and COVESA have a different API but generally speaking to me there is no further difference. Even AUTOSAR DLT can be used without ARXML for example.
Kind regards,
Sebastian