Issue
Since views appear at different stages of the pipeline, each has its own format (e.g., ushort, float, complex, RGB, etc.). This leads to several issues:
- The front end needs to know the exact format and adapt the texture format accordingly, forcing us to rely on low-level graphics libraries to handle each format appropriately.
- It is not user-friendly and lacks proper documentation.
- The format changes depending on the image type (e.g.,
RGB for composite) and the compute mode.
Additionally, the GUI directly accesses queues to retrieve the latest image, resulting in several problems:
- The syntax for retrieving images is overly complex (e.g.,
API.get_compute_pipe()->get_stft_slice_queue(1).get()->get_last_image() to get the YZ view image).
- Queues and internal structures are exposed to the front end.
- There are no security or memory checks. If a queue or pipeline is uninitialized when accessing an image, the program will crash.
Proposed Solution
To address these issues, we could create wrapper functions in ViewApi to ensure the structures are initialized. If the structure is initialized, the wrapper would return the frame converted to R8G8B8 mode (each channel is on one byte); otherwise, it would return nullptr.
The conversion should be done in this function instead of directly in the pipe because they will be called only 20 to 30 times per second so it's not noticeable. While directly in the pipe will greatly impact performance.
An example implementation might look like this:
void* get_hologram_last_frame()
{
if (!api_->compute.get_input_queue())
return nullptr;
auto* last_frame = api_->compute.get_input_queue().get()->get_last_image();
// Generic wrapper that converts to RGB based on the frame_descriptor.depth
auto* converted = convert_to_rgb(last_frame, api_->compute.get_input_queue().get()->get_fd());
return converted;
}
To further abstract, we could use an enum:
enum class View
{
Raw = 1,
Processed,
Lens,
SliceXZ,
SliceYZ,
Filter2D
};
void* get_view_last_frame(View kind);
camera::FrameDescriptor get_view_fd(View kind); // Still used to retrieve width and height
Issue
Since views appear at different stages of the pipeline, each has its own format (e.g.,
ushort,float,complex,RGB, etc.). This leads to several issues:RGBfor composite) and the compute mode.Additionally, the GUI directly accesses queues to retrieve the latest image, resulting in several problems:
API.get_compute_pipe()->get_stft_slice_queue(1).get()->get_last_image()to get the YZ view image).Proposed Solution
To address these issues, we could create wrapper functions in
ViewApito ensure the structures are initialized. If the structure is initialized, the wrapper would return the frame converted to R8G8B8 mode (each channel is on one byte); otherwise, it would returnnullptr.The conversion should be done in this function instead of directly in the pipe because they will be called only 20 to 30 times per second so it's not noticeable. While directly in the pipe will greatly impact performance.
An example implementation might look like this:
To further abstract, we could use an enum: