plain music streaming example

[gmorell]

I've been fighting with this for the past few days and figured I'd ask. I was able to run the simple pipeline example where you can play a track through the speakers without any trouble. I wanted to adapt this to instead produce an hls stream. I've tried a number of differing approaches but I'm stymied by something each time and was wondering if there was something obvious I was missing.

I've been toying with a few of the approaches in membrane_demo and other repos and the closest approach I've had to working without any error so far is:

child(:file, %Membrane.File.Source{location: path_to_file})
|> child(:decoder, Membrane.MP3.MAD.Decoder)
|> child(:converter, %Converter{
  input_stream_format: %RawAudio{channels: 2, sample_format: :s24le, sample_rate: 44_100},
  output_stream_format: %RawAudio{channels: 2, sample_format: :s16le,  sample_rate: 44_100}
})
|> child(:encoder_aac, Membrane.AAC.FDK.Encoder)
|> via_in(:input,
     options: [
       encoding: :AAC,
       segment_duration: Membrane.Time.seconds(2)
     ]
   )
|> child(:sink, %Membrane.HTTPAdaptiveStream.SinkBin{
  manifest_module: Membrane.HTTPAdaptiveStream.HLS,
  target_window_duration: 5 |> Membrane.Time.seconds(),
  persist?: false,
  storage: %Player.Storage{file: "stream-#{file_id}"}
})

But it seems that instead of streaming the file it just gets encoded and thrown on the ground?

The other HLS approach, as per the internal documentation should work to produce a solely audio stream.

child(:file, %Membrane.File.Source{location: path_to_file})
|> child(:decoder, Membrane.MP3.MAD.Decoder)
|> child(:converter, %Converter{
  input_stream_format: %RawAudio{channels: 2, sample_format: :s24le, sample_rate: 44_100},
  output_stream_format: %RawAudio{sample_format: :s16le, channels: 2, sample_rate: 48_000}
})
|> child(:encoder_opus, Membrane.Opus.Encoder)
|> child(:muxer, %Membrane.MP4.Muxer.CMAF{
  segment_min_duration: Time.seconds(4)
})
|> via_in(:input,
  options: [
    segment_duration: Membrane.Time.seconds(2)
  ]
)
|> child(:hls, %Membrane.HTTPAdaptiveStream.Sink{
  manifest_config: %Membrane.HTTPAdaptiveStream.Sink.ManifestConfig{
    name: "manifest",
    module: Membrane.HTTPAdaptiveStream.HLS
  },
  track_config: %Membrane.HTTPAdaptiveStream.Sink.TrackConfig{
    target_window_duration: 5 |> Membrane.Time.seconds(),
  },
  storage: %Player.Storage{file: "stream-#{file_id}"}
})

This produces an error,

[error] <0.899.0>/:muxer Error occured in Membrane Element:
** (ArgumentError) argument error
    (stdlib 5.2.3.1) :maps.from_list([nil])
    (elixir 1.17.3) lib/map.ex:263: Map.new_from_enum/2
    (membrane_mp4_plugin 0.35.2) lib/membrane_mp4/muxer/cmaf.ex:526: Membrane.MP4.Muxer.CMAF.generate_output_stream_format/2
    (membrane_mp4_plugin 0.35.2) lib/membrane_mp4/muxer/cmaf.ex:312: Membrane.MP4.Muxer.CMAF.handle_stream_format/4
    (membrane_core 1.1.2) lib/membrane/core/callback_handler.ex:139: Membrane.Core.CallbackHandler.exec_callback/4
    (membrane_core 1.1.2) lib/membrane/core/callback_handler.ex:69: Membrane.Core.CallbackHandler.exec_and_handle_callback/5
    (membrane_core 1.1.2) lib/membrane/core/element/stream_format_controller.ex:82: Membrane.Core.Element.StreamFormatController.exec_handle_stream_format/4
    (membrane_core 1.1.2) lib/membrane/core/element.ex:237: Membrane.Core.Element.handle_info/2
    (stdlib 5.2.3.1) gen_server.erl:1095: :gen_server.try_handle_info/3
    (stdlib 5.2.3.1) gen_server.erl:1183: :gen_server.handle_msg/6
    (stdlib 5.2.3.1) proc_lib.erl:241: :proc_lib.init_p_do_apply/3

that after digging through the original code, implies that the HLS setup seems to think that it needs a video stream in the mix.

[mat-hek]

Hi @gmorell, if you'd like to stream in real-time from a source that's faster than real-time, like the file, you should plug realtimer. Since it relies on timestamps, it should be after the decoder ;)

[gmorell]

Thanks for replying, had somehow missed that plugin.

When adding it after the decoder, I'm getting an error where the resulting buffers are nil upon the call get_dts_or_pts() inside the plugin, but don't know enough about the membrane internals to actually resolve that particular aspect.

Tried it with different MP3 files as well, just to be sure

[mat-hek]

Yeah, forgot that mp3s don't have timestamps in them. You'll need https://hexdocs.pm/membrane_raw_audio_parser_plugin/Membrane.RawAudioParser.html with overwrite_pts? option set to true