Possible to send multiple images (stream) - e.g. webcam on ESP32CAM?

[HowardsPlayPen]

Background: my aim is to use an ESP32CAM as a webcam using the commonly found code on the web - but to also include ElegantOTA for simple OTA updates (which brought in ESPAsyncWebServer). Ideally I wanted to be able to stream the images - and handle more than one client (the commonly available code only handles one client at a time)

So this library looks great - better / more performant than ESPAsyncWebServer (and still supported)

Question: I can see the documentation say:
"You can not send more than one response to a single request."
which seems to categorically rule out my request(? Unless there is a distinction between multiple Response objects versus sending multiple blobs over a single connection?)

I could use WebSockets - so again this library looks great if that is the case.

Apologies if this is a pointless request or asked before (I could not find it and I believe I went through all Issues)

Summary:
Can I cache Response objects and send multiple blobs (e.g. call setContent / send multiple times?).

[zekageri]

I would also be interested in the streaming functionality

[hoeken]

@HowardsPlayPen. That documentation is referencing the webhandler which is just your basic web request with one response to one request. Other web protocols are definitely possible. I'm not entirely familiar with streaming image protocols though. Do you have any information on exactly what would be implemented?

Depending on the details, it would probably be best to implement this as a handler. For example the EventSource class is a handler that sends a response, but then keeps the connection open to stream events to connected clients. The way that is setup it can handle multiple clients easily.

Websockets are very well supported too. My app is entirely built around websockets and its the main reason for creating this library as the ESPAsync implementation is super buggy. If you handle the responses asynchronously you can do 150-200 msg/sec. I'm going to write a simplified example of how I do that with xqueues shortly.

[hoeken]

Websockets xQueue example: https://github.com/hoeken/PsychicHttp/tree/master/examples/websockets

[HowardsPlayPen]

I have now got a local version of ElegantOTA updated to have an extra option to support PsychicHttp and it is working (using the Response option you separately mentioned for adding gzip headers) - so I am now coming back to look at this question again.
My understanding (possibly wrong I admit) is that HTTP1.1 should just mean keeping the socket connection alive and send new "reply()" with further messages (e.g. images) - that is what the existing code for the ESP32-CAM seems to do anyway.

I will use the WebSockets going forward for all manner of uses - but for the webcam side your mention of implementing a new handler sounds useful. Could you give a few lines of description of how you would do it (I say this as I just read that one should not cache handlers but instead cache socket handles(?) - so I would appreciate a simple pseudo code example of how you would see a handler being retained for sending multiple responses?

Thanks for all your help so far

[hoeken]

@HowardsPlayPen if you have a link to a blog post or something describing what you'd like to do that would help me understand it better.

that being said, I would probably start with the EventSource stuff. It has a custom handler, client, and response to generate the specific EventSource information. You might not need all of that, since the Response generates its own HTTP header, the client has custom functions to send events, and the handler overrides the onOpen, onClose, and handleResponse to add its own custom hooks.

Hard to say exactly what would need to change without seeing something with more details on what you're trying to do. Can you send me that esp32-cam code you're looking to replicate?

[HowardsPlayPen]

The current cam code only responds to a single client at a time - so has a loop taking over the main thread. The HTTP header at the beginning is therefore telling the client to expect multiple parts and each new image has a Header showing it is a new part (but clearly this is obvious when you read the code so I do not mean to teach you to suck eggs..)

WiFiClient client = server.client();
String response = "HTTP/1.1 200 OK\r\n";
response += "Content-Type: multipart/x-mixed-replace; boundary=frame\r\n\r\n";
server.sendContent(response);

while (1)
  {
      cam.run();
      if (!client.connected())
          break;
      response = "--frame\r\n";
      response += "Content-Type: image/jpeg\r\n\r\n";
      server.sendContent(response);

      client.write((char *)cam.getfb(), cam.getSize());
      server.sendContent("\r\n");
      if (!client.connected())
          break;
  }

All I want to do is remove the above while(1) loop and merely push a new Handler / Request into a container to hold all active connections - then wait for the next cam image to be available, i.e.

cam.run()

Then use Handler.reply() for each of the active connections using the buffer

      Handler.reply( (char *)cam.getfb(), cam.getSize());

One example GitHub project is:

https://github.com/rzeldent/esp32cam-rtsp/blob/main/src/main.cpp

see line 232 handle_stream()

I will have a look at the EventSource code though as that might be show the obvious solution.

[hoeken]

Okay yep I see how it works and yeah you should definitely be able to modify the event source stuff to do this. Send that first set of headers in the handler and then modify the client to send the stuff in the while loop and you should be able to handle multiple clients no problem. Let me know if you run into any issues.

…

On Thu, Dec 28, 2023, 15:24 HowardsPlayPen ***@***.***> wrote: The current cam code only responds to a single client at a time - so has a loop taking over the main thread. The HTTP header at the beginning is therefore telling the client to expect multiple parts and each new image has a Header showing it is a new part (but clearly this is obvious when you read the code so I do not mean to teach you to suck eggs..) WiFiClient client = server.client(); String response = "HTTP/1.1 200 OK\r\n"; response += "Content-Type: multipart/x-mixed-replace; boundary=frame\r\n\r\n"; server.sendContent(response); while (1) { cam.run(); if (!client.connected()) break; response = "--frame\r\n"; response += "Content-Type: image/jpeg\r\n\r\n"; server.sendContent(response); client.write((char *)cam.getfb(), cam.getSize()); server.sendContent("\r\n"); if (!client.connected()) break; } All I want to do is remove the above while(1) loop and merely push a new Handler / Request into a container to hold all active connections - then wait for the next cam image to be available, i.e. cam.run() Then use Handler.reply() for each of the active connections using the buffer Handler.reply( (char *)cam.getfb(), cam.getSize()); One example GitHub project is: https://github.com/rzeldent/esp32cam-rtsp/blob/main/src/main.cpp see line 232 handle_stream() I will have a look at the EventSource code though as that might be show the obvious solution. — Reply to this email directly, view it on GitHub <#38 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEHSV3MAS6U56F4H7R5ETYLXIQ5AVCNFSM6AAAAABBBVWNLOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZRGQ3DQMRSGI> . You are receiving this because you commented.Message ID: ***@***.***>

[DaeMonSxy]

I would be more than happy to see some esp32cam - webserver codes in examples.
there are meny codes for non async, then few async-webserver codes- non of them seem to be stable to me and usefulll for daily use.

[06GitHub]

Has someone already implemented video webcam streaming with PsychicHttp ? If yes, could example be shared ?

This ESPAsyncWebServer code provided by me-no-dev https://gist.github.com/me-no-dev/d34fba51a8f059ac559bf62002e61aa3 works OK (for my project).

I would like to migrate this ESPAsyncWebServer code to PsychicHttp (if possible ?), or use another working solution for video webcam streaming, so maybe someone has already done the job ?

Thanks

[06GitHub]

For those interested, example below handles streaming video with PsychicHttp.
Limitation : handles one client (ie video stream is not sent to all clients connected to server, only to client which triggers the request).

Code, screenshots and output are provided.

CONFIGURATION
Arduino IDE 1.8.19
arduino-esp32 v2.0.17
ESP32-S3

CODE

/*********
  Camera video streaming with PsychicHttp server
  CREDITS
  https://github.com/hoeken/PsychicHttp/blob/master/src/PsychicResponse.cpp
  https://gist.github.com/me-no-dev/d34fba51a8f059ac559bf62002e61aa3
  https://RandomNerdTutorials.com/esp32-cam-video-streaming-web-server-camera-home-assistant/
*********/

char* TAG="CAM"; // ESP_LOG

#include "Arduino.h"
#include <WiFi.h>
#include "esp_camera.h"
// PsychicHttp https://github.com/hoeken/PsychicHttp
#include <ESPmDNS.h>
#include <PsychicHttp.h>
PsychicHttpServer server; // main server object http (not https)

//Replace with your network credentials
const char* ssid = "MYSSID";
const char* password = "MYPASSWORD";

// # camera module pinout (OV5640)
#define PWDN_GPIO_NUM    42
#define RESET_GPIO_NUM   -1
#define XCLK_GPIO_NUM    -1  // XLCL is not provided by ESP32, it is embedded on breakout board (crystal)
#define SIOD_GPIO_NUM    40
#define SIOC_GPIO_NUM    41
#define Y9_GPIO_NUM      46
#define Y8_GPIO_NUM      39
#define Y7_GPIO_NUM      2
#define Y6_GPIO_NUM      38
#define Y5_GPIO_NUM      6
#define Y4_GPIO_NUM      14
#define Y3_GPIO_NUM      48
#define Y2_GPIO_NUM      47
#define VSYNC_GPIO_NUM   4
#define HREF_GPIO_NUM    17
#define PCLK_GPIO_NUM    45

// content_type, header and boundary definition
#define PART_BOUNDARY "123456789000000000000987654321"
static const char* _STREAM_CONTENT_TYPE = "multipart/x-mixed-replace;boundary=" PART_BOUNDARY;
static const char* _STREAM_BOUNDARY = "\r\n--" PART_BOUNDARY "\r\n";
static const char* _STREAM_PART = "Content-Type: image/jpeg\r\nContent-Length: %u\r\n\r\n";

void startCameraServer(){
  server.listen(80); // start PsychicHttp server on http
  
  // endpoints
  server.on("/stream",[](PsychicRequest*req) {
    ESP_LOGI(TAG,"/stream");
    camera_fb_t * fb = NULL;
    esp_err_t res = ESP_OK;
    size_t _jpg_buf_len = 0;
    uint8_t * _jpg_buf = NULL;
    char * part_buf[64];
  
    PsychicResponse resp(req); // response
  
    // set content type   
    resp.setContentType(_STREAM_CONTENT_TYPE);
    ESP_LOGI(TAG,"_STREAM_CONTENT_TYPE %s",_STREAM_CONTENT_TYPE);
    
    while(true) { // infinite loop to capture/send frames
      // capture frame
      fb = esp_camera_fb_get();
      if (!fb) { // frame capture is KO
        ESP_LOGE(TAG,"ERROR : Camera capture failed");          
        return(ESP_FAIL);
      } // end frame capture is KO
      else { // frame capture is OK
        ESP_LOGI(TAG,"fb %p width %d fb->buf %p fb->len %d",fb, fb->width, fb->buf, fb->len);
        if (fb->format != PIXFORMAT_JPEG){ // convert frame to JPEG format if not JPEG
          bool jpeg_converted = frame2jpg(fb, 80, &_jpg_buf, &_jpg_buf_len);      
          if (!jpeg_converted) { // JPEG conversion KO
            ESP_LOGE(TAG,"ERROR : JPEG compression failed for streaming");
            esp_camera_fb_return(fb);                       
            fb = NULL;
            _jpg_buf_len = 0;
            _jpg_buf = NULL;
            return (ESP_FAIL);
          } // end JPEG conversion KO
        } // end convert frame to JPEG format if not JPEG
        else { // frame format is already JPEG
            _jpg_buf_len = fb->len;
            _jpg_buf = fb->buf;
        } // end frame format is already JPEG
      } // end frame capture is OK
      ESP_LOGI(TAG,"_jpg_buf %p _jpg_buf_len %d", _jpg_buf, _jpg_buf_len);
      // send header
      size_t hlen = snprintf((char *)part_buf, 64, _STREAM_PART, _jpg_buf_len);
      res = resp.sendChunk((uint8_t *)part_buf, hlen);
      if (res != ESP_OK) {
        ESP_LOGI(TAG,"header : part_buf %s hlen %d", part_buf, hlen);
        ESP_LOGE(TAG,"ERROR : result sending header is NOT ESP_OK %d",res);             
        return(res);
      }
      else {
        ESP_LOGI(TAG,"header : part_buf %s hlen %d", part_buf, hlen);          
        ESP_LOGI(TAG, "result sending header ESP_OK");
      }
      // send frame
      res = resp.sendChunk((uint8_t *)_jpg_buf, _jpg_buf_len);
      if (res != ESP_OK) {     
        ESP_LOGE(TAG,"ERROR : result sending frame is NOT ESP_OK %d",res);          
        return(res);
      }
      else {
        ESP_LOGI(TAG,"result sending frame is ESP_OK");
      }
      // send boundary
      res = resp.sendChunk((uint8_t *)_STREAM_BOUNDARY, strlen(_STREAM_BOUNDARY)); 
      if (res != ESP_OK) {
        ESP_LOGE(TAG,"ERROR : result sending boundary is NOT ESP_OK %d %s",res, _STREAM_BOUNDARY);          
        return(res);
      }
      else {
        ESP_LOGI(TAG,"boundary _STREAM_BOUNDARY %s",_STREAM_BOUNDARY);         
        ESP_LOGI(TAG, "result sending boundary ESP_OK");
      }        
     // release memory
      if(fb->format != PIXFORMAT_JPEG){
          free(_jpg_buf);
      }         
      if(fb){
        esp_camera_fb_return(fb);
        fb = NULL;
        _jpg_buf = NULL;          
      } else if(_jpg_buf){
        free(_jpg_buf);
        _jpg_buf = NULL;
      }
    } // end while true
    return (ESP_OK);
  }); // end /stream

  server.on("/", HTTP_GET, [](PsychicRequest *request) {
    String output = "Your IP is <b>" + request->client()->remoteIP().toString()+ "</b>.<p>Click "+"<a href=http://"+WiFi.localIP().toString()+"/stream> http://"+WiFi.localIP().toString()+"</a> to stream video.</p>";
    return request->reply(output.c_str());
  });
  
} // end startCameraServer

void setup() {
  delay(2000);  // give time to display server ip address on serial
 
  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG;
  //config.pixel_format = PIXFORMAT_GRAYSCALE; // for gray image  
  config.frame_size = FRAMESIZE_VGA;  
  config.jpeg_quality = 12;
  config.fb_count = 1;
  
  // Camera init
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    ESP_LOGE(TAG,"ERROR : Camera init failed with error 0x%x", err);
    return;
  }
  
  // Wi-Fi connection
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    ESP_LOGI(TAG,".");
  }
  ESP_LOGI(TAG,"WiFi connected");  
  ESP_LOGI(TAG,"Camera Stream Ready! Go to: http://");
  ESP_LOGI(TAG,"IP address %s",WiFi.localIP().toString().c_str());
  
  // Start camera server
  startCameraServer();
}

void loop() {
  delay(1);
}

SCREENSHOTS

OUTPUT

[   203][I][esp32-hal-psram.c:96] psramInit(): PSRAM enabled
[  3552][I][psychic_videostream_OK.ino:185] setup(): [CAM] .
[  4058][I][psychic_videostream_OK.ino:185] setup(): [CAM] .
[  4064][I][psychic_videostream_OK.ino:187] setup(): [CAM] WiFi connected
[  4070][I][psychic_videostream_OK.ino:188] setup(): [CAM] Camera Stream Ready! Go to: http://
[  4079][I][psychic_videostream_OK.ino:189] setup(): [CAM] IP address 192.168.1.32
[ 19723][I][PsychicHttpServer.cpp:236] openCallback(): [psychic] New client connected 51
[ 19733][I][PsychicClient.cpp:68] remoteIP(): [psychic] Client Remote IP => 192.168.1.25
[ 19743][I][PsychicHttpServer.cpp:236] openCallback(): [psychic] New client connected 52
[ 19838][I][PsychicHttpServer.cpp:262] closeCallback(): [psychic] Client disconnected 51
[ 22178][I][psychic_videostream_OK.ino:52] operator()(): [CAM] /stream
[ 22185][I][psychic_videostream_OK.ino:63] operator()(): [CAM] _STREAM_CONTENT_TYPE multipart/x-mixed-replace;boundary=123456789000000000000987654321
[ 22198][I][psychic_videostream_OK.ino:73] operator()(): [CAM] fb 0x3d8009c0 width 640 fb->buf 0x3d800a10 fb->len 8148
[ 22209][I][psychic_videostream_OK.ino:90] operator()(): [CAM] _jpg_buf 0x3d800a10 _jpg_buf_len 8148
[ 22219][I][psychic_videostream_OK.ino:100] operator()(): [CAM] header : part_buf Content-Type: image/jpeg
Content-Length: 8148

 hlen 50
[ 22232][I][psychic_videostream_OK.ino:101] operator()(): [CAM] result sending header ESP_OK
[ 22252][I][psychic_videostream_OK.ino:110] operator()(): [CAM] result sending frame is ESP_OK
[ 22262][I][psychic_videostream_OK.ino:119] operator()(): [CAM] boundary _STREAM_BOUNDARY 
--123456789000000000000987654321

[ 22273][I][psychic_videostream_OK.ino:120] operator()(): [CAM] result sending boundary ESP_OK
[ 22348][I][psychic_videostream_OK.ino:73] operator()(): [CAM] fb 0x3d8009c0 width 640 fb->buf 0x3d800a10 fb->len 8442
[ 22359][I][psychic_videostream_OK.ino:90] operator()(): [CAM] _jpg_buf 0x3d800a10 _jpg_buf_len 8442
[ 22368][I][psychic_videostream_OK.ino:100] operator()(): [CAM] header : part_buf Content-Type: image/jpeg
Content-Length: 8442

 hlen 50
[ 22380][I][psychic_videostream_OK.ino:101] operator()(): [CAM] result sending header ESP_OK
[ 22395][I][psychic_videostream_OK.ino:110] operator()(): [CAM] result sending frame is ESP_OK
[ 22404][I][psychic_videostream_OK.ino:119] operator()(): [CAM] boundary _STREAM_BOUNDARY 
--123456789000000000000987654321

[ 22416][I][psychic_videostream_OK.ino:120] operator()(): [CAM] result sending boundary ESP_OK
[ 22500][I][psychic_videostream_OK.ino:73] operator()(): [CAM] fb 0x3d8009c0 width 640 fb->buf 0x3d800a10 fb->len 8464
[ 22510][I][psychic_videostream_OK.ino:90] operator()(): [CAM] _jpg_buf 0x3d800a10 _jpg_buf_len 8464
[ 22521][I][psychic_videostream_OK.ino:100] operator()(): [CAM] header : part_buf Content-Type: image/jpeg
Content-Length: 8464

 hlen 50
[ 22533][I][psychic_videostream_OK.ino:101] operator()(): [CAM] result sending header ESP_OK
[ 22549][I][psychic_videostream_OK.ino:110] operator()(): [CAM] result sending frame is ESP_OK
[ 22558][I][psychic_videostream_OK.ino:119] operator()(): [CAM] boundary _STREAM_BOUNDARY 
--123456789000000000000987654321

[ 22569][I][psychic_videostream_OK.ino:120] operator()(): [CAM] result sending boundary ESP_OK
[ 22652][I][psychic_videostream_OK.ino:73] operator()(): [CAM] fb 0x3d8009c0 width 640 fb->buf 0x3d800a10 fb->len 8513
[ 22662][I][psychic_videostream_OK.ino:90] operator()(): [CAM] _jpg_buf 0x3d800a10 _jpg_buf_len 8513
[ 22671][I][psychic_videostream_OK.ino:100] operator()(): [CAM] header : part_buf Content-Type: image/jpeg
Content-Length: 8513

 hlen 50
[ 22684][I][psychic_videostream_OK.ino:101] operator()(): [CAM] result sending header ESP_OK
[ 22700][I][psychic_videostream_OK.ino:110] operator()(): [CAM] result sending frame is ESP_OK
[ 22709][I][psychic_videostream_OK.ino:119] operator()(): [CAM] boundary _STREAM_BOUNDARY 
--123456789000000000000987654321

[ 22720][I][psychic_videostream_OK.ino:120] operator()(): [CAM] result sending boundary ESP_OK
[snip]

[hitecSmartHome]

How does this not trigger the watchdog?