ESP32 Web Sockets - Transferring Raw Image Data

hi again folks it's been a while since my last video I haven't been able to get much tinkering done lately but I wanted to share some progress on a small project I'm working on a stereoscopic camera using two esp32 camboards I've got a breakout board that will hold two esp32 camera modules I actually shared how I made this using a PCB strip board and electric engraver in a previous video sadly I haven't been able to complete the entire project but I have managed to at least do an image capture from One camera using web sockets with the esp32 board set up as a server and my PC running python as the client so I thought I'd share the C code for the esp32 board along with the python code for the client and demonstrate how it all works as usual I will post a link in the description that takes you to the code first to the esp32 code in the Arduino IDE before getting to the sockets portion I'll go quickly through the standard stuff first this is readily available in numerous examples and even covered in one of my videos discussing the esp32 camboard we have our standard setup for the esp32 Wi-Fi connectivity and esp32 Camera remember to set up the Wi-Fi network name and password onto the setup function we configure our serial connection there are some camera configuration settings to take care of as well I will note that I am configuring the camera to capture grayscale images with a resolution of qvga that's 320 by 240. I'm interested in doing some computer vision work and this should suit my needs just fine we have some standard camera initialization code here and we have the Wi-Fi initialization code and then we initiate our websocket server at the end of the setup function and now I'll cover the the websocket setup itself first of course we have the websocket server header file to include I have instructions at the top of the code listing that explains how to make sure you have the library properly installed next we declare the websockets server websocket here and now we Define the Handler function for when we receive a websocket message there are various parameters associated with an incoming websocket message including the message type and payload depending on the message type received we can take different actions for when we have a websocket client connected we'll output a message to the serial monitor and likewise we'll send a message to the monitor when the client has disconnected now the next type is the most interesting in this example when we receive a text type message from the client we looked at the contents of the payload that's the the text data in this case now if the string sent from the client is the word capture it will trigger the camera to capture a frame now the next few lines are sort of strange because of the way the esp32 camera module captures images we need to implement a workaround to make sure we return the latest picture grabbed in the frame buffer otherwise you'll end up getting an old captured image essentially we're grabbing an image and releasing the the frame from the buffer before grabbing another image that will actually use in order to make sure we've got the latest image and once we've captured the image in the frame buffer we return the image data through the websocket using the send bin message type okay that's send binary message type in this function call we pass the frame buffer pointer FB point to buff along with the frame buffer size this will return all the bytes of a captured image to the websocket client in this case since we've set the resolution to 320 by 240 it'll return 76 800 bytes once the response has been sent we release the frame buffer if the client doesn't send a capture message then we just send back the client's payload contents in a text type response essentially just echoing what the what the client sent okay and finally in our Loop function we call websocket Dot Loop to wait for websocket data and that covers the esp32 camera code portion now to the python python code running on the PC we make use of the websocket client python library and import that again I've got instructions in the code header for exactly how to install it along with some good references for how to code websockets now I set up some constants for the image resolution and then set up the websocket connection itself in the ws.connect functioncall we use the esp32 cam IP address that is printed over the serial interface to the serial monitor when we first run the esp32 code that we went through this establishes the websocket connection and now we enter the while loop where the user enters text commands that will be sent over the websocket to the esp32 board we take an input string Str and pass it over the socket if the user entered the word capture then we have code that expects to receive a binary response as we discussed the binary data we get back is part of the response from the websocket server is stored in this variable called binresp now we know this is a grayscale image data from the esp32 camera and we declare an image array of the appropriate dimensions we go through each byte of the binary response data in that Loop and populate the image array once we've gone through all the data we display the image now keep in mind that in this example we focused on image data from the esp32 camera module but we could just as easily stream any data over the websocket it could be sensor readings or any sort of telemetry as as an example so let's quickly run a demonstration of the code you can see from the Arduino serial monitor that the esp32 board is running and we will first send a regular text message that we expect to be mirrored back to us foreign and now let's do some image captures okay you see it all works as expected and I hope this has been informative and you try this out for yourself if you've got an esp32 camera module to play with in a future video I'll provide an update on the complete stereoscopic camera setup once I've put it all together until next time have fun tinkering

Channel: The Tinker Foundry

Views: 3,344

Keywords:

Id: WndUbc3Rx5c

Length: 8min 29sec (509 seconds)

Published: Sun Mar 12 2023