Raspberry Pi Autofocus Camera & libcamera

today in the workshop we'll take a look at an autofocus camera for the raspberry pi we'll see how to use this camera to capture pictures and video and we'll test the quality and capabilities of this inexpensive device we'll also use lib camera to stream video to a linux workstation it's coming into focus today so welcome to the workshop [Music] [Applause] hello and welcome to the workshop and today we are going to be looking at yet another raspberry pi camera this one though has a feature that we haven't seen in any previous pie cameras and that is the ability to auto focus and of course that could open up a number of interesting applications for us now another thing about this camera is that it has the same resolution as the raspberry pi high quality camera yet it is the same form factor and the same price as the raspberry pi version 2 camera so it's definitely a piece of hardware we're going to want to examine a bit further now before we get started i want to let you know that this camera is just going on sale right now it's been on a kickstarter campaign since november and arducam who are the manufacturers of this camera were giving out samples of the camera and i received one of the samples so this is an early version of the camera when you're watching this later there may be some changes to it but i believe this is actually what you're going to get in the production version and so before we begin let's take a look at what it takes to add autofocus to a raspberry pi camera the arducam autofocus camera is a 16 megapixel raspberry pi compatible camera it has an identical form factor to the standard raspberry pi version 1 and version 2 cameras so it can be mounted anywhere that these cameras can be used this camera uses a sony imx519 image sensor it has a resolution of 4656 by 3496 pixels which is identical to the resolution of the raspberry pi high quality camera it's capable of 1080p 30 frames per second and 720p 60 frames per second video the camera also includes a case with a quarter inch thread for mounting on a standard photographic tripod as with all other raspberry pi cameras this camera connects to the csi camera port which is available on all modern raspberry pi's autofocus camera system requires three main components an image sensor a focus motor to move the lens of the image sensor and a control system to control the movement of the focus motor when building a camera for the raspberry pi one needs to determine how they can control the focus motor without any additional connections in order to figure this out let's take a look at the only connection we have to the camera which is the csi camera cable this is a 15-pin ribbon cable that has the following pin outs as it turns out there are a couple of pins that we could use to control our focus motor the i2c interface on the csi cable is normally disabled as most cameras don't make use of it this is an independent i2c bus from the i2c buses that are included on the raspberry pi gpio this interface can be enabled within the config.txt file which is in the boot directory of your raspberry pi operating system in our application we can use this i2c bus to control the position of the focus motor we could use this capability to build a software controlled focus application allowing us to focus the camera without actually physically touching the lens now to add autofocus capability we can choose through several different auto focus methods that are commonly used with modern cameras the phase detection method is commonly used on dslr and other advanced cameras it breaks the image into multiple images and then compares the phase differences between them many modern cameras also use the assist lap method this uses an infrared led or laser to measure the distance to the subject and determine the lens position another autofocus method is contrast detection this detects the differences between individual pixels and maximizes these differences in order to focus the object the raspberry pi autofocus camera uses the contrast detection method as it does not require any additional sensors or dividing of the image the opencv library which is an open source computer vision library is used to implement the contrast detection in the raspberry pi autofocus camera so now that we know a little bit more about the autofocus method let's go and take a look at our raspberry pi autofocus camera now here's the arducam autofocus camera and i've got it along with a couple of other raspberry pi cameras the original camera the 5 megapixel one and the 8 megapixel version 2 camera and i just brought these other cameras here so you could see a size comparison and as it is this camera is identical in size it'll mount in the same mounting holes as the original raspberry pi cameras so if you have an existing holder for your camera you can substitute this one it also comes with this plastic mount that it's attached to right now and if you look at the back of the mount they've got a very nice feature they've got a quarter inch thread over here so that you could mount it onto a tripod now one thing i found about this thread however is it doesn't go that deep and so the tripods that i've been using i had to insert a washer between the tripod and the camera mount in order to get the thing to mount securely but otherwise it's a very nice feature it also comes with a plastic cover that you can snap onto it like this and with this cover it weatherproofs the camera very very nicely so if you're using this for an outdoor application that's a really handy thing otherwise the only thing that's packaged with the camera is this instruction manual over here and it's not much of an instruction manual it shows you how to plug the camera in and then it sends you off to a pdf document and a website that you can get some more information about the camera on but there you go the raspberry pi autofocus camera from arducam now in our previous work with raspberry pi cameras we made use of a camera stack called raspi still raspi still allows you to use the command line on the raspberry pi and it lets you do things like take still pictures and use it for video and also manipulate a number of the camera parameters well raspi still has now been replaced with another open source camera stack called lib camera now lib camera can do everything that raspi still did plus it has a number of additional features and improvements so before we start working with our auto focus camera let's take a quick look at some of the features of lib camera lib camera is an open source camera stack it runs on linux android and on chrome os lib camera has been designed to control embedded camera hardware the hardware manufacturer will provide a custom pipeline handler in this case this has been provided by raspberry pi lip camera supports all of the raspberry pi camera modules you can also add custom camera modules to lib camera as arducam has for their autofocus camera raspberry pi has provided a number of lip camera apps these apps emulate the legacy camera stack on the raspberry pi specifically they emulate raspi's still and raspi vid allowing you to use your older commands with lib camera these can be extended as the source code for lib camera apps is open source and is completely available here are the lib camera apps provided by raspberry pi lip camera hello gives you a preview of the camera module lip camera jpeg is for capturing still images the lip camera still has all the features of raspi still while lip camera vid has all of the features of raspi vid lib camera raw gets raw frames from the image sensor for advanced processing and you can use lip camera detect with tensorflow to capture a jpeg when a specified object is detected notice that this last one requires a build so now let's put lip camera to use with the raspberry pi auto focus camera now before we start using our auto focus camera we're going to need to install a bit of software onto our raspberry pi this camera is part of our ducams pi variety series and they have an installation program that also installs a number of apps that we can make use of in addition we're also going to need to install a driver to work with this camera so let's go and fire up a raspberry pi with the latest version of the raspberry pi operating system open up a terminal and install the software we need for our autofocus camera now unlike other raspberry pi cameras which are essentially plug and play you're going to need to install some software in order to get the arducam autofocus camera working and the first thing you're going to need to do is to install some shell scripts from arducam's pi variety packages so i've got the command over here to do that to get them from github i'm going to hit enter right now and install these and they've been saved so it's a pretty quick installation now you need to give permission to the installation script in order to allow it to execute and we do that with a chmod command now after we've done the permissions it's standard procedure to do an update on your raspberry pi okay now we can start installing the actual packages themselves the first one we want to do is install the lib camera dash dev package and now we need to install some modified version of the lid camera apps package the easiest way to type that in is to use your arrow key to get the last command and change the underscore dev to underscore apps next we'll need to install a kernel driver for the camera again i'm going to use my up arrow to get the last command i have to go back a little bit more though this time and it's the imx 519 kernel driver and after doing this part of the installation you're going to want to hit yes to reboot your raspberry pi and we're almost done now the final step in installing the rdcam autofocus camera software is to install an application that allows you to do a manual focus and i'll be demonstrating that application in a little bit so let's hit the enter key right now and do that and we're done and we've completed the installation of the arducam autofocus camera software on the raspberry pi okay i've got my raspberry pi auto focus camera hooked to the raspberry pi and it is aimed right now at the back of my workbench and we've got the command line open and we're going to start using lib camera to take a few still pictures or to display a few still pictures as well now the very first thing you can do with lib camera just to make certain that everything is working is just do the lib camera hello so let's type that in and i'll hit enter and as you can see i've got an image and it's a live image of what's on my workbench and it goes up for a few seconds and it comes back down now you'll probably notice that image was fairly small and that's not lib camera's fault the fact is that i'm using a 4k video display and so it appears pretty small and it's probably hard for you to see but we can do something about that i'll do lib camera slash hello and i'm going to add one more parameter to it i'm going to add a dash f and this should make it go full screen so let's try that again and there we are a full screen of the back of my workbench okay now hello is this basically a demo program and you can use some of the other lib camera applications in order to do things like save jpegs etc so let's try this right now with libcam jpeg now for this we need to add a couple of parameters we need to tell it where we want our jpeg file to be saved so we do a dash o and that means output and then the name of the files i'm going to call this workbench and so that should save a file called workbench.jpg so let's just enter that and it's captured the still image so we can go and take a look at that image now if i open up my file manager and here's workbench.jpg and there we go there's a lovely picture of the back of my workbench so we can use workbench.jpg in order to do that now another way that we could have done that is with lib camera slash still and that is the equivalent of raspi still the commands that were valid on raspi still are all valid on lib camera slash still so let's try that and let's just do a help right now just to see what kind of parameters we have over here and so as you can see there's a huge list of parameters that we have that we can add to lib camera slash still and there's all sorts of parameters we can specify the width we can specify timeouts etc we can specify exposures there's a lot of things that we can do over here so let's do and slash t parameter is a timeout so how long is this going to stay on the screen it's in milliseconds so this should produce our image for four seconds now if you want to make the image stay up there permanently you can change that parameter to zero and now the image is up there basically forever i can use that to position my camera if i wanted to and it's not going to go away until i close the window over here and you can also combine lib cameras live still with an output parameter so let's just do a t 2000 maybe for two seconds and then a slash o and we'll make it and we'll also put our slash f in here to make this go full screen so this command should display in full screen for two seconds and then take a picture called test2.jpg so here we are full screen and it's still image captured we can go back into our file manager and there's test2.jpg and there you go so by using lib camera it is quite easy to grab images from the raspberry pi autofocus camera or indeed from any raspberry pi camera now in this demonstration i'm going to show you how you can manually focus the camera now you'll notice i've already opened up a camera window and it's live right now and it is a bit out of focus as we can see as well so i've opened the second terminal window and in here i'm going to change to a directory an arducam pivotary v4l2 driver slash focus directory on the raspberry pi so i'm going to enter that first and now i'm in that directory and i'm going to run a python 3 script called focuser example and it's got a couple of parameters after it as well and so i'm going to enter that right now and it brings me up a window where i can use a key press and now i can use my up and down arrow so i'm going to move the up arrow and if you watch the image i can bring it into focus bring it out of focus because i've gone too high and so i can go up and down and i can manually focus my image and so this is a way that you can do a manual focus on the rgcam autofocus camera now of course the big feature of this camera is the fact that it's capable of focus and it's very easy to add that to lib camera still all we need to do is do the dash dash auto focus parameter and it will do an auto focus when it first starts again i've added the dash f parameter so that it can display on full screen and so what i'm going to do now is hit enter over here and you'll notice it's focused right on the rdcam manual that i'm holding in front of the camera and that my background is now out of focus and so we can close that and we'll try it again just with the background and now it's focused on to my background so autofocus works when it first comes on however there's no way at the moment to auto focus it after it has come on however you can do that if you add another parameter and so let's bring that back up and add one more parameter i added the dash dash key press now what that will do is it will allow me to use my terminal to enter a letter f you have to put an f in and then hit enter and it can refocus the camera so let's just enter this right now and once again you can see it's focused on my background now we need to get the terminal here and of course with this being in the full screen mode it's a little bit awkward but i'll put it down over here i'll get my cursor down here i'm going to put a letter f in and i'll hold this in front of the camera and we'll press enter and now as you can see it focused on it and now my background is out of focus so i can put another letter f in there and press enter and now it is focused on the background so by putting that dash key press parameter in there i'm able to use the keyboard to refocus the camera now the auto focus drives a motor which moves the camera lens in and out and if you observe the lens while you're doing an autofocus you can actually see this operation so right now i've got it at minimum focal and i'm going to move it up to maximum and you can see the camera lens move out and i'll move it back down to minimum again and there you go so you can actually observe the physical operation of the lens moving in and out when you're performing an auto focus so now that we've seen how the autofocus camera works with still pictures the natural next step is to see how it works with video now one thing i've got to tell you right now that'll probably be a bit of a disappointment i know it was a disappointment to me is that in video operation it does not perform continuous autofocus at least this current iteration of the camera does not do that it just performs the auto focus operation once nonetheless it is still useful for video so let's go and see how we can do video with our auto focus camera now using the raspberry pi auto focus camera or for that matter any camera with lib camera is pretty simple because we can use lib camera slash vid and that pretty well emulates the raspi vid command now i say pretty well emulates because there is a demo that is available with raspi vid that doesn't seem to be available with this but otherwise there are a lot of things you can do with lib camera slash vid and i'll show you this we'll do a help over here to demonstrate the commands let me just maximize this so it's actually more readable and as you can see there are just a ton of parameters that you can add to lib camera slash vid like there are parameters to dictate the width and the height of the output video the default is an hd video at 1920 by 1080. you can do the timeout to see how long the video displays on the screen again you can specify an output file we'll do that in a moment to record a video you can specify how the preview works and the preview can also have some arguments on it so that you can give it the width and the height so for example the example they show here 0 0 that's the coordinates on the screen that it's going to be at 640 by 480 so you can set the size of your preview window again a dash f for the full screen like we've been doing before there are all sorts of things you can rotate and flip the video you can set the shutter speed you can set the gain you can set the exposure the meteoring the auto white balance algorithm there is so much over here that you can do with the video so as you can see there is a lot you can do with lib camera slash vid so let's just didn't mean to close that just meant to minimize it but i'm not good at minimizing there we go all right so let's do and that should just give us a video screen for about 10 seconds and so we'll enter that and there we have a video screen for about 10 seconds although it's not particularly in focus at the present moment now remember we aren't recording that you'll notice the top of the display if you saw that gives you the parameters of the video the frames per second the exposure etc and you can even as you saw on the help screen dictate which parameters you want to see over there so let's just do the same thing again only this time let's put auto focus on it and now it focused in at the very beginning of the video but unfortunately it's not a continuous autofocus so if i were to bring something up in front of it right now it isn't going to focus on it now we can also uh of course do a video and record it so let's do a 10 second video and specify an output file and we'll do this as test dot 8264 and that's the file format oop i did a comma there meant to do as period excuse me and that's the file format that we use with these videos h264 and you can display an h.264 video on some video players like vlc and the video player that comes with the raspberry pi will display it you can also convert that to another file format and if you want some information about that the previous video i did on raspberry pi cameras gives you a lot of details about converting the video as does the article that accompanies it and i'll leave a link in the article that accompanies this video so that you can find all of that information so let's just enter that and so we auto focused and we're filming a wonderful video of the back of my workbench which is going to go for 10 seconds and then we can go up to our file manager and we can see that we have a video over here so i can open it up let's just play that a bit bigger and there we go and there's a video of the back of my workbench i probably won't be publishing that one on youtube but it does demonstrate indeed the video capabilities of lib camera slash vid used in this case with the rdcam autofocus camera now although the camera won't do continuous autofocus it is possible to do a manual focus continuously while you are filming videos so what i've done for this demonstration is i've done a lib camera slash vid with a time out of zero and you can see the window over here with the view of my workbench on it and as i move around you'll see everything's fairly out of focus and then i ran the same script i showed you before the python script that allows you to do focus and so i am able to focus the video and so i could move around and you know change the focus on this if i wanted to to get a little bit better and i could manually adjust my focus and so while that isn't the same as a full-fledged automatic continuous autofocus it does prove that you can indeed focus while you are filming a video now i'm just displaying this right now but it could easily have been recording it onto a video file as well in h.264 file so it makes me think that it would be possible programmatically for you to eventually emulate a continuous autofocus but at the moment i really don't know how to do that but i'm hopeful that someday someone will come up with that because that would indeed be a wonderful feature to have on this autofocus camera now streaming video is a great application for a raspberry pi camera you could use streaming video for a number of different applications things like watching the progress of a 3d print or security applications or perhaps watching the birds in your bird feeder and lib camera makes it very easy to stream video with this or any other raspberry pi camera so let's go and take a look at exactly how we do that now there are a number of methods you could use to stream video with a raspberry pi and some are better than others is a very popular one called g streamer but you can actually do this through lib camera vid and that's a pretty handy feature however you're planning to stream your video you're going to need the ip address of your raspberry pi so you can find it on your network and so you can use an ifconfig for that and i have configured back a bunch of information about the network side of your raspberry pi and you'll notice there's three ip addresses on here now this one you can eliminate this is what's called a loopback address and pretty well every computer has a loopback address of 127.0.0.1 and the loopback address is just used for diagnostic purposes packets sent to that address don't even leave your ethernet adapter they just turn around and go back in so i've got two other addresses here in my case i've got 192.168.39.119 and one up here 192.168.1.113 the reason i have two ip addresses is that i have both wi-fi and an ethernet cable connected to my raspberry pi if you're only using just the ethernet or just the wi-fi you'll only have one address now i happen to know that the dot 1.113 is the one that is on my ethernet cable because my wi-fi is on a different subnet the dot 39 one is the wi-fi so i'm going to note this address because i'm going to need it on the receiving side and exactly copied it down over here i didn't really want to do that so let's just get rid of that okay so we can use lib camera dash vid in order to do this so let's do lib camera dash vid and then we'll do a timeout of zero and then we'll do the inline parameter and then after that we'll do a listen parameter and we'll specify an output with a dash o that's not a zero it's an o and the output is going to be tcp colon slash slash zero dot zero the which is the broadcast address and then we'll give it a port number with a colon and an 888 and we'll hit enter and you won't see anything happening on the street on the screen here until you've connected the other end to the stream and at that point you will see your video here on the screen so let's go to the other end which is going to be an ubuntu workstation running vlc and see if we can pick up on this stream i'm on my ubuntu workstation and i'm using vlc media player to pick up the stream and so i've opened vlc i'm going to go into media and open network stream and i need to give it an address now you'll notice i've already typed it in here but it's tcp slash h264 colon slash slash and then the ip address which in my case is one nine two one six eight one dot one one three and then the port number which is a colon eight 8888 so i'll hit play and there we have my stream and i can move around and i can get the streaming video now there is actually quite a bit of latency over here so it's not a perfect stream but it certainly is a very simple way of using lib camera with a raspberry pi camera in this case the autofocus camera to do streaming video now autofocus aside this camera also has some pretty impressive specifications and i wanted to see how it stacked up against some other raspberry pi cameras so that's exactly what i'm going to do right now i'm going to compare this camera to a few other raspberry pi cameras so let's get going now here are the cameras that we're going to be using for our camera test along with some of the equipment that we're going to be using in order to perform the test now here of course we have the 16 megapixel arducam autofocus camera with its shell on here we have a raspberry pi version 2 camera here's the version 1.3 camera and here is a raspberry pi high quality camera along with the 16 millimeter lens on it now in addition to all of that i've got some other equipment i've got a rather long ribbon cable that i'm going to be using for testing all of the cameras i've got a tripod over here and this tripod can be adjusted in a number of directions which is really good because although the raspberry pi camera from arducam the autofocus camera does come with a tripod thread it's on the back of the camera whereas it really should be angled 90 degrees and so this way i can take the tripod and aim it down and put it on here another thing i mentioned about this camera thread is that it isn't deep enough and so the camera will spin on the thread in order to resolve that i've got a washer over here that i'm just going to put on in order to alleviate that problem now the raspberry pi high quality camera of course has a tripod thread mounted in the correct place and it can just simply be screwed onto the tripod these ones don't come with a thread but for that i've made this little adapter i actually made this a while ago for doing some other tests and it allows me to mount the raspberry pi camera onto the adapter here i just have to screw it onto here and i've got a quarter inch thread on the bottom of this and i just made this out of a piece of perf board and an old gopro adapter which i don't need because i don't actually own a gopro and i've got a test target over here now this is a test target used for photography it's got a number of colors on this side and on this side it's got a gray scale and in order to make this accurate i'm lighting everything right now with a number of 5600 degree kelvin video lights so i'm not using the workshop lights for all of this so we're going to test each of these cameras out and compare their performance now obviously i would expect this camera to outdo the version one and version two cameras but it'll be really interesting to see how it compares to the raspberry pi high quality camera of course i've got a very good lens on this and this might give this camera an advantage another advantage this camera may have is the size of its sensor but we won't really know until we try so i'm going to do a couple of video tests and a couple of photography tests and we'll see how this all compares to test the four cameras for still images i simply used lib cameras ash still but there are a few things that you should make note of before you look at these images the first one and most important one is that the colors that you see on youtube don't necessarily represent the original color of the video this is because youtube distorts the colors for most of my videos i use something called an lut or lookup table that can compensate for that but i'm not applying the lookup table to these images as it does unnecessarily distort them having said that let's take a look at the images from the four different cameras the raspberry pi v1 camera actually doesn't produce that bad a picture when you consider it only has a 5 megapixel sensor now as this is a fixed focus camera i had to physically move it until i could get the picture relatively in focus so it may not be perfect but all in all it isn't that bad the grayscale performance of the camera was also pretty good pretty impressive again for the original v1 camera now i was rather disappointed with the image on the raspberry pi camera v2 but one thing to note is that i have the no ir or noir version of the camera this is the version that does not have an infrared filter and so i believe the pinkish hue that you see on the image may be a result of the lack of an ir filter you can really see the pinkish hue on the grayscale picture in fact on the edges of the picture it's quite predominant so in actual fact when you look at it the raspberry pi version 1 camera gave me better performance than the version 2 camera remember the version 2 camera costs the same amount of money as the arducam autofocus camera the colors that the raspberry pi high quality camera gave off were quite impressive and that's not too surprising it's the most expensive camera of the one that i reviewed especially when you add the cost of the lens the lens also gives it quite a different focal length from the other three cameras so i had to move it back quite a bit so that is reflected a bit on the image but all in all it has very good color performance the grayscale performance of the high quality camera was also very good and here's the arducam autofocus camera which gives a very impressive picture not surprising considering it's 16 megapixel sensor but a bit surprising considering the fact that it doesn't have as good a lens as the high quality camera the grayscale performance of the autofocus camera was also quite good now here's a side-by-side comparison of the raspberry pi high quality camera and the rgcam autofocus camera both images are quite good and the same goes for the grayscale performance it's actually very difficult to see a difference between the two different cameras so all in all i'd say the rdcam autofocus camera in these very simple tests performed just as well as the much more expensive raspberry pi high quality camera for the video test i used the lib camera slash vid application setting the width to 1920 and the height to 1080. i then use the mp4 box utility at the command line to convert the files into an mp4 video now note that during the conversion some of the quality would have been lost and also keep in mind what i said earlier about the color distortion that youtube imposes on videos the raspberry pi version 1 camera actually produced reasonable video it's certainly not good enough video to use let's say for a youtube video but for robotics and security applications the video was just fine once again i received a lot of color distortion on the raspberry pi version 2 camera toward the red end of the spectrum and again i suspect that this is because i'm using the no ir version of this camera the raspberry pi high quality camera with the 16 millimeter lens produced quite good video in fact this would even be passable for something like a youtube video and i also received very good results from the arducam autofocus camera while it doesn't have the same high quality lens it still produced very good and very sharp video and that completes our video tests now i think you'll agree with me that the rdcam autofocus camera is a pretty impressive device especially for the price now the biggest limitation i found with this camera is its lack of continuous auto focus and video operation but remember the focus motor is being driven through software and so i'm sure somebody will eventually figure out how we can continually focus in video even if they don't it's a great little camera for the price now if you want some more information about this camera or if you want to grab a cheat sheet that has all of those cryptic commands i use to get it to work you'll find all of that in the article that accompanies this video and there's a link to that article right below the video another thing you might want to do is sign up for my newsletter my newsletter is not a sales letter by any means it is something i periodically send out to let you know what's going on here in the workshop and it's free to join there's also a link below the video to that if you want to discuss this camera the best place to do that of course is the dronebot workshop forum and you can become a forum member for free and there's details on doing that below the video and once you're on the forum you can discuss this and other electronic projects with all sorts of like-minded individuals and finally if you haven't yet please consider subscribing to the youtube channel all you need to do is click on that subscribe button and also on the bell notification and that way as long as you have notifications enabled on youtube you'll get notified every time that i make a new video so until we meet the next time please take good care of yourself please stay safe out there and i'll see you soon here in the dronebot workshop goodbye for now [Music] you

Channel: DroneBot Workshop

Views: 41,083

Keywords: raspberry pi camera, raspberry pi, raspberry pi autofocus, libcamera, libcamera raspberry pi

Id: JR1p1dwpT3I

Length: 42min 10sec (2530 seconds)

Published: Tue Feb 01 2022