How to Add OCR to Your Ionic App with tesseract.js

what's up Simon eggs welcome back to a new tutorial and today we are getting back into OC our so this stands for optical character recognition and we will use the great tesseract J as plucking actually tesseract as a library on its own but tesseract J is can be used with JavaScript so we will use it in our ionic app using capacitor using the camera to capture an image of text and then hopefully getting the message so we did a tutorial on this a few years ago using version 1 now there's more than two it is slightly different and it works great there's actually also on the github page a demo of using a video I looked it up it's basically capturing an image of the canvas every few seconds so not really any magic so we will just do the basic version which involves our capturing an image and then hopefully getting the text from the image so I already started a blink new ionic application using capacitor as always and then we can go ahead and using the latest version of tesseract J as we've at next also since we're using capacitor and I also wanted to test this a bit on the browser you can install the ionic pw8 elements if you just want to run it on a device that's fine as well but that will help you to run it on the browser now two little steps so for the ionic progressive web app elements you need to go into your main TS file and add this little snippet which defines this custom element you see coming up in the browser when you open the camera so main TS right here second thing is go into your TS config app jason and actually add types note so this was in the beginning mandatory for me then I removed it and checked again and it worked again but I think this is really necessary for the plug-in to work we don't need that file anymore and also um initially I edit a little can't do it I initially added this to my tias config I'm not sure if it's required I'm just wanted to show it in here since I used it in the beginning and don't know which of these two settings actually fixed it so just in case that would go to your TS config now we can go into our application the first thing we need to do with the latest tesseract plug-in wasn't whatever you want to call it is to define a worker and this is already pretty interesting since you can have yeah we need the import as well you can actually have multiple workers in your application which means you could handle different jobs of character recognition at the same time now if you just want to let you the import force from Tesla req J as the create worker then we need anything else no not really that's just a little bit typing here you can also create multiple workers as I said but we will focus on one worker for a simple reason if you just wanna like recognize one image that the user is capturing within your application you don't really need multiple workers that isn't gonna improve your performance every workers just handling one recognition at a time so you can throw like for workers on one recognition therefore let's just do the one work out and actually is recommended to load this pretty early in your application so I will put this right into my constructor if that's like the first page of your application I would be at least careful if you increase the or decrease the performance of your applications so this stuff we do inside worker actually takes a few seconds and we can add a little luck for this so this will actually give us a nice little progress in the end as well and we can close this so for now let's just put a console look in here with a progress and then we have created our work up now once we get the worker we also need to tell the worker our widget language to use and again if you have multiple languages you might need a little switch and initialize the worker at a later point if you know that you're just capturing English French Spanish whatever it might be then simply put it like this load the work up load the language we will talk about self-hosting files in the end as well and initialize it with your language so this is now for English since this actually takes a few seconds let's put a lock in here as well we might need some brackets we will also keep track of the work already we could actually use a behavior subject but let's just use a boolean today we really don't need interesting now we're ready because let's see the application already served in here and we see quite a few locks and you see we're still not finished we are now finished that took like perhaps five seconds so really calculate this and load the worker at an early point because if you start to load the worker only when you want to start the recognition you will add these five seconds to the recognition as well which isn't really great UI now let's use a dummy image which I actually copied from the examples on github and we will also have a little variable for the OCR result and finally for I'm sorry the capture progress let's set this to zero in the beginning we already see that's the progress so we can use this inside the lager but there's a little problem right now the status is whatever and we don't really want that progress the only progress we actually want is if the status is recognizing text so that's the stages you will see when we recognize an image so let's put into our home page another function to recognize an image and we will just use our dummy image for now so if you have anything else you can also upload it or use it but it should be text because we're trying character recognition right so the result is now simply this dot work recognized and we pass in our image which can be a URL base64 string could be a lot of different things and once we get back the result let's also lock it since the result is actually an interesting object but for the real text result there's only one thing interesting and that's inside data and the text so now we just need a little button and let's also use an image so actually know what we're going through and then I unbutton let's make this one full and we can actually disable this button so it's disabled when the worker is not yet ready because in that case you shouldn't shouldn't call the OCR functionality so recognize image and then let's give this a try okay there's a little typo and we should add a click event which is recognized image great so we see this dummy image and we also see that the button is still disabled while our worker is setting up now we can call it and we see status recognizing text we actually don't put the progress anywhere so that should be done within our view as well and this actually takes a few things it's well it's not too long but perhaps it's challenging also notice on a device it's actually a bit faster now the result looks like this especially data that text is interesting since it's exactly the information we see in here there's a lot more information in this we're all not going into every detail are we see that we get basically the confidence for every second every line at some point we also find I think in symbols yeah basically for every object that were the OCR till detected we find a lot of information where in the image it is so you could have a list click on it and mark where in the image the character was recognized it's a really powerful result but if you just want to have the result it's simply inside text so let's also display the text I will bring in a little card so the card will also be displayed when the worker is ready and when our capture progress is bigger than zero so let's try this out one more time recognized image and once we move into progress we see capture progress going up again on the browser it seems to me a bit slow not sure why most of the time brother is actually faster than a device but there we go the full text that we see in this image was I think completely recognized but of course this is just a static image we also want to capture our own image so let's put in another button to call capture image and for capture image guess what we will use capacitor plug-in to capture an image which always looks like this mark the function is as if you're using a weight and then we need a bunch of imports for capacitor so let's move up plugins so we can destructure the camera plugin from the object and these just follow type safety we will use the data URL let's look the image for testing you could actually also use different formats I think but this one just worked like it's the easiest way it's basics before you can directly assign it to our image I think we need to use not this one but actually the data URL but we will see after recognizing it it will be a base64 image now last time I completely messed up my camera when using it on the browser so I really want to do it this time on a device there we go let's try and capture an image and I actually got this book here for testing so let's try this one perfect so we got the photo capture alright sorry I had to really apply the application so let's try again I guess use this book cover let's use this and let's call recognizer image so for smaller cheeks it's actually quite fast and we see everything you need to know about starting building business from the world's most influential intrapreneurs that's actually the text we wanted to skin so it definitely works on my device so hopefully works on your device as well just a quick addition I had a few issues so perhaps using a regular file instead of base64 string is a better idea and that's most of the time better idea I had crashes with other applications as well so if you encounter any problems with the memory try to save the file first and then load it from your file system also second edition if you call create worker there's actually also the possibility can we find this somewhere to use a local file it is also linked in my article I can't find it right now basically you can add a little snippet to your angle at Jason to copy files from the tesseract where is the package from the tesseract tesseract okay yeah there it is so you can copy the work Minjae s from the tesseract J s and also the tesseract core wasn't Jay s file inside your angle adjacent so it will be moved from note modules to your output follower and then will then create worker you can actually use work uh path and also I don't know Excel actually what's the second name but a message is you can use a self-hosted file or file file hosted version of these files in your ionic application or on your server so the application isn't downloading these worker files all the time same for the language files you can also host them locally or on a server they are kind of big so I wouldn't include them in your application but if you don't want to download them from somewhere you can have them in your local environment in a better way and actually I guess faster for your application if it's any way out for everyone on the walls it won't make each difference perhaps you could use some CDN but I guess that would already be included with the general version so this is improvement also you can use multiple work as I said in the beginning but that won't help if you just want to convert one image I hope you enjoyed this updated version about the OCR recognition or the optical character recognition using ionic using angular using wasn't to kind of beta off the tesseract j/s plugin if you got any questions about this let me know it's usually below and I hope you will build a great app with this feature if you enjoyed this video please hit the like button and stay subscribe so you get notified about all the new tutorials quick wins and other app development and web development videos on this channel if you want to learn more about ionic with in-depth courses a community of like-minded developers so you can learn and build your apps faster you should definitely check out the ionic academy which is my coat school to help you with everything I own egg with a huge library of courses material and a supportive select channel so we can get your app Howard I hope you enjoyed this video I will see you inside the next video have a great day and happy coding Simon [Music]

Channel: Simon Grimm

Views: 5,883

Keywords: ionic framework, learn ionic, angular, ionic angular, ionic guide, cross platform, hybrid app, ionic for beginners, ionic course, ionic, cordova, javascript, ionic 5, learn ionic 5, ionic 5 for beginners, angular 9, ionic 5 tutorial, ionic 5 angular, ionic 5 course, ionic academy, ionic tutorial

Id: c6MyNU0jp28

Length: 15min 39sec (939 seconds)

Published: Tue Aug 18 2020