Installing Koha 20.05 on Debian 10 with ElasticSearch, Plack, Memcached, and MariaDB

christian here from mcdonald educational technologies and in this video i'm going to walk you through the process step by step on how to set up the koha open source integrated library system on top of debbie and tim now like a lot of open source projects out there that are really really good one of the things that i've always found that's lacking in many open source projects that i use is adequate documentation so this video hopefully will serve as a launching platform for you to get coha running in your environment so you can begin to play with it i'm not going to go into using cohot from the perspective of a school and a library in this video but i am going to show you how to actually get it configured this video will likely be the first of several videos that i make uh around the koha system um and if you're excited for those make sure to subscribe and hit the bell icon so you don't miss those i plan on talking about taking a stand-alone quahog installation and deploying it in a highly available scenario where we actually clusterize all the components of the stack including apache elasticsearch memcache d and maria using uh galera so but before we get to that let's go ahead and start off with just getting coha installed on a standalone server and um as our as our first step so we're going to go ahead and jump right into a vm that is i've already pre-made for this demo this vm is nothing more than just a plain installation of debian 10 buster um i believe it has been updated let me just make sure that all the packages yep all packages are up to date so the only thing that i've done so far with this installation is that i've installed debian with the basic system utilities and also ssh so ssh is installed i'm logged in as root currently and other than that it is a very vanilla installation so coha has several prereqs that we need to take care of and some of these prereqs are not required um when you actually do attempt to follow the documentation um by default koha uses the zebra indexer for reasons that i'm not going to go into right now you really if you're deploying coha in 2020 um the the the focus forward with the koha community is to use elasticsearch as the backend indexing platform so what we're going to do in this video was we're not even going to bother with zebra we're going to install this with elasticsearch and we're also going to install with mariadb as a again a first step into the future videos where we take the sql data store and we actually clusterize that using galera but again we're not going to go into clustering at this point so we're going to go ahead and jump into some of the prereqs for installing koha on this standalone debian vm and besides the installation it has a static ip and i've created a dns entry on my local dns server just so i can hit it with a domain or host name in a web browser and i'll talk about why that's important later coha has multi-tenancy so you can actually serve multiple libraries from one installation and it uses apache rules on the back end to direct host names to direct to specific instances of coha running so we need to make sure that we have a static ip and also a a domain a dns configured with a host name all right so let's go ahead and jump into installing some prereqs before we actually install coha the first thing that we're going to do is run a command here that's just going to install a few things because this is a debian install and i'm logged in as root i don't have access to sudo which is not really necessary but all the commands are prefixed with sudo so i'm going to go ahead and bother rewriting those so i'm going to go ahead and install sudo i'm also going to install the gnu pg gpg tools for validating the gpg keys of the repositories and i'm also going to install software properties common which has the at add repository command which just makes adding repositories a little easier so i'm gonna go ahead and add these and install them again i'll post all these commands on paste on pastebin and once once those packages are installed we'll go ahead and clear the console and proceed with installing um installing open adopt open jdk so we're going to go ahead first we're going to add the open jdk gpg public key to this box oops didn't want to type that twice and we see okay so that key was successfully imported next we're going to add the repository for open adopt openjdk so go ahead and add that again if you get an error with this command that the add app repository command is not found you need to add that software properties common package that's where that command is found okay once we add the repository um we're going to update so just do a sudo apt update so sudo apt get update this is going to update those packages and then we need to actually install the adopt open jdk now we're going to install adopt open jdk 11 and we're going to install the open j9 jvm so we're going to say sudo apt-get install and then adopt open jdk 11 and open j9 this is going to install the open j9 jvm which is a little lighter in my experience in memory even though it might cost a little more in cpu so it's a trade-off you could use hotspot if you want both will work so we're just going to let this download here all right once the java um once the the java is installed we can go ahead and uh check to make sure that it installed by running java and then uh with the version uh switch and you can see here that we've got open jdk runtime um it's open j9 based on jdk 11. so that's important because elasticsearch actually runs on is a java application all right so next let's work on elasticsearch we're going to go ahead and do pretty much the same thing but for excuse me but for elasticsearch so we're going to start with the elasticsearch gpg keys and we get uh okay as the result for that um you might need to install the apt uh http transport um i'm not i don't recall if that's installed by default but we'll go ahead and run that and it was not so in order to in order to touch repositories that are protected with tls https you're going to need to install the apt transport https package from there we're going to run this command to just write out the repository to your app sources list d folder for elasticsearch so we'll run that then we're going to do an app to get update again to pull in those packages and then we're going to say apt get install elasticsearch and this should install elasticsearch against the repository that we just added now again you need to make sure that you are installing elasticsearch version 6.8 elasticsearch 7 is not supported by koha as of october 28th so you'll need to install the 6.8 branch for the latest version that's supported by koha okay once elasticsearch is installed we're going to reload the system control and then we're going to enable elastic search service so that it automatically launches upon system startup and then we're going to start the service so that it's ready to go when we actually install coho now there are two ways to check if elasticsearch is running the first is systemctl status elastic search and you can see that it is running and then the second way that we can check that if it's to check that it's running is to actually hit it with curl so elasticsearch is just a little web server and by default it's served over port 9200 so we can just say curl localhost 9200 oh we don't have curl so let's install curl uh we'll say app install curl we'll install curl here and we'll try that again so curl localhost 9200 and you can see we do get a response back we get the version we get the cluster name we get the the name of this node on this cluster this cluster only has one node at the moment we're not going to talk about clustering elasticsearch in this video and then the tagline you know for search so elasticsearch is working and by default it's only going to be listening on localhost which is fine for this we don't need elasticsearch to be listening on any other on any other ips besides localhost okay so once we have elasticsearch installed we're going to go ahead and get maria installed now the default version of maria that is in the debian 10 repository is maria 10.3 we're going to skip maria 10.3 now if you do read the qoho documentation they recommend 10.3 i have been using 10.5 in production for a while and i have not found any any problems at all i think it works really well and there are some ancillary benefits to using 10.5 over 10.3 that we might address in a future video but for now we're just going to go ahead and proceed with installing maria 10.5 so the first thing we need to do is get is grab the maria repo setup script maria provides the maria community they provide a script for actually configuring repos on machines and it's just the maria db underscore repo underscore setup by default obviously this is not going to be executable so we're going to run chmod plus x to make that executable and then once we do that we need to run it with the server version mariadb dash 10.5 and this will do all the necessary changes to automatically add the the package signing keys run apt-get update and um and all that stuff so it basically is a little script that does everything that we did for java and elasticsearch for us so it's kind of nice then we're going to run apt update and then apt install mariadb server and we're going to go ahead and install mariadb backup now we're not going to use mariadb backup in this video mariadb backup is maria's preferred backup system um if you've ever done maria clustering you might be familiar with bracona extra backup um this is a a lock free um backup solution uh mariadb uh backup is now the preferred backup provider when doing galera clustering so we're just gonna go ahead and install that it's it's not required um it's not required with a standalone server but we're going to go ahead and install it it's not going to hurt anything so we're going to go ahead and install mariadb and this should go pretty quick okay all right so at this point we should have um we should have the jre the java runtime installed we should have elasticsearch installed and running and we also should now have maria db installed 10.5 so the next thing we're going to do is actually get coha installed so we're going to go through the pretty much the same process we're going to add the koha gpg keys first and we get the familiar okay let me go ahead and clear the console to bring everything to the top next we're going to go ahead and echo out the um the 10.5 repo into our app sources list.d folder so we're going to go ahead and do that then we're going to run apt update to update the repos and then we're going to say apt install koha common and run this this is going to um this is the command that's going to install everything that we need for coho including apache all the apache modules uh pearl all the pearl modules and um and do all everything that we need for for coha so we're gonna go ahead and say yes and run this this will take a few minutes so i'm gonna speed through the video here so now we're actually ready to create our coho instance for our demo library so before we do that we want to talk a little bit about how coha handles multi-tenancy the way that this works is when you create a library quahog creates all the apache rules necessary to do the matching with the host names and there's really two different websites that are served by apache when running quahog there is the front end that patrons of your library will interact with this is called the opac or the opac and then there's the back end staff client and by default you can check what those host names are going to look like by going into the etsy coha etsy coha and then koha sites file and here you can see that there are a few different uh parameters there's domain there's the intraport the intra prefix the intra suffix the opac port the opac prefix and the opac suffix so this is where you can customize how your domain names are created when you actually run the command to create a library tenant so obviously the domain name we're going to change this i'm going to change this to my local ad domain so this is again this is the fully qualified domain of my inside network here my active directory and then the intraport we can go ahead and leave at 80 and the intra prefix we're going to leave that blank and the intra suffix we're going to replace the dash intro with dash staff now you can again customize this however you like but the way that i do this is whatever the name of my library instance is the opac domain is just that dot and then the domain um the fqdn and then the op the staff client or the intra as it's referred to here in this configuration file i just take that same name and with the suffix dash staff so that's what you'll direct um your opac users to if you have a kiosk in your library and you're automatically opening up a website you'll want to direct that kiosk to the opac and then your staff will use the staff portal okay so this is what that looks like now there's really not any reason to host these on different ports unless you you have a good reason if you are accessing this via ip addresses obviously changing the ports is going to be a requirement for that but because we're just going to go ahead and set up the dns side we can just do that and they can both be served on port 80. so we're going to go ahead and make those changes and we're going to save this configuration file out all right once we've done that we're going to go ahead and set up some apache modules that need to be enabled the first is the rewrite module so we're just going to say sudo 80 in mod and rewrite okay oops sudo a t in mod cgi that's required for actually interfacing with all the pearl backend stuff and then we're going to go ahead and reboot restart apache using system ctl restart apache 2. oops apache 2. all right we can go ahead and check to make sure that apache is still running by running system ctl status apache 2 and we can see that it is running and it should be listening on port 80. next we're going to go ahead and create our koha instance now when you install koha using the debian package whether on debian or ubuntu it comes with a nice little suite of commands that make some basic administration and configuration of your coho server a lot easier if you're installing from source or you're downloading the latest dev snapshots you're not going to get these commands but if you install from the package from the repo you're going to get these debian commands and they're really really nice so the first one we're going to run is sudo coha create this command is what's used for actually creating a new instance and then all we have to do is say create db switch this is going to tell it to create a new database on maria and then we're going to give it a name in this case we're going to say lib demo and we're going to hit enter all right so koha should now be running so let's go ahead and go to a web browser and let's go ahead and go to the lib demo dome the fully qualified domain of the opac and you can see here that the uh the server is saying system maintenance the coho online catalog is offline all right well let's try to hit the staff um the staff like the staff portal or the intra portal and i remember this is going to be the same domain as the opac with the exception of the dash staff suffix in the host in the hostname portion and we'll go we'll go to the lib demo staff and you can see that we now are welcomed to the coha 20.05 web installer now um we need to actually get the first username and password admin username and password from the koha xml file and there's a few ways that we can do that um but we're going to do it manually just by poking into the quahog configuration for this for this tenant for this library instance that we created we're going to do it manually instead of using the commands that exist to actually just show us on screen what they are immediately because we're going to need to get familiar with these with the quahog configuration both at the server level which is where you configure the domain prefix and suffix information and then also at the site level um so we're going to go into a site level configuration um this is located under etsy coha sites so we're just going to do a cd to etsy coha sites and then in this folder you're going to see uh subfolders for every single library tenant notice there's only one here for libdemo so we're just going to go in cd into lib demo and then we're going to nano coha cohab.xml all right so you're going to scroll down to conf the config section and under the config section you're going to see user and pass this is going to be the default initial uh staff user so user zero in the database coha in this case it's prefixed with coha underscore then the tenant uh library name and then there's a randomly generated password there so we're just gonna copy that password and we're not going to save that and we're going to come back to the installer and the username is koha underscore and then the name of this library instance so libdemo and then that password i'll go ahead and log in okay we're going to keep english as the setup language or as the system language and it says that all the required per perl modules appear to be installed and all of the dependencies are installed so we'll go ahead and continue to the next step um here is where it is creating the um the database and the user again the database and the user are named according to um the application in this case it's cohab underscore and then the name of that library tenant so go ahead and go to the next step and it says that the database was created and the user that was created is granted all privileges on the database so we'll go ahead and click next so now we're ready to create the database table so now it's going to create the tables build the schema in the database and seed it with some initial data database tables were created we'll continue on to the next step now it's asking for some basic configuration questions so we'll click next now it's asking whether or not you would like to use unimark or mark 21 if you are unsure you probably want to go with mark 21 i am not a library scientist so i am not the person to ask concerning really the differences between mark 21 and unimark but this is how libraries globally around the world store information so that libraries can easily communicate with each other so this is the bibliographic record information and there's two different primary two different flavors of mark [Music] unimark which is primarily used in european countries in a few european countries and mark 21 which is the one that's preferred globally so we're going to go ahead and keep that as mark 21 which is the default and from here we're going to we're going to just keep the defaults you might want to consult with your librarian or your library staff on some of these options but we're just going to go ahead and keep them as default all right we're going to click on setup coast basic requirements and now it's going to redirect us to the onboarding process all right so here's now where we're going to set up the administrator identity so we're just going to go ahead and type in my name and i'm just going to create my card number is zero if you if you seeded your installation with libraries you're going to see a list of of libraries that are here in this case i'm going to go ahead and choose centerville and my patron category will be staff and this is going to create my user with the super library and account permissions so i'm going to go ahead and create a username and a password and we're going to go ahead and say submit okay this is asking for some item types here um we're going to create one item type we're just going to call it book b-o-o-k again this is more of questions for your library your librarian these are more library science questions but we're just going to create a an item type code of book and we're just going to say books for the description click submit we can add more in the future pretty easily new circulation rules we're just going to leave the defaults here because we want to just get to the get to the actual um coha staff interface all right it says congratulations you finished and are ready to use koha we're going to say start using koha and now we're presented to the login screen so we're gonna go ahead and type in the username and password that we defined and we can log into the staff portal it's running it's ready to go if we go to the opac we can see that the opac is running as well so we'll just go to the non-um the non-staff domain oops i think i typed that in wrong lib demo and then the fqdn of my internal domain and you can see that the opac is running as well there's obviously not going to be much here so now let's do some optimizations so we have by default coha is using the zebra indexer we're going to change that to use the elastic search indexer and we're also going to enable plaque which is an apache module for optimizing the the perl execution and um yeah so that should be it for the optimizations by default you do not get those but we should enable those so we're gonna go back to putty back to the ssh instance here and we're going to go ahead and enable another module so we're going to do a t and mod and we're going to enable the headers module and the proxy underscore http module so it enabled both of those we're going to go ahead and restart apache again so system ctl restart apache 2. and then we're going to enable um plaque so there is a built-in quahog command with the package installation for enabling and disabling plaque and it is just cohab-plaque and we're going to say coho dash plaque with the enable switch and then the instance of our library which is lib demo okay it says plaque enabled for lib demo opac and the intranet which is the staff uh the staff portal uh sorry i forgot to actually start plaque so after sudo uh plaque enable lib demo we're gonna say sudo koha plaque start live demo at this point we're going to go ahead and restart apache so system ctl restart apache 2 and then if we go back to the koha staff portal and click on about coha we're going to see a new a new line for psgi and you can see that it says plaque and it's in the deployment mode so plaque is now enabled so this is a really useful optimization it makes things a little more snappy memcached by out of the box already comes configured and the name space from m cache d by default is again koha underscore and then the name of this this tenant which is libdemo all right so the last step here is we need to enable elasticsearch so again by default koha is using the zebra indexer the zebra indexer doesn't do too well with clustering elasticsearch out of the box includes clustering capability and moving forward elasticsearch is the future for cohas so we're going to go ahead and configure this standalone instance excuse me to use elasticsearch so we're going to go back to putty once we have elasticsearch installed which we already do we need to add the ic the uh and the the analysis icu elastic plugin and we can do that by first cd over to the user share elasticsearch bin and bin folder and then inside of this folder there is a command called elasticsearch plugin and this is a command that is used for installing plugins so we'll just say elasticsearch plugin and install and then the analysis icu plug-in oops i typed that in wrong of course i did analysis icu plugin okay once we've installed that we need to actually restart elasticsearch to so that it's actually using that plugin so we'll just say systemctl restart elasticsearch oops system ctl cannot type today system ctl restart elastic search and let's go ahead and just do a sanity check here and make sure that elasticsearch is running it is running we can go ahead and curl localhost 9200 to make sure that it is actually it is actually running here and it is so it is running and it is responding to get requests there and now we're going to go ahead and enable elasticsearch and and get that running so before we do that we want to go back to the koha staff website the staff portal and we want to go to koha administration and we want to type in under the global system preferences we want to type in search engine and this is going to find a key where it's asking what our search engine is we're going to say elasticsearch we're going to change that from zebra to elasticsearch we're going to click on save and then we're going to go back to coha and we're going to actually enable elasticsearch so we're going to use the koha dash elasticsearch command we're going to say rebuild and then we're going to say d and then the name of the library which is libdemo now this command what it's going to do is it's going to take the current index from zebra and it's going to take that and build the initial index in elasticsearch now if you're installing elasticsearch on a koha instance that already has tons of bib records this is going to take some time because it's having to touch every single record and every single value in the mark records and indexing them accordingly because this is a new a new library when i run this this is going to actually not take very long at all but you're going to see a lot of cryptic stuff going across the screen but once we do that we're going to go ahead and go back to koha and we're going to click on about koha and you can see under the server information that now we have another line for elasticsearch and the the current node which again we're only running elasticsearch here on the local machine this is a standalone installation but if we had other elastic search nodes configured you would see a list of nodes um it's running and then you can see the indices uh for the biblios and the authorities which both are empty at the moment um because there are no bib records and no authority records in uh in this instance yet so at this point um elasticsearch the index will automatically update as your librarians add records to coha and it will be using elasticsearch instead of zebra and i found that elasticsearch is a little more performant than zebra and more importantly elasticsearch does really well with clustering when we actually take this server and we build two new servers just like it and create a cluster of coha servers elasticsearch is is is was built with clustering in mind so at this point you should have koha successfully running on debian 10 with elasticsearch 6.8.13 memcached maria 10.5 and plaque enabled if you have any other comments or you have questions about koha or anything that you saw in this video please make sure to hit the like button subscribe and post your comments down below i look forward to reading your comments if i did something wrong or you have a suggestion for how to improve i'm always looking for feedback and i look forward to reading your comments below thanks

Channel: Christian McDonald

Views: 951

Keywords: mced, tech, technologies, mcdonald, educational, koha, library, management, system, opensource, oss, free, mariadb, galera, elasticsearch, cluster, debian, buster, 10, squeeze, stretch, memcached, plack, json, xml, apache, apache2, systemctl, wget, curl, http, high, haproxy, community, vm, esxi, vmware, putty, ssh, root, admin, update, apt, get, apt-get, gpg, 6.8.13, 10.5, tls, covid19, covid-19, coronavirus, series, linux, ubuntu, clustering, ha, high-availability, availability, pfsense, packetfence, servers, server, science, marc, marc21, unimarc, index

Id: osTey96Qs4U

Length: 35min 14sec (2114 seconds)

Published: Wed Oct 28 2020