Making Search Great: Giving WordPress Powerful Search and Filtering with Pods

you're watching the recorded session from pods camp 2016 in Austin Texas sponsored by WP engine welcome to making search great not again but for the first time giving WordPress powerful search and filtering let me introduce our speaker Scott Kingsley Clarke I am Scott Kingsley Clarke and I'm the lead developer of this project you've been seeing a lot of I am somewhat responsible for most of it but we do have three team members including me and Jim you've met Phil you haven't met he couldn't make it today but he is in Ohio working hard on pods 2.7 this week and we're trying to get it to beta and actually I'm going to show you a little bit we'll sneak preview into the beta at the end of this week I'll have a little bit extra time but a little bit about me since we didn't necessarily go around the room and then introduce Jim and I specifically but I am a lead developer I'm a senior web engineer at 10 up it's a firm that works with a lot of large clients and we've got 120 people or so and it's pretty fun my background with a WordPress I started in 2007 so a lot of you also started in 2007 Carol and so I've got a lot but it was mainly because I was lucky to work around a lot of really great people I think my the key to my success has been mentoring and talking with people who have a lot more knowledge than I do and tapping into them and not being afraid to ask questions also not being afraid to ask Google or whatever search engine because that happens every day and probably airing yeah that's right how many times a day you think you could Google yeah there you go I mean that there's no shame in admitting you don't know everything because it's impossible to know everything PHP has thousands if not hundreds of thousands of functions it's crazy and same thing with all these different plugins there's so much out there so it's really important to just look around and try to dig around and if you have no idea how the solution should go talk it out with someone else try to explain your issue that sometimes helps a lot so going into the actual presentation we're talking about search and this is a period of time in my life which was a lot of pain because I was working on a large project and we had to search a lot of content and WordPress by default it only searches post title the content exert and it relies on mice ql which is the database back-end of most WordPress sites and that could be pretty slow because if you are working with millions of rows millions of posts across all sorts of different post types we're talking about a really large enterprise level project you can't just go searching through that database to pull out posts based off a field that is incredibly intensive on the database server especially when you have 50 hundred maybe a thousand people on the side at once all trying to search and so a couple of the things that is also not good at is relevancy relevancy does not really get all that better than when you search for a couple of keywords it pulls out what we returned but it doesn't really sort it in any super significant meaningful way so you might be searching for hats but really Santa hats is on trend this year and we've had 10 posts about it but we haven't had any posts about it before that until 2010 and so you search hat and the 2010 post comes up that has no relevancy on a number of for a number of reasons for a client that may want their more recent hat to show up at the top also there's no advanced filtering obviously you just type in the search box whatever you can get away with and that's all you get so I'm going to talk to you a little about a thing called elastic search and what this is is a database server that gives WordPress away and other projects way to sort of take away the searching part of it take that out of the equation have another service provide that and this allows us to take advantage of a lot of really useful features built in by furs for highly scalable projects it gives you the ability to create relevant raters results you can sort them by my date or you can sort them by a specific configuration coming the combination of date and how many times the keyword appears which feels that keyword appears in weighted results so maybe you want to search half but you want content to come up that had hat in it in a specific custom field give that the most weight you can do that you've got a lot of complex ability to customize this highly customizable you have the ability to do auto suggestion so you're typing into the search box and it starts pulling up key words that match the content that is in the elastic search also cleanse a matching so say you search for hat it also come up with hats or say you search for hats it also come up with hat also the ability to handle miss misspellings also to stuff like that and geographic searches by location latitude longitude stuff like that so it's it's a really cool thing it's recently come into heavy use at ten rock over the past few years and we even build a plugin called elastic press it's a free plugin available on wordpress.org you can also go to elastic press dot IO and it allows you to basically take your WordPress site and send it out to an elastic search instance this isn't something you can just do on the P engine or other shared hosts elastic search is actually a service you have to have you can't just install on a server it there's a process to do it there's a process to maintain it there's a process to keep it updated all those sorts of things so there are services available but with our plugin it integrates with WordPress and it lets you filter by taxonomy terms post meta values and keys lets you do the fuzzy search on all sorts of fields even once to mention here you can search across multiple blog so you have a whole multi-site instance with ten different shops or actually let me give you a better idea so you've got news organizations and you are a conglomerate and you have all your sites all together in this one big WordPress instance and each two adds its own site has something its own content its what categories all that so in this case in lastik press you can actually tell your search when you go to the site search search across all of the sites so maybe in your search box you might have an option say search across from all of our sister sites or whatever you might want to call it this lets you do that sort of thing really very easily and it's an extremely performant I can tell you that for sure after a year working in the thick of it results return by relevancy and calculations are really highly customizable like I was saying you can do so much with the code here through filters through WP query integration it's highly extensible and performant with the WEA integration though we're not just talking about search like just take the search part out of the equation now we're talking about admin area so you go to the admin area that can be completely pulled in from elasticsearch any parts of your site that may be slow maybe you have to do a specific query to say I want to pull on my home page the top 10 results of the most hits that I've had on a specific series of days or maybe for specific categories and exclude a certain post so you could do all these different things complicated things and have it all pullin from elastic search and this is it wasn't an easy feat I was involved with a lot of those integrations with WP query and it was a it was a really cool thing to be able to see it actually all work because I mean like I was saying deal with more billions of rows this can every little second can really make a difference for your client and also it has some really cool built in modules for related posts sort of like jetpack has related posts it pulls the related posts all from the content in your post and it actually allows you to display the related posts at the bottom of your posts just like like I said jetpack Lilly the same thing and they have also their own elasticsearch implementation within jetpacks usage they use the REST API and do all these different cool things but anyways the last search isn't really limited to jetpack and automatic it can be installed in all searches from places and elastic press sort of connects you to those there are also other options because it may be hard to come across a lots of search solution for a client with a very shoestring budget I'll show you a workaround for that in a minute but if you want something that's entirely my school powered you could use something like searchwp it has its own database tables which creates its own index of optimized queries so that you can query your content and query across all different types of content the keywords it indexes your PDFs and documents and media it's got some really cool stuff it's not as customizable as elastic press in the sense that all of the cool things are lastic search does for you automatically it can't do all those things but it does have a lot of cool things in it it also provides you some stats too from search stats but those can be provided as well through elastic search solutions elastic press is two point something yeah two point one point something pretty sure it's two point four three yeah that's right enough comm slash I'm sorry on github.com slash ten up slash elastic press that's the github version of it but it's also on wordpress.org if you wanted to install directly from WordPress drawback here is it still uses MySQL it's going to be a big problem for your your pipes a authors also though a another solution here facet WP you may want to have a little keyword box but really when you're working with products let's say you want to build with Amazon you have a search box sure so searchwp can give you that search box give you some cool stuff or elastic press but what faceted repeat gives you is the ability to do drill down searching so you might want to search by category and click on through and filter on further by price maybe I really won't trust one manufacturer so I'll just choose them as the seller and maybe even I might choose if I'm looking for a computer I want to find one that may have a better processor so I'll just say only these ones are higher and you've got a pretty cool faceted solution so if I said that VP makes that proved actually really easily possible through the UI it's also ajax-based so when you're doing it happens right then and there just sort of like Amazon works as well it's another important thing you're it's compatible with existing content your custom fields in your taxonomy so you don't have to do too much to integrate with it but it does have integrations with WooCommerce searchwp which lets you add that keyword field easy digital downloads w female polly laying all sorts of other plugins again the drawback is it's - well and it still provides a lot of value however that the database stuff is still a bottleneck there are solutions people are building with elastic search and faceted search I've talked with Matt Gibbs who created faceted WP and he's looking at a solution there as well for something maybe possibly related but let's move on - there's a the ability here to do all this stuff we're talking about with pods it's limited but it's all inside here and you can filter any pod and you can use it as a shortcode and it works with pagination it has search box that has a relationship dropdowns it's limited in that you can't actually have certain fields show up as your own filter but it's all going to be expanded soon in the near future it's not accessible or extendable there's no relevancy soaring again but it also still uses MySQL and another cool thing about what I'm doing with elastic for us is I'm working on a new prototype which uses elastic press elastic search to index all of your advanced contacts as well so it can let you create some really cool stuff there it's it's complicated because the advanced common types can be anything Nick might create 96 different advanced calm types and that I've never even created myself so it's important on that side to make it very dynamic and available but that flogging is is in the works right now and I'm excited about it and I'm going to show you a little demo of it about this the plugin let's talk about the elastic press itself so I've got a wordpress site here I've got the ability to create my content so let's go ahead and create one called review reuse I'm going to do my best to not show you any errors on the screen alright so I've got my reviews here basic couple fields here let's go ahead and go back there and add a custom field as well and let's call this one topic and for the sake of simplicity we're just going to get a regular text field but we do have quite a lot of options here Dave alright so there is my content type now when I want to index all my content and we're going to use this content type later on I would want to search through site and let's try that real quick so let's say the tectum is sticky so going to only one post that sticky on there let's say it's let's try a different one hello couple all right so there's a few they're actually a ginormous amount so here I'm just search for here but really I've pulled up Ihram and this may not even be what I want to be shown it only searched the title and the content so I didn't if I had a category it wouldn't have searched the category main stuff like that it's kind of limited but if I go and install a plug-in all the last press that's cool optical plugin it is three point one point one by the way Cassandra so I go to this last request area and I don't have an elastic search setup yet I need to set up my house and what you can do is use elastic search service there's a lot of them actually the one I use for my sites is called search Li there are enterprise level things available elastic press team at 10 up also provides a hosted service as well and elastic elastic search through search Li it's actually pretty cheap considering what it provides but they do have a free starter plan down here lets you create two different indexes at two different sites with up to five megabytes of storage of content so in this case this simple site can cover that no problem so I go into my account I'm going to create a site and it's really easy to UM does a say pops camp demo create it and here's my new panel for the elastic search it might be a little confusing but you don't have to carry all about this just pull this URL and go back into your elastic press when you first install it'll walk you through a lot better I've already had it installed before and I'm going to put my elastic search host in there now it's all set so I'm back at my area here I can activate all these cool things like I can activate the search overriding so it can handle all the searches or the post is pretty cool too and how about those two admin I don't have or commerce install to only show up at all but I'm going to sync all my content out so now what it's doing is sending out 566 posts of all sorts of pop-up post types and it's going to go send it out to my elastic search and over here I can see as it starts to index I've got a one and a half megabytes of content already and it'll chug around check along for a second and we can move on to the next page but basically it's pretty easy to use pretty easy to set up may take a little bit for it to first get everything but as you add new posts it's going to index that in the background if you edit a post you edit the content it's also going to index it and update that index as well probably it's actually probably synced it's just taking a minute if I go yeah you see these oh look type that's sort of like elasticsearch solution except they provide their own Search Appliance so it provides the ability to override WordPress entirely but it is their search so every time it loads up the search box it has its own code its own style and stuff it's similar how its customizable so something like swift type let me pull it up for other people who may not be familiar with it yeah we run we run we ran this on the pod site for a little bit I was trying out all these different solutions because our search was horribly slow for a while and so I tried this out and I just found that it just wasn't really providing me with what I wanted in terms of customization being able to filter by different taxonomy and honestly it was you have to build you had to use their code to build it whereas when you're using elastic press you're just using deputy query have your use W query before you know well it's a common used piece of code which a lot of things were very familiar with and you can take an existing loop on your site which may be customized and convert it to elastic press or in my default that search component we enabled a minute ago from elastic press that will take over your regular search so you don't have to change anything other than yes right but it's it's quite quite a lot less that was just quite cheaper we see here how much does this one again oh there it is yeah that says this is a lot cheaper than that um yeah so how did you talk search li was just you know less than a hundred bucks for at least their top three plans and that's pretty that's another reason why we moved on to something else but this provides everything to a tweed it's a service it's not just the searchability it's a service so it allows you to do all these different searching things whereas elastic price separates that service from the above the Search Indexing so elastic search sits in the back and elastic press is in the front it provides all the stuff that Swift type provides all together your elastic press right yeah well it's not my fly in there yeah so yeah it's a good good example of a solution that's related you know to how it went with the demo all right finish the syncing and let's go ahead and search on the side again a search by ear so again we came up with some stuff but as you noticed we have content that was slightly different for a number of reasons this one had ear maybe appear more often in the further further down the content it may have had the ear in some sort of category name or the author name it does search a bunch of these different things just right out of the box so it's a it's a pretty easy setup if you can get over the hump of creating your account it's certainly getting the URL from your account plugging it into elastic breasts it's kind of a couple of clicks really easy setup you saw me I just set it up and I don't know what was it ten five ten minutes but I was talking through it so it's easy setup and it works with double PC aligned as well so let's say I let's talk about that million row project going back to that one we have millions of rows we have let's say 20 different fields across those different content types we're looking at a huge database table for the post table just alone that stores all the content for WordPress so we're looking at maybe let's say 8 million rows of posts across all these different content types and let's say for 10 different fields across those let's say every piece of content has ten fields that is 80 million rows inside of your post meta table and that's not optimized it's just a key which is the name of the field and the value and I want to go and do some joins I want to go do find some information across all these from posts and when it happens in my school side it takes quite a long time and if you have 10 users doing that for all unique searches you are going to have a really large bottleneck you can solve this by scaling up all of your servers and your database service split up into its own database server and do all these different things have a big master slave total database set up and you can sort of alleviate some of those issues but you don't want to go through a lot of trouble if you have a simple solution right there and it's already working and it's working nicely and it's easy to use and it's it provides more functionality than WordPress itself so elastic price is really it's changed my life when working with WordPress but it changed my clients life as wallet because that 10 up we're going with this client and their sites were loading within 15 to 20 seconds each load and and that was just just to load up the initial results and when someone did a search with typically maybe 20 to 25 seconds and they sort of hid this underneath a few other things but when it came down to our logic press integration we cut that Mike we're now a quarter that's like five and ten seconds for the initial load depending on what they're doing and when we use caching that improves it even more so what I did was I took another step so not just the search information is cached I'm also caching when I'm displaying what I'm pulling out or the database so for each row that I am using to index inside of elastic press I've actually I don't have a corresponding set of meta inside of the post meta table the meta table only stores one value which has all of the data that I need that could be edited through a more customized application but it provided me the ability to have eight million rows in that post meta still I mean and it won't have 80 million rows it'll have eight million instead so I cut that by 10 for 100 percent in or without what is ever but yeah so it's it's a really really useful solution and when you think about those other types of situations you can get into with client that has a growing needs elastic press can really grow with that and it's still growing elastic press we have this exact client they had 20 filters on their site and we needed to filter my taxonomy spy meta values by all these sorts of different things we needed to customize this sort by date by numbers all these things are stored as just text inside the post meta table so that would eat thing you want to do I want to serve my date that's more weight on that time to load all those MySQL results so elastic press really makes that a lot easier by leveraging elastic search yeah it's I mean it's important to optimize your site for your clients because they don't know any better they just want you to build their site and when you tell them well WordPress can't do this well then they're going to say well WordPress must suck I need to move over to something else so it's important to know those solutions are out there available and are cheaper for their budget that's that's definitely a good point right there also another point there is what if that client says I want to search by old address I used to have you're searching in the text box at searching maybe names some very basic meta information but it may not have asked to those address the address information for the meta fields may not have access to certain things that you may want to search by so those the ability to search by an extended amount of information optimize it in a way that is quick that's no brainer for me I have install off the press on every big site I use so the last search isn't the end-all solution especially at lower levels basket fees used to its larger size as well it just depends on how you sort of hide some of the MySQL model mix facet fee is also you have to pay for annually um it's probably just a little bit cheaper than search ly elastic search plans but it's it does provide you with ability to do stuff like this so let's say I have a I'm trying to buy a car I know I want to buy a car for sure let's go ahead and say I want to find a BMW that matches my needs let's see here you know I want real look real will drive and let's say I want it to be can't pay that much no way let's try something my budget so NW to 3 there you go and of course this is a custom list but what you have the ability to do with facet WP is you can create a sidebar widget which has all your facets you can customize what kind of faceted it could be drop-down it could be a check box like you see here it could be a range value like in this case of the prices and it could also be a number of other different input types they allow and you can also customize by sorting so you can say I want the best value but also the best miles per gallon so there's some cool stuff you can do this is all out of the box you can but you probably building it yourself because yeah I yeah I do this in an hour probably because I can use the view query I can use leverage all these different things that are inside of elastic press to just I just type a little bit and done but the only problem is doing the interface and all that stuff that takes a little bit longer to make it really nice but right yep that's right and it is it doesn't require you to have a special server this one's on a small shared hosting plan relatively easily and also there's the 6fv I talked about which gives you a lot of power as well just without having to deal with elasticsearch and deal with a lot of data this is still a really great fit and then there's something I had mention also it's called relevancy and they have a free version they have a premium version as well a lot of people do use this can't be a testimonial for it however I did use it a couple times and it just didn't fit what I needed to do in the performant way for the amount of data I was working with and you can see here it has quite a few features and it's it's not a bad solution it's a great solution if you if you're just trying to figure out what you want it may actually fit your needs entirely but doesn't do the faceted search kind of stuff that kind of interface but facet of the key is I think the the current recommendation for people who who do that look at WordPress yeah that's what this was doing right here it's all Ajax and it's just happened right then and there we didn't do a page load it just changed it automatically right there I've even heard of people having multiple facets on the same page so they maybe have a facet section group right here then one below it to do other filtering on different pieces of content it actually can even work with WordPress out of the out of the gate you'll have to do any custom templating to support it but this right here is showing that you can sort of customize your output and display as easy is just putting a shortcode into a page and having your your fastest no wait or whatever and this is fully fully customizable you can have it up top you can have it on the right depending your theme of course and then in terms of that the theme is constraint in terms of where you want your widgets and your facets but there is a facet the shortcode you could put at the top your post which would display at the top instead well this is that it also is another theme restriction but it's one it's pretty responsive I would say right as the ability autosuggest is in the current feature of elastic elastic cross itself it's a feature of elastic search so you have the ability to build that we're actually working on multiple new modules that allow you to do those sorts of things but yeah just imagine you go to search it sort of like swift type suicides lets you just type in your search and it starts to fill autofill with some different options so like Google you go to google yeah but that's the ability is there in the code and I can make use of it right now if I wanted to that's right so you just call the elastic search query and get back what you want and return it anywhere you wish to have a WP query and for Nick use that we're at all lunch oh yeah how often would you say use it in a non advanced content type site all times he any query he would write to list the post to list the team list you can turn those into saying once you talk to elastic search or elastic press for this it's just a under speed T underscore integrate since it's a flag you add as argument and it happens and there's also I I mean to pull that further back I wrote a script that will cache all of the elastic press queries so it never hits elastic search for anything that's ever hit and that's called for the micro Ashtabula query this works with elastic press or just a regular WP query and this lets you further optimize your stuff to you avoid extra database requests and it's like I said it's fully integrated with elastic press so that you can cache everything and cache it pretty nicely in its caches to the object cache however if you are using pods and I have a couple minutes here I can show you a cool plug-in that we also built to further customize your site for it to load faster so there's a plugin called pod alternative cache you've seen it a couple times tonight when people in the plugins area this further optimizes your site because you can you could have a host like WP engine which is great lots of host support object caching lots of hosts don't support tons of object caching so let's say that million post site if I tried to do that on a shared house every time I would cache a set of queries I'd learn out of cache because it would just get so big so what happens when you do that you basically lose all the rest of your object cache that was in there so I say here it's sort of like a spaghetti maker so you put a bunch of spaghetti stuff in there it squeeze it out it comes out on the other side but it has to come out as side it otherwise there's no room for it so that's what the object cache does so even if you expire some of your cache things for say like you're building something you want to only cache for a day if you have ton of stuff that roll into that cache before the days up that catch main that may not even last the whole day and that happens quite a lot often with some some larger sites and I was on to the P engine hosting I was trying to figure this out and I figured out that they had a limit on their object cash into a specific megabyte limit or something like that it was maybe 200 300 a goodbye to whatever it was at the time and that was a problem because I had maybe a gigabyte of of built out object caching data I wanted to store from time to time and a problem that it compounds further is all of the pods configuration is cached and but it's you don't really notice it because you're loading around that you're going through the site and you're adding a content type here adding a custom field here but it's building all this in a cached object so it didn't have to go to the code database and say what was this again it knows it so when you throw in a gigabyte of object caching that goes away so every page load you have new object caching it's it's always loading everything fresh because it always keeps throwing everything out so alternative cache gives you that ability to have an alternative cache for pods itself to utilize for it stuff so it never gets obliterated by a gigantic ginormous object caching site and one of those benefits again of this plug-in is it integrates with pods alternatives cache so you can cache all of your extra queries do that as well so there's cool things you can do there any other questions about I can talk about caching too but any questions about searching filtering optimizing your experience for complicated or simple sites okay I believe search that the feed does that I think they have that ability to wait a specific post so it always appears for certain keyword black boots elastic search has the ability to wait things to but it's actually good interface to go over this or I can show that to you right now I don't believe it does but that's just because I haven't ever had to do that before so you got your content let's say a detail here yeah you don't really have you can just see that information here all the real raw data that we store for elastic press so that would be something you could add though it could be on top of lat express it didn't have to necessarily use the live exercise but right there's no interface yeah but it's a cool idea I definitely think that would be a not super common but a pretty common need for cases where things are very important to client and they won't budge yeah yeah like they're pushing a new product for Christmas and this needs to always show with the top featured that's a thing you can actually do feature sticky kind of stuff like that with the last request so you don't have to necessarily target for a specific keyword but any time that thing comes up inside of the search it can always be on top so that's an option right there that's so that's a little trickier because when you're dealing with going through the query multiple times you're not you may not be able to take into account that the one you want to appear may match but it's not on that page so that can be problematic also another problem could be that if you build a system and that what if you go to page two is that I can still look at the top there's things like that you kind of have to think about so in in a way that you can handle that what elastic press is dis sort so you can sort by specific taxonomy and say this one is featured for this season and so you say featured and that term has the feature term yes or whatever it might be and in your sorting for WP query you can filter the default search sort for WordPress you can sort it by featured taxonomy whatever there's a little information about it on the only elastic press site and how that you would actually query that but you can do multiple sorting so you would search by whatever and then you would sort by your taxonomy for the feature then relevancy so that always your products will appear on top that you want featured so that's that that's a good solution right there for elastic breaths yeah you could it would obviously more complex but you can do that sort of thing with the order by as well it when you do a save on a post you want to make it featured or something you can store a custom meta value or a taxonomy term background that they don't see and then that could be curated and add to the order by but disorder by is you see you can just type in title order by title and space medicine specific met eval you or maybe it's a text on ax me I don't have will have an example here of that but that's possible well I think we're we're done for minor I guess we'll go ahead and take a break so let all of this information coalesce somehow thank you for watching this recording from mods camp 2016 in Austin Texas as always you can get help with pods on our website at HTTP pod 0 slash forums you can also get help on our select chat at pods that io / chat we're inside the hashtag support channel Monday through Friday Mondays and Fridays all day and Tuesday through Thursday the first hour of each day you can also get help on our WordPress org support forum at wordpress.org slash support slash plug-in slash pods

Channel: Pods Framework

Views: 4,364

Keywords: WordPress, Software, Pods, ElasticSearch, ElasticPress, SearchWP, FacetWP, Search

Id: 2EBPkggfY4E

Length: 39min 23sec (2363 seconds)

Published: Sat Oct 13 2018