Updating BIG-IP HA systems with a point release

hi in this video ask f5 shows you how to patch your big ip high availability system with a point release to resolve the cve 2020 5902 vulnerability in this demo we'll start by installing the point release on our first device named big-ip2 we're gonna upgrade this big-ip 15.1.0 h.a pair to big-ip 15.1.0.4 point release so that we can fix the remote code execution vulnerability before you install any new versions you should use the tmos shell to run a test load and identify if there are any configuration errors so we're going to start by going to this is big ip2 go to the command line and this is recommended in the upgrade article we're going to go ahead and i guess we can do this from tmsh load cis config verify and this doesn't load the configuration it just does a test load and when you do a test load you can identify any configuration errors that would cause a problem with the configuration loading after an upgrade and so it's just a kind of a test and validate and if there's no errors if there's just nothing and it returns the prompt that means that's good nothing wrong with your configuration just a test load doesn't actually load anything just validates that the configuration is okay so here we have a success next we'll use the command line to verify the service check date so next we want to check the service check date and i believe we have a video for it but we'll just throw it in here just in case because it's really easy so we're doing a point release off here so the service check date's going to be the same we want to have a later date than that date november 5th 2019 so in order to do that we're going to run this command for more about this command see ask f5 article k7727 also linked in the description section and there we go we're date the date is later so we're okay there but if you had a date that was before the license check date then you'd want to re-license and we have that covered in this article 7752 shows you how to re-license okay so back to the gui and we did everything we needed to do there from the command line next we'll get the configuration in sync now we're going to make sure that we have synchronized the pair or the device group and so we're going to go over to big-ip one and see we have changes pending on both these showing there's changes pending that need to be synchronized so we'll do that i'm going to make sure that they're in sync before you do an upgrade so here's our failover device group and here's the this has the most recent changes so you want to sync from the most the device with the most recent changes and that happens to be big ip1 so that's we're going over to the gui big ip1 we're choosing that as the device we want to sync we want to push the sync from change is pending so big ip1 recent changes selected we want to push just hit synchronize okay now we're in sync after your configuration is synchronized create a ucs archive and then save a copy to a secure location we're going to upgrade this standby unit first big ip2 is standby it's not taking any traffic so we're going to upgrade this one first before we do that we'll take a ucs archive and we'll just call this ucs for demo and we wait for the ucs saving active configuration it's saved okay and then you want to uh once you've created it you want to make sure always to download it to a separate system wherever you save your ucss off the box just in case the device is unrecoverable for some reason you have your ucs backup on a separate system not on the big-ip keep a copy on big-ip keep a copy elsewhere just in case for disaster recovery next we'll import the point release iso file and now we are going to import the image the 15.1.0.4 image we're going to go to software management image list and import because we're going to import our big ip 15.1.0.4 and there we go you just wait till it imports the new image for more about downloading software see ask f5 article k167 also linked in the description section after you import it it'll appear in the list of available images big-ip 15.1.0.4 uh image point release and first as noted we're upgrading the standby system so anyway it shows up there now we'll verify the md5 checksum and so now we will check the md5 sum go to shared images okay so we have our md5 sum so you do just md5 sum dash c to check and then specify the md5 sum file that contains the md5 sum the ucs imported the image check mp5 sum and if it's okay it'll say okay for more about the md5 checksum see ask f5 article k8337 also linked in the description section if your sync type is set to automatic you should temporarily change it to manual we'll just go to big-ip 1 here and something you want to do is temporarily i'm going to go to device groups click on the device groups and if you have a sync type of automatic you want to change it to manual just temporarily you don't want it to try to be syncing while you're on two different versions in the middle of your upgrade so if it's on automatic change to manual now we're ready to install the software and then reboot to the new point release version go back here okay now we're ready to install and so we have our disk right here hd 1.1 we just have the one boot location with the final build of 15 1.0 release it's the active volume default boot location so we're going to create a hd 1.2 to install our 15.1.0.4 image check the box install okay so we just have the one disk typically it's just hd1 or md1 if you have a raid so we got our hd1 selected and then we are going to type in 2 to create hd 1.2 volume to install to then we click install all right we're installing to hd 1.2 the 15.1.0.4 software and i have our install status there again standby unit that we're installing this on it's not the active the active is big ip1 that's handling all the traffic all the application traffic is going through big ip1 so you can watch this gui or you can go to the command line do this watch tmsh show this off status and you can see it every two seconds it updates the command upgrading hit ctrl c to quit out of that go back to this screen 15 1.0.4 was installed successfully to hd 1.2 so we have here install status is complete the active volume still hd 1.1 with 15.1.0 so we're going to boot into hd 1.2 which has our upgraded version so we have a success here we're going to go to boot locations and we're still on the standby device we're making sure big ip1 still handling all the traffic it's active big ip2 is standby and let's go ahead and boot into hd 1.2 let you confirm the changes you're making the action you're going to take which is booting to 1.2 with 15.1.0.4 and this option is just if you had made any changes to the configuration you can install configuration when you boot into it but since we didn't make any changes between the time of installing and booting then don't really need to do it by default it installs the configuration when you're doing the upgrade and you can turn that off but that's like you can make changes to a db key for that but just all we need all we need to worry about for this video is by default it installs the configuration if you make changes between the time you install and actually do the boot you can select yes for this and then of course pick the source volume but we'll just leave it at no because we didn't make any changes and then activate and it just asks if you're sure yeah i'm sure if you want to see what's going on you can have connect use a console connection which is not a network connection to watch it reboot um and here where we are still active on big-ip one it's showing that config sync is disconnected current config stick state because big-ip2 is [Music] being rebooted and that's okay that we expect that to say that we'll wait for the reboot process so we rebooted we rebooted the device to hd 1.2 which is running 15.1.0 at the point release that fixes the problem and device is back still on standby we've got big ip1 and active and they're on different versions you don't want to synchronize next we'll verify the new version is active on our newly patched system we'll uh just verify the version we're on okay so here's the 15.1.0.4 on 1.2 it's active so we're running on that boot location now we're upgraded and you can also check your configuration this is just a demo system so we've only got one virtual server but you can just double check to make sure that your virtual servers have all loaded and they're all up we can now force a failover to the newly patched system so now you're ready that you've checked the configuration is loaded okay we want to go over to big-ip one so now we're ready to have the big-ip2 system that was upgraded ready for that to take traffic so we go to big-ip one traffic groups check the box for a traffic group one and force to stand by current device big-ip one next device big-ip two confirm force to stand by okay now big ip1 is standby big-ip2 is active and taking traffic and so at this point you'd want to check the traffic flow so check all your applications that are being load balanced are managed by big ip2 just make sure everything is working as expected all the traffic is flowing normally we're done with big ip2 now we can go to big-ip one it's standby so we wouldn't just repeat the procedures now that one of your devices is patched you're ready to install the new version on the next device on the next device repeat all of the steps shown here note that the steps for the next device are similar to part one of this video with the exception of syncing the configuration it is important to not sync configurations while the devices are running different software versions okay so big ip1 standby unit has been upgraded successfully next we'll verify the new version is active on our newly patched system boot this to the point release 4 volume hd 1.2 verify what we're doing here no need to install the configuration because we haven't made any changes from the time we installed till now when we're booting activate yep we're sure we want to boot to another volume so the first thing we want to check is there's no error message and the configuration has loaded and more uh more importantly or equally as important is that they are available they're showing green status green so that means that the pool members are all up and so the virtual server's up we can now force traffic to fail over to the newly patched system so we're going to go ahead and fail over traffic to big-ip one and to do that okay so that's big ip1 standby we'll switch over to big-ip2 and we're going to go to device management traffic groups so now we're going to test to make sure traffic's okay on big-ip one so we'll force traffic group one to stand by active device big ip2 next active big-ip one forced to stand by and confirm you can also see the current next active etc forced to stand by okay so we failed over to big ip1 it's active now and they're both running the point release and one thing you can check is just make sure that you're active on hd 1.2 with the point release that you installed so both devices now are active on 15.1.0.4 so a successful upgrade for both after you've finished installing the new version on all devices you must synchronize the configurations from the device with the most recent changes last thing is we will go ahead and we haven't made any changes we synchronize before upgrading and we haven't made any changes everything should be the same but if you had made made changes make sure you're synchronizing from the device with the most recent configuration in this case we're going to synchronize we're going to synchronize from big-ip 1 because it's showing has the most recent changes because we just loaded the configuration on it after a reboot and upgrade so so it'll show that and but best practice is not to make any changes in the middle of the upgrade so you shouldn't have any changes on big-ip 2. so we're going to go ahead and we're on big-ip one it has the most recent changes we'll go ahead and that's checked push from one to two and sync that way we can get everything back into sync all right and there we go our failover group is in sync shown in sync there we'll go look at go scroll up to big ip1 and then we'll go click over here to big-ip2 and it is in sync so both systems are upgraded active standby in sync if your sync type was set to automatic before the patching process you can now restore that option last thing you want to do is go to the device group and if you had changed it to manual temporarily you can go ahead and change it back to automatic if necessary if that was your uh preferred configuration is automatic then you change it back to automatic because you want it in manual for the upgrade we're just showing uh if if you did have a configuration load error this is what you'd see the device would reboot and become available but then it would have this message in red here saying it's encountered a configuration problem and so you just continue and login and then you would see this message here telling you that the configuration failed to load so that's just an example of something that could go wrong if there was a configuration issue you would know right away when you logged into the gui um it would show that the configuration failed to load and so that's what that looks like and at that point you would want to if you were skilled you would troubleshoot and figure it out yourself um if you weren't familiar with troubleshooting big-ip you would call and open a support case and have them figure out why your configuration didn't load and that's it if you have other questions about this vulnerability refer to the security advisory at www.f5.com 5902 thanks for watching

Channel: AskF5

Views: 5,049

Rating: 5 out of 5

Keywords: big-ip standby unit, troubleshooting big-ip, standby unit, command line, ha pair, md5 sum, configuration load error, device group, standby system, standby, boot location, load balanced, cve, device groups, traffic flow, traffic, tmos shell, md5sum file, ucs backup, sync type, image list, linked, sync, upgraded, upgrade, hd1, tmsh, import, md1, log, license, installs, f5, big-ip, cve-2020-5902, askf5, hotfix, ucs archive, configsync, md5 checksum, iso file, failover, fail, over, config sync, rce

Id: MqHislURnK0

Length: 23min 18sec (1398 seconds)

Published: Sun Feb 28 2021