Hello everyone and welcome to the April edition of
Techspert Talks. I'm Troy Raney and I'll be your host for today's session. Today's presentation
is Migrating Qlik Sense Client-Managed with our own Sebastian Linser. Sebastian, why don't
you tell us a little bit about yourself?
Hello, my name is Sebastian Linser. I'm
a Principal Technical Engineer at Qlik Support. My main focus there is Qlik
Sense Client-Managed, and I have been around with Qlik Support for 7 years now.
Great. Today Sebastian is going to be talking us through the whole process from start
to finish when it comes to migrating Qlik Sense implementations from one environment
to another; we're going to go through the whole backup process; he's going to demo that
migration process; we're going to talk about certificates and highlight some best practices
and things to be aware of. Sebastian, what are we going to be demonstrating today in detail?
So, we want to demonstrate how you migrate a Qlik Sense Client-Managed installation from one PC to
another. So, in our case we have the scenario that we have a physical machine we need to get rid of;
and we need to put it on a virtualized machine.
Okay, and we're going from the same
operating system in both machines?
Yes, in our case Windows Server 2019.
So, basically when everything is finished; we should have the same apps, same tasks,
everything running, but on different Hardware once we get finished?
Yes, that's the idea.
Great, can you share a
little bit about your setup?
Yeah, I do have a Stand-Alone database on a server
called Qlik Server 3 you can see it here in the Services list. We do have a PostgreSQL
Service. Right now it's PostgreSQL 14.
Okay. So, this is the Repository Database
storing all the settings for this implementation; and it's on its own machine?
Exactly, when we check in PgAdmin, the standard Qlik Sense databases: Licenses,
QSMQ, QSR, and Sense Services exist; and we can also see that we have the standard
user Qlik Sense Repository down here.
Okay, and that's pretty much
staying the same right?
Yes, we will still stay in 14, but we have
to do some minor adjustments to the database which I will talk about later.
Okay. So, where do we start with this whole migration process?
We start by connecting to the Central Node. I only have a one node system here.
Okay. So, this is the physical machine where we're starting from?
Yes, this here is our Central Node. I have one app and I have the monitoring apps. So, it's not…
Yeah, pretty simple implementation, but that's fine for demo purposes.
And we are on the February 2024 initial release.
All right.
When we scroll down here to Service cluster, we do have a share
which basically serves Share for Qlik Sense.
Okay. So, the apps and log files are in a
separate machine in a share drive and they're staying put?
Exactly.
Now what are the requirements for
the machine will be migrating to?
You have to make sure you fit all the
requirement checks: Windows Server 2016, 19, or 22. You can use Windows 10 and 11, but
it's not recommended to use in-production.
And that's really just because it hasn't
been thoroughly tested yet, right?
Yes.
Okay.
The processor Supporting AVX, at least
if you want to use natural language.
Right.
At least 5 Gb of disc space; while I would say you should have more than 5 free before…
Yeah, bare minimum.
An SMB share. Important .NET 4.8.
With .NET can't you just have a later version installed instead?
You can have this installed as well, but you do need to install 4.8.
Right.
It will not work if you only install 6.
It's okay to have multiple .NET versions.
Yeah.
You see, we used a 14 database on the second server.
Right.
We would also be able to use a 13 or 12 if we
have it as a Stand-Alone, but you should not use the 12.5 bundled anymore, because it has
a couple of security vulnerabilities which you want to upgrade away from.
Yeah, there's a lot of material out there about that.
And important is: there is no Internet Explorer Support anymore.
Oh yeah, TLS.
TLS is at the moment at 1.2. So, if you
have Windows Server 2022, and you could potentially enable TLS 1.3; for the moment please
do not. It's not yet fully TLS 1.3 compliant.
Oh it's good to be aware of. All right. So,
assuming we've got all those prerequisites taken care of (especially making sure we got
.NET 4.8 installed; what is the first step?
So, the first step now is to shut down Qlik
Sense. I have a Stand-Alone system. So, I do not have the Qlik Sense Repository Database
Service, my database is on Qlik Server 3; then we have the Service Dispatcher,
which we need to stop at the end.
Okay, now the Qlik Sense is
stopped is it time to back up?
Yes, up first, we open the MMC to back
up our certificates. We click here on File add/remove snap-in: certificates: add my
user account. So, you want to be logged in as the Service user and then you want to get
the personal account of the Service user, plus the computer account.
Ah. Okay.
If this point here have any restrictions,
then please ask your IT to help you out.
Just want to clarify: the Service user
is the account that's actually running the Qlik Sense Services right?
Yes, and you can see it here. So, if you double-click on the Service, and
you go on the log-on tab; you can find..
There it is.
And we are logged in as domain administrator. So, so now we do have
the Qlik client certificate on the current user personal certificates. We would right-click: all
tasks: Export. We say “Yes,” with the private key, then all extended properties. We don't need the
certification pass; next; we set a password.
This is a password just
for the exported package?
That is just for the key, so that
not everybody can just import the key again. Then put it on the desktop; and now
we call this one: Current User Qlik Client.
That's a good practice. So, you
know where to import it again.
Now we need to go to local computer; and all Qlik
issued certificates are issued by your server name under the installation minus CA at the end.
Cool.
So, here we have two: we have Qlik
Server 1 and Qlik Service Cluster; and Export it with all the properties.
Okay, password again, now…
You're just creating a password
when you type that in?
Yes, I use 1-2-3 for simplicity here. So, we call
this one here: local computer personal Server.
Okay.
Finish.
And I see: just repeat the process
for all those Qlik certificates?
Yes. This cluster.
All right.
And now there is one more which is under
Trusted Root. You just press the letter of the server name, like in my case it's
Q and you jump right into the list.
Okay.
You see that this one has a private key attached. In front, we
have this little key icon. So, we do want to have the private key exported here as well.
These check boxes are really critical.
Call Trusted Root. So, we are
done for the MMC part here.
All right, we've got all the
certificates exported; what's next?
Next if you have your Qlik Share locally
on the disc, you would back up this one and move it over to the new machine
in our case we have an external. So, we can skip that step. Then we have the database.
Yeah, we've got those four databases: Licenses, QSMQ, QSR, and Sense Services. What
does the documentation say about this?
We see here we would take a backup of all
the databases we have for that installation.
Okay, this is listing all these commands
to run through a command prompt?
Yes. So, if you would have the Embedded Database,
you would run it from Program Files > Qlik Sense Repository > PostgreSQL. If you have as me a
Stand-Alone database, then it's usually wherever you have installed it. So, the standard is Program
Files > PostgreSQL and database version and Bin. You can copy the file path.
Great.
Open a CMD prompt as admin; CD that; and now
the first database is to dump the QSR. Export the QSR database; the user for the export
will be PostgreSQL. So, we have to enter the PostgreSQL password; and then the file QSR
backup should be created in the C drive.
And there it is. So, just repeat for all
the databases using the commands from the documentation. Is there a simpler method?
Yes, if you do have a Stand-Alone database, you have a second way. You go into properties
here; and you just rename to Licenses2; QSMQ2.
Okay. So, by renaming you're basically
breaking the connection to the old system.
So, I will create fresh databases right after
that step. Installation on the second machine into a fresh database; and at the end, I just rename.
So, the new system will talk to the old databases; but I skipped the whole import/export set.
Oh, this definitely seems like a easier way to do it.
The first database called Licenses; and then select the user Qlik Sense Repository;
we go to the definition here. It's important to select Template0. We now repeat this step
for all the other databases: QSMQ, Qlik Sense Repository. The Qlik Sense installer will not
create databases in the stand alone instance.
Ah.
If you are Stand-Alone, you have to create those databases up front, and that
is why we walk through that process together.
Okay. So, you’re just creating
databases with the exact same name?
Exactly, now we have have
completely fresh databases set up.
Great, what's the name of the
machine in our new environment?
So, that one is a complete fresh machine: Qlik
Server 2. We have to start with the MMC. So, right now we're on the new server where we don't
have any certificate yet; but we need to restore the certificates before we install so that it
uses those certificates during the installation.
Okay. So, that's the first
step before we even install?
Yes.
So, the plan then is to take all those certificates that you exported
previously and import them to this machine?
Yes, we go to current user – personal, and
we import the certificate now. We browse in SenseCert; we have to change this here to PFX; and
then we see here current user Qlik clients. So, we open this one up. Now it gets
important. You type the password; you do not check the Enable Strong Key Protection
feature. You need to check the box Mark this Key as Exportable; and you want to include all the
extended properties. Next, already in this store, and we select Finish.
All right.
See now, it created certificates; and we do have a
Qlik client; it will still say Qlik Server 1 which is normal because we import the old certificate
from the old server even if we are now on Qlik Server 2.
Okay.
Now we go to the local computer – personal;
and we do the same thing: import; PFX; say it should be exportable; finish. We can check now:
we have the Qlik Service cluster; and we take one more. So, now when we double-click on one of those
certificates, we see: yes, it has the private key, but it says: I don't have enough information to
verify this certificate. And, yes, the certificate pass isn't there because we didn't put the root
certificate yet. So, we go to Trusted Root.
Okay.
Certificates > right-click here; all tasks; import; PFX; Trusted Root.
Same drill here; and Finish. Refresh once, in this list we do have the Qlik Server 1.
So, now I guess we get the installer from Qlik Community > product downloads page?
Yeah, we can say latest release and patch; and we know that we need February ’24; Qlik Sense
Enterprise on Windows; we have the setup exe, and the last patch. Now we have downloaded our
files. Make sure that the unblock box is checked; every time it comes from the internet, you might
see the unblock, you might not see; it it's depending on your IT policies. You right-click
here, and you say Run as an administrator. We will on purpose, open Task Manager just just to
show there is really no Qlik Service installed here on this machine.
Okay.
We always get those kind of warnings messages:
your MMC is open, your Task Manager is open, please close it before proceeding. We close the
MMC, the warnings go away. You can now; sometimes it still says it's running, then the trick there
is: you check here the users tab and you might see another user logged-on on the server.
Okay, good thing to be aware of.
Okay, we close the Task Manager now again, and we
hit Next; Install; accept the license agreement; since we're installing a new Central
Node, we create lust we leave the name as it is: Qlik Server 2 domain local.
And is that just the name of the machine?
Yes, that's the name of the machine. We do
have the option to get a new database created; that would be the Embedded Database option.
Okay.
And we have the second option: to connect
to an existing database. Since we know our databases on Qlik Server 3, we use that
option; standard port for Qlik Sense: 4432 or whatever your admin tells you; you
can change it too. Database user; password…
That all important password.
Now it will try to connect to the databases; as you see, it could. If it couldn't connect
or one of the databases here would be missing; you would get an error message.
Okay, but it actually created a connection to those fresh new databas that
you created yourself with exact names.
Exactly.
Okay.
In this one here, you have a couple of additional
settings; leave them default; point to the root folder; in our case, we make it simple: the root
folder never changed; which is the root folder: apps, archived logs, and Static; and if we go from
this machine to that one: they are here. They will not be overwritten during the installation so all
your apps be the same after the installation.
Is that a good step, to kind
of try to navigate to that location from this machine just to make sure?
Yeah, test in this up front given again you're logged in with the Service user is able to access
the share from that server you are currently on.
Okay.
Now you choose the installation folder; and now because we
are migrating, we are unchecking this box. We want to change back to the other databases;
and we hit Next. And we will get the Bundles customer agreement again; and we hit Install.
So, once this is finished, we still need to run that bootstrap command that changes
the names from the old server name to the new one?
Yeah.
And test everything out. I like
how it tells you what it's doing.
All right, we are installed.
Good.
And you set it to leave the Services
unstarted. So, everything is just waiting, what do we need to do next?
And the next step is to go back to the database machine. The QSR database is
basically still empty because Qlik Sense hasn't started up while 2 which is our old database that
actually has all those those tables already.
Okay. So, we still need to take that step of
restoring the old database to the new system?
Exactly, now we rename this one to license one
and now you see why I choose the two before. So, I can keep them apart but now we go back
to the databases with number two and just rename those back to the original name you
could of course also delete the databases and rename them I keep them here simply
because of roll backck purposes that's a.
Great, tip I definitely believe in renaming
instead of deleting is it time to start up the Services on our new machine we need to start
the first one which is the click Send Service Dispatcher and that one is the foundation for
the Repository to fully start up. So, once that is done we open a CMD prompt as admin we need to
navigate to a specific folder and here we need to have the Repository EXA which is this one.
Okay, okay copy that folder go back to the command promt paste all right. So, we're in that
directory what are we going to be doing here?
Yes, we are going to do a backup restore
procedure if we scroll do something if your machine has a different host name in our case we
had Qlik Server 1 and now we have Qlik Server 2.
Okay. So, we have to go in and change
the host name and all these settings and it says here restore the certificates to
security Qlik Sense Services we did that.
Okay, install Qlik Sense on the computer we did
that too start the Repository Service database in our case we have a Stand-Alone database. So, that
is already done. So, which Step are we on there is this tiny bootstrap command oh and step seven.
Okay. So, we copy this one and we paste it here you will see it immediately goes away but
then it should usually come back setting up all the certificates to all the correct
ports and. So, on all right. So, that's going through and adjusting the certificates
adjusting entries in the database as well?
Yes, it said Qlik Server 1 it's replacing it with
Qlik Server 2 yeah. So, in this case here close that window we restart the Service Dispatcher
once to get the new certificate in place. So, can you clarify for me again what we're checking
here the bootstrap command changed the name of the certificate now to Qlik Server 2 is now it
does it is Qlik Server 2 issued by Qlik Server 1 now we start the Repository as a Service we
can also start the other Services as a Service is it important to start the Repository first?
Yes, the Repository is the base Service for all the others now the important thing is to watch
that lock file system Repository I see it has a error contact in the license server well in
that case we first need to see if the license Service actually started up it's always the one
with the later time stem. So, in this case that one it is actually started up to listen on
Port 92900 but it cannot communicate with the Repository Service this is actually kind of.
Great, we get to see how you would troubleshoot this. So, you're opening up net stat to check the
ports first I want to see is Port 9,200 up and that one is then I want to see is
Port 42 42 up and that one is not. So, there's some issue with the Repository then?
Yes, one of the reasons. So, would you just restart it or what would you try that's a
good question we can try to stop it in any case we should have a look at firewall settings
firewall is off. So, that it's not an issue.
Okay, then the de here is most likely our problem
we did had one Repository Service still running and that might have been our command line oh?
Yeah, mode. So, now let's start this one again there we go. So, now 4242 is working at least
sending something to it but we still need to see if this thing starts up. So, we need
to go back to the Repository system lock H keep ret trying. So, let's see we all right.
So, we restart the Dispatcher Service that tries to create that connection to
licensing server and check the log.
All right, we got a new log file listening on
9200 and we see now all the files with the data left. So, the restart of the Service Dispatcher
actually fixed it the thing is started up now it actually says I cannot get the other Services
is it not started correct and what we also can see is we see startup phase completed which is
thing we want to see. So, now back to Services we start all the other Services up.
Okay. So, the cause of this issue in this case was a command prompt that was
keeping the Repository Service open and once we stopped that and restarted the Service
Dispatcher things started to work as expected?
Exactly, and we might have to press control C and
just closing the command prompt might have been holding back something for the license Service
to restart now we should have a couple of Qlik Sense icons and we could for example open the
Management console we should be able to use the same user as before domain administrator this
it looking good. So, far this is working. So, we see Qlik Sense February is up the
app list should show the app test one.
All right, the same apps we had on the previous
environment that's good now go to the hub we have test one and we had the monitoring apps that's?
Exactly, as we had it before and we can open this app which means our engine is working.
All right, that seems like a successful migration. So, what you now start this data
load you check are my data connections all still well it and working are tasks working
or do I get failed tasks and I have to invest why is this not working on the new machine M
reasons for that is for example obbc drivers which you had on the old server which you didn't
install yet on the new server but the app itself tries to connect to a standard obbc driver?
Yeah, there could certainly be a few more details specific to each environment but now let's
move on to Q&A please submit your questions to the Q&A panel on the left hand side of your
on24 console we've already got a few questions coming in I'll just read them from the top.
All right, first question: after a migration, if work is accidentally continued on the
old system, is there a way to migrate the additional work from the old environment to
the new one or what issues might that cause?
It could potentially cause some issues if they
used an Embedded Database with Qlik Sense.
Okay, you would have the database on the same
machine as Qlik Sense is installed and then we would have one database on Qlik Server 1 and a
second database on Qlik Server 2 in our case here we have both servers connecting to the database
in Qlik Server 3 that's right it's a Stand-Alone.
Yes, you would get other problems because
you potentially can't even start up. So, they would have to remigrate the database.
Okay, All right, next question: is it possible to export and import multiple apps at once?
Yeah, import is I think still one by one.
Cool, could you show us where to do that?
We do go to apps and now you see you have a couple of them. So, we can just go here then we say
more actions export export with data and Export.
Okay. So, it's exporting all those apps
you had highlighted in one bulk run?
Yes, it just puts them in a folder you see
the folder displayed here C program data Qlik Sense Repository Export. You see that was the
export I started and here are all my apps.
Very cool. Well, next question: how to apply
a patch? Is this necessary on every node?
Of course, if you have a multinode system you
do need to apply a patch on every node. So, your first step here is shut down all the end
Services it makes installation of the patch much easier I have prepared here a little Qlik Sense
update. So, we can run this one as administrator which is the February 2024 patch 2 update and
here we are patch successfully applied. Now we need to wait a little bit again and then we should
be able to open the QMC. that was a pretty simple process here we are now put your eyes on the
bottom right click since February 2024 PCH two.
Great, well that was a pretty simple process
and we need to repeat that for every node in a multi-node system?
Yes,.
All right, well let's move on to the next
question: what if we need to upgrade Qlik Sense as well? Should we migrate first and then upgrade
or should we upgrade first and then migrate?
That's an interesting question of course that
might come up you have to stay in certain boundaries never go from a four-year-old Qlik
Sense release to the latest one but if you are one year back for example you want to migrate from
February 2023 to February 2024 that is possible installing directly February 2024 and then just
putting the database from February 2023 back just make sure never started up at the beginning that's
the most important step to check the box of do not start the Services after the installation.
Okay, it's nice to know that that's possible but for best practice would you suggest
migrating first and then upgrading?
Yeah, I would go to the same version and
then upgrade slowly to the next one.
Great,.
All right, move on to the next question: what are the prerequisites to migrate
an app from On-Prem to Qlik Cloud?
Okay, for this we just open the QMC up again.
If you have a signed license key and we do have one in our system mhm then you can set up
a cloud deployment. So, you have here the option Cloud distribution you will not get
this if you have a left in control number then you just simply don't see this option.
Okay, in this setup you can add your Cloud tenant if you have one if not you
can talk to your account manager.
Okay, some distribution policies and you can then
have a look at all the apps which are already in the cloud. So, basically the the simplest rule
is everything which get reloaded On-Prem and is published you would push in a copy to the
Cloud as well ah and then in your Cloud tenant you can assign some rules to say that this app
always goes in a specific space for consumption your users every time. So, usually you would need
some Cloud accessible data sources to reload your apps in the Cloud or you play around with Qlik
Gateway but the best approach here if you need to move your users over and you still have some
data On-Prem and some may be already accessible from the Cloud system would be to reload On-Prem
as much as you still need and then just publish this app for consumption to the Cloud.
That's a good suggestion, I like that.
All right, next question: what about
migrating our reloads and custom reloads?
Yes, the whole task list in our case it
were the two operation monitors they will get automatically cuted over because
that is all in the Q is all database.
Okay. So, that is part of what we did migrate?
Yes,.
Great, good to know.
Okay, next question: can we create a miror image of the Qlik Sense client manage machine
as installed and working there and then reuse it on a new machine which is a newer version of
Windows operating system or other new features?
Can you get more specific on the mirror image?
It seems like they're looking for just an exact copy of their implementation on a different
operating system maybe to test it out.
I suppose the easiest way here would be
if you take the QSR back up and then you put it on this new machine to test the
behavior you can always do that you can then just apply a license on the new machine
and you would be able to use that machine.
Okay. So, that's the real roadblock is the license
for having two machines with the same license?
Exactly, there there's not an automated
workflow to create a mirror image of the exact installation and then put it over
to another machine you would still have to do the installation Step at some point and
then move the database to the other machine.
Okay, next question: are monitoring apps fail to
reload after migration any tips or suggestions that sounds like could be complicated.
It's a good question. So, let's go over the obvious points: you see in my case they just work
mhm the reason here is in my data connections for monitor apps users is most likely the first one
which comes up you have a very long connection Local Host is the machine name I use which
works on every machine even though I have a different host name. Now you might not have
specified localhost here you might have had Qlik Server 1. domain. local and…
the qualified domain name.
Yeah, down here in the in the proxy you would
have the same in Qlik Server 1. domain. local in that case you would have simply to change
this one to Qlik Server 2 since we migrated to Qlik Server 2 then you would take this whole
string go back to the data connection you paste it in here now and you can hit apply.
So, now we changed that connection.
So, that connection string in the the task that
doesn't get updated with the bootstrap command?
That does not get updated because your
data connections stay untouched.
Thank you. Okay, next question: how
would this differ from moving from on Prem to an AWS machine. So, that would be?
Exactly, the same migration my system here is based on theend there for this demo. So,.
Okay, in AWS of course you have to spin up your machine first but other than that it's a
Windows operating system and then you can dig in and install Qlik Sense on it.
Okay, next question: is there a similar process for migrating NPrinting? I
guess a lot of customers that are On-Prem are certainly using NPrinting.
Let's ask the Oracle. NPrinting, this here backing up and restoring NPrinting
that is most likely the one we want to have.
Okay.
Here are much more restrictions: NPrinting can only be restored in the exact same
version and if you wanted to migrate you must follow the instructions on on upgrading.
I'll be sure to include these links because this is good documentation.
All right, we have time for one last question: is there a different process
for migrating our Web Connectors QWC?
That's a good question they worked really hard
to get most of the web connectors now included inside Qlik Sense itself. So, before you might
have used the click web connectors different tool than Qlik Sense but nowadays you have the
web connector list for SharePoint click cloud storage S3 buckets and Dropbox on that side side
and you also can access the metadata storage from S3 buckets for example on the other side and most
of the big query connectors and. So, on are now available inside the product itself. So,.
Okay. So, most of those things are actually built in now. So, if you're still using web
connectors you're probably on an older system.
Or you have one specific connector which
hasn't been ported yet. In this case, as simple as just installing up the web connectors
in any case you have to install this one on the second system as well might be as simple as
literally just replacing the config files.
Yeah, most of the settings are stored in
deploy config which you can copy and replace.
Okay. Great, well Sebastian this has been really
great. I hope this will be a good and valuable resource for people going forward when it comes
time to migrate their implementation. I appreciate all the time you put into preparing this.
Thank you so much; and if there are any additional questions, feel free
to reach out in the community like I will try to answer them as good as I can.
Thank you everyone. We hope you enjoyed this session. And thank you so much to Sebastian. We
appreciate getting experts like Sebastian to share with us. Here's our legal disclaimer. And thank
you once again. Have a great, rest of your day.
