>> What's up, you-all? Do you use
Azure DevOps server or services? Have you ever needed
to move work items to different projects,
subscriptions, or servers? If so, then you don't want to miss
our next episode of DevOps Lab, where Martin Hinshelwood will talk to us about the Azure
DevOps migration tool. [MUSIC] >> Welcome to the DevOps Lab. Today we have a very special
guest and my good friend Martin, and we're going to be
talking about moving work items around in Azure DevOps. What is up? Martin, how is it going? >> It's going pretty gates. I'm stuck in lockdown in the UK. We're in total, full,
complete, utter lockdown. You' re not even allowed to not go beyond five miles from
your house at the moment. >> Wow. We are not quite in
lockdown in the States, but I haven't set foot out of my house for, I think
about a year now. Before we dive in, why don't you tell our viewers a little bit about yourself, Martin? >> Yes, sure. My name's
Martin Hinshelwood. Actually, I'm a professional
Scrum trainer which [inaudible]. I'm a professional Kanban Trainer
and I'm also a Microsoft MVP. I would normally say in DevOps. There's not really such a category, but I like to pretend there is. But I've been at Microsoft
MVP for 13 years I think. >> That's how I met you. >> That's how I met you as well. >> That's right. Moving work items
around, what's up with that? What's the problem? How
can you help us with that. >> Well, it's actually a problem that Microsoft has actually
tried to solve in the past. We had a work item
migrator before that was very complicated and lots
of people had trouble with it. It actually migrated
source code as well, which is probably what made
it the most complicated. But lots of people want to move partial projects from on-premise to Azure because they
can't move everything. They're not allowed in
their organization. Or maybe they started down the route of one team project to rule them all. Unfortunately, I think
I started that one. But one team project
to rule them all. Then they realized this
is a bit too big and they want to split it out
into smaller projects. Or they went the opposite route and they had loads of people working on one product in multiple team projects and they want to put them into one. They want to merge them all together. >> Yeah. The big problem
that we have right now, or for a while, for a long time now, is that people
that are using Azure DevOps, whether it's server or service, once you've create your project and you started adding
work items in there, it's really hard or almost impossible to move them to
either another subscription, another server, or even
another project, right? >> Exactly. Since I built my tool, there are a number of things
that have made that easier. You can move work items between
projects inside of the same org, but you have to do it
one work item at a time. It's good for I made a mistake and
created it in the wrong place. But it's not good at
for I've got 50,000 work items over here and I would
prefer they move over here. >> Yes. You've got a tool that can help with this migration
of work items, right? >> Yes, I do. I actually
started work on it in 2016. >> Cool. >> Worked for quite a while on it. I decided just to make it open source because then I maybe get some help, maybe get some feedback
from other people. Over the last few years, I can't do math anymore, I think it's COVID brain. But since then, we've had actually over 80
contributors and like 160 pull requests come in for people
making changes, making fixes. I know a number of large
consulting organizations that I'm good friends with
that have taken forks of it and do their own thing as well. I think it's just filled a little niche and been
very popular with people. >> That's awesome. Why don't
you show us a little bit, show us how we can get the tool, how we can use it, things like that? >> Yeah. No, that's awesome. I have up here the Visual
Studio Marketplace. If you're in the Azure DevOps site, you can look up at the
tools that I have built and a couple of them are to do with process templates and processes. If I go into the migration tools, which you can just search for
migration tools and you'll find it, there's really just
a placeholder page. You don't integrate
it with your system. You run it separately. It has to be a very long
running tool. I don't know. Have you ever migrated 50,000
things from one place to another? It generally doesn't happen quickly. If you think about the way
a work item is constructed, you've got revisions for that work
item, you've got attachments, you've got links, you've
got all kinds of things going on and we need to bring
all of those things across. If you click on "Get Started," it
will actually just take you to our documentation and it'll take
you to the Getting Started page, and you'll be able to install
it off of Chocolatey. That's the easiest way to do it. You can just go download the source
files, not the source files, the binaries from our Git repo, but going to Chocolatey is
the easiest way to get it. But I do know that some people
don't really like it very much, but it is there. If you're inside of a firewall, you might need to get somebody
else to download them for you. That has happened before. But if I go to the Git repo itself, you'll see that we've actually had 437 releases of our
product over the years. You can just go download
the latest version. Once you have it installed, it is not the easiest tool to run. I do have a whole bunch of warnings on here around it's not
designed for a novice. You better know how the
APIs work before you start. Something we find is this
is a complex problem. People have lots of weird stuff. They've done NTFS in Azure
DevOps over the years. There's lots of different versions. Sometimes you need to
tweak the code for exception handling because it's really built and supported by
people who are doing migrations, so that it only copes with
what they've had to deal with. But it is fairly straightforward. We actually use a JSON
file to configure it. You run the JSON file and it
does all the bits for you. If you go to the Tools folder, which is tools/migration tools, you can just run
migration.exe and tight init. It will run the tool, and it will create an empty
configuration file that's all set up for us and more
or less ready to go. While that's doing that, which it's finished already, I have a here's one I
prepared earlier, enabled, like any good cooking show, we need to have something prepared. This is the config file. It actually has a check
in it because we do change the config file
format quite a lot. It does have a version check in it. You need to go update that
manually if you're updating it. But we setup a source server, where are we going to
get the data from. Here, you just put in the details. The source works with any version of TFS from 2010
onwards, all the way up. You might have trouble with TFS
as a target if it's TFS 2010. But I think that's 2012 onwards. But as a source, you can do 2010. This works for those
really old migrations that you don't really want
to touch the server as well. We have three modes of
authentication in less access token, which is the most popular one now. But you can also do prompt. You just get a prompt to login using your Azure credentials or your desktop credentials and Windows login and for
those people on TFS. Yes, that is a real
personal access token. I will be deleting it after ritual. Then your source at your target, it does support TFS from
different languages, as well, as well as a little
language map for the words we need that might be in a different language
inside of the system. Then you set up one
or more processors. Here, I've got the very
default processor. I have barely changed it from
how it came out of the box, but there are a couple of things
that are very useful here. You can choose to replay
revisions or not. If you've got a lot of work items, replaying revisions
can take a long time. You can slow things down, so you can configure everything here, and you can put in a little
WIQL query here as well to help you control the scope
of your migration as well. I quite often, and
this is a good tip, I'll do all the open
work items that have been changed in the
last 90 days first. Run through that, then people can get working while we work on the backlog of things and leave all the closed stuff to last because most people
don't really care. That's the majority,
people don't care. There's so many different
options in here. I'm definitely not going
to go through them all. A lot of them are self-explanatory. They're reasonably well-documented. I will caveat that with reasonably. Good things like, do you
want to migrate attachments? Do you want to do links? All of those are enabled by default. If I were to run this
particular Config File, this is the one I've set up for you, just a default migration, a reset. I'll show that in a second. It's actually going to
migrate work items from one project to another project inside of the same account
just for ease of use, so I can just set up a PAT token. This is my test environment. There are no work items in
the target environment. I'm actually just going to run this, I will let it run, but it's going through
a couple of things. It's going to load all of the
work items from the source, and it's going to create
the areas and iterations, and then it's going to iterate over all of the work
items and create them. I don't know if you're able
to spot the stuff going past. It's doing links, it's
doing attachments, it's doing all of the things
you would like it to do. >> Cool. This is a
real deep migration, but you're going to all the links, all the attachments, all the history if I wanted to or
not if I configure it. So this is a real migration? >> Yes, absolutely. There are some caveats to that because
a lot of stuff is still best effort because that's just how the APIs work,
but not migrated. There are only seven
work items there, and it took 36 seconds, so you can imagine if you've
got 50,000 work items. Most of these work items
really have no more than, I think Not a Bug has 16 revisions, but I've worked with
companies where they have 500 revisions on a work item. It takes about two seconds per save. If you hammer the TFS
APIs too heavily, they will throttle you. It will still continue to work, but they will throttle you. That was a default migration run. Now in here I should have. Cross your fingers. I've now got a bunch of work items, and if I open up feature one, you will see this is
the test for Abel. There we go. There's my
child work items linked. If I go into the history, well, I'll need to zoom out a little
bit to be able to see that, doesn't work at that zoom level, but you have the original save and then a bunch of
different changes, and you can see that the dates are the date that the change
actually happened. I changed the description
for you Abel, at about 9:30 tonight, which was a few hours ago in there. You can see it migrants
all the work items, it does all the comments, all of the attachments,
all of the links. I'm not sure if I can find
one that has an attachment, but trust me, it does do that. One of the things that
I think is very useful is if I just run that
migration again, it's actually not going
to migrate anything. It looks up all of the work items
as it goes through and it only migrates revisions that has
not migrated in the past, so you can have people
continue to work. It will not work two
way, only one way. It's a migration, not
a synchronization, but if I change one work item, it will migrate that
one work item as well. That's a simple migration, Abel. I'm on the Scrum template over here. I want to be on the Scrum
template over here. I've made zero customizations,
so everything's good. If it's the same either side, it will just do it, it iterates
through all of the fields, but if you have what
most organizations have and some crazy person has customized
their process up the wazoo, then you might have to do a bunch
of things with mapping fields. In fact, we have a number of different field maps available
that you can create an ad, including one that's just Regex. Create a Regex to
figure out what you're getting out of this field and what
you're putting into that one. Merging fields together,
changing drop-down lists states, all kinds of stuff, so we
have those capabilities. That's always a bad idea, customizing your process, but people
do it, so that's interesting. Then we have a number
of other types of migrations as well that we can do. If I just open up the
documentation there, because that's the best
place to find this. I don't really want to
show all of the governs, but we have a number of different
processors that are available. The one I just showed was
the Work Item Migration, but we can also migrate
teams and team settings. We've got 30 teams on this project, you want to migrate them across. You can do Areas and
Iterations on their own. You can do Work Items. I've got 50,000 work items I
want to delete, you can do that. You can update work items in place. Instead of doing a migration update, you can just overwrite
work items in place. That's really good, Abel, when somebody's done a bulk edit
and broken 50,000 work items. I've seen that happen as well. >> I may have done
that once or twice. >> Yeah, me too. I actually
had to write a time machine once for a customer because they overwrote it and
we needed to regress. We can also migrate shared queries, and test suites and plans as well. >> Nice. Very nice. >> The thing we struggle
with is test results. We can't do tests results, but suites and plans
we can bring across. I even have a tool in there
that'll allow you to import pictures from Azure Active
Directory when that doesn't happen, and do attachments and various
other things on their own as well. >> Wow, man. That is
super cool, Martin. Look, if you use Azure DevOps Server, or Azure DevOps Services, or you still happen to be
on some version of TFS, you know how painful it can be trying to move work
items across projects, across subscriptions,
even across servers. It sure sounds like
Martin's Azure DevOps migration tool can be just the
solution you're looking for. For more information, check
out those links down below, and join us next week
for more DevOps Lab. [MUSIC]
