Drupal migrations for non-coders / DrupalCon Global 2020

okay it is time already so thank you very much for joining us at rupakan global this is drupal migrations for non-coders my name is mauricio dinarde i go by dinnercon on the web and my pronouns are he him the slide deck is already available in that url and i'm going to share it at the end as well everything that i'm going to present is already available including the code examples that is my email address if you need to get it uh you know if you have questions after the session feel free to email me i am from nicaragua beautiful country 85 degrees all year round so if you live in a cold place you can come down here beautiful nature you can go watch a lava lake like this one or you can go to the website for for a visual tour i collaborate with our organization cola guy which is based in boston we have you know members all over the place i myself in nicaragua and we specialize in migration support migration so i invite you to check our upcoming trainings next week but even you know after those dates we usually present trainings i am also very passionate about teaching so i have this personal project called understand drupal.com where i uh you know write articles and in the future videos as well about drupal in spanish english and french so if you speak any of those languages and want to learn more about drupal and particularly about drupal migrations i have been writing a lot about that in particular uh i wrote a 31 a series of articles about migrations and i also have a course on the topic so i invite you to check it out and let's get it you know to the topic uh we're going to talk about drupal migrations and even though we're going to approach it from a non-developer perspective if you have some experience from with other platforms in particular anything that is really related to etl or extract transformer load if you are familiar with that drupal migrations is an implementation of that api so feel free to you know if you have knowledge about this already you it applies as well and i like to explain things by giving real life examples so let's have a a look at how we would use the migrate api with an analogy of making breath so when you want to make breath you need some source ingredients or materials so in drupal migrate api that corresponds to source plugins that means you need to have a source of your data is it a xml file a json file a csv file a sql database and so on so you need some source of you know raw materials or in this case the data that you want to get into drupal and those are provided by the source plugins and it corresponds to the load operation in the etl pattern after you have collected everything that you want to migrate into drupal you need to process it and transform it this is corresponds to the transform operation in drupal is provided by process plugins and we're going to see some examples of that later but basically it's a way to mix and match for example if you have games that you want to combine like different first name and last name and you want to combine them in a single uh column or a single fill in drupal you using process plugins is how you would you are going to do that and basically any transformation that you need to do this is how you do it using process plugins and the load operation in the itl pattern corresponds to the destination plugins and this is how you instruct drupal what entities do you want to create do you want to create users or notes or files or media entities or paragraphs and so on so this is how you install you know this thing that i already collected and processed put it in this basket or in this shelf and this is destination plugins and all of this we're going to see practical examples so when i started working with migrations i had been doing drupal development for about five years and even writing code and so on so i remember that i thought you know this is going to be easy and then i was from with so many words so many things that were very was were very confusing to me and i was a little bit frustrated you know in the end i just wanted to move some content into my website and everything started to click when i started to map all these buzzwords that i didn't understand at the beginning into these process plugins and particularly into one of these categories this is a source a process or a destination plugin because in every migration you are only going to have one source and one destination so instead of having to think about 15 things uh i only care about seven for example using the the process that the processes that we see in in on the screen right now so let's say i'm going to work only from csv and i'm going to create users so i don't need to worry about any other source or any other destination in between i am going to use one two five you know as many process plugins as i need one thing to note and clarify is that when i say a migration can only have one source and one destination that is like for example if you want to migrate the article content type or the basic page content type but in your migration project you're actually going to have a lot a lot of migrations so some of them might be creating notes others might be creating users and in the example that we're going to see later uh we are going to see that you know every migration has one source and one destination but to migrate the whole project we're going to be using different files so that's one thing to take into account so this is a very simple example of a transformation that you want to do let's say that you have a csv file with first name and last name as separating columns and in your destination you you you have a node and you have a name field but you want to put them together so there is a process problem that we're going to see an example of this later that is going to be able to put them together and go from source to process to destination and follow that workflow before jumping into the examples there these are the most common things that i uh see people you know having difficulties when i started with migrations the first one is that any migration in drupal is a two-step process first you need to write the general file the german definition file and after you have the file you need to execute it and this is where things started to get a little bit complicated or confusing because to execute the migration you have two options from the command line if it is a migration defined as code or configuration or from the user interface and and we're going to see what this is so this is a very simple migration and this is like a fully working example this is what a migration definition file looks like it has an id which i uniquely identified each migration in my whole project it has a label that allows me to identify this migration in the ui and in the command line if i were to use that and then we saw the three step of the ipl process my source my process and my destination uh in my source in this case i am reading from a csv file and as you can see it's like three lines and with that i am able to read from csv file in my destinations i'm saying i want to create notes of type ud stuff this default bundle refers to the content type it can be article it can be basic page in this particular case it is called uds staff and then in my process section is where i do the mapping so in this case i am saying the title field of the node i want to set it to whatever is the first name value of my csv file i have a column in my csv file called c first name so on your left you have the destination either property like the node title or field let's say the image field and on the right you have whatever comes from your source um now we we said that migrations can be defined as code and configuration and this is very important to understand migration defines code they work out of the box with drupal core you don't need anything else to work with them and they in order to use them you create a custom module and in that custom module you have a migrations folder and then the name of the file itself is going to be just like the id dot yml you can say you know ud staff csv image.yml if you make any change for that file you need to reveal the caches for those changes to take effect let's say that after migrating the title i want to migrate uh a field then you need to clear the caches for those changes to take effect and many times you also need to roll back and import again and we're going to see an example of this in a moment and in order to execute the migration remember that this is a two-step process you have the file already but you need also need to execute it so you do it using the command line and there is a model called migrate tools that provides rush commands to be able to execute those migrations another option is to have migrations defined as configuration entities and for this you need the migrate plus module enable and install on your website here the place where you put the files is different it's going to be config installed inside your custom module and the main pattern is also different it's going to be migrate plus that migration then the id of your migration and then yml because these are configuration entities this is basically how you manage any other configuration in your website if you want to make any changes you need to import or sync again there are many ways to do that there are commands that allow you to do a partial configuration import there is the migrate config devel module that allows you to automatically import any changes as soon as you make them or you can use the interface or you know any cmi related module that allows you to you know import again any changes will help you with this and only migration in finance configurations are able to be executed from the user interface and that is using the migrate tools modules which we're going to see later again very important to understand the difference between code and configuration migrations because if you put them in the wrong directory or with the wrong name or you are using the wrong modules you will not be able to execute them and in general some tips before jumping into the example start from an existing migration it is often the case that you know you you have some use case that might be common so you just like look for an example make sure that the example works as expected and then start making your changes to adapt to your specific needs uh in the 31 articles series that i mentioned before there are about 20 examples of many different things that you can just literally copy and paste and from them start modifying to your specific needs in general read the official documentation in my humble opinion the migrate api is one of the best documents subsystem in drupal as a whole pay close attention to the syntax of your general definition files even like an extra space or if you remove a space or the indentation is wrong the whole migration might fail and that is a common source of frustrations and sometimes the errors that you get are not very descriptive so make sure that when working with uh migrations you know the white space is the syntax that you use in your general files is proper when having a big migration go one field at a time because if you have let's say 10 fields at once and something breaks you might not know when it you know in which of the 10 there is a problem but if you go one feet at a time you are making sure that any progress that you make is useful is working and then you can go to the next one if you are familiar with code you can commit every step in the way again like if something breaks at some point you can go back in the history of your git repository and come back to this in a working state another thing is that not everything needs to be modified in the migrate api you might do some cleanup or preparation of the data before or after the migration sometimes there are eight cases that you know you can spend hours if not days trying to make a you know to come up with a migration that takes care of everything perfectly or you know you spend an hour before or after preparing your data or cleaning up your data and you save some development time so not everything is to be done in the migrate api and if you need help uh the migrate api offers a lot of support in slack in the pound migration channel so i invite you to go there and this is probably the most important thing that i can give you uh understand that there is a you know leah author of css secret says understanding the process of finding a solution is far more valuable than the solution itself and i mentioned this because the migrate api is just like a tool set you have so many tools that you can use for many different purposes and it is up to you to combine them into something that works for you take your time to read to understand to play with different examples to break on purpose things that are working so you get familiarized with the errors and once you understand you know why things work and when they break why they break you're going to have a much easier time working with migrations so i'm going to do a live demo of course nothing can go wrong so bear with me so um before starting with the migration itself it is very very important to understand your source in this case i am going to be migrating from a csv file so i need to know what is the structure of my csv file in this case i can see there are you know a a row of headers in this case a staff id first name last name and so on and then i have like one record for each of them yes the structure that i need so always always understand the source in this case my csd file um so this is the example that i'm going to be starting with as as you can see all that you need is an id a label a source a process and a destination so this is the very basic uh that you that you need my id needs to be unique across the whole system um and remember the spaces that i mentioned before if i were to remove this space it will break my whole migration will break so be mindful of that then my i am using a plugin called csb this is provided by a module called my resource csb and then i need to provide a path to my file so this is the path if i were to inspect the hierarchy of my drupal installation this is where i'm going to find it and then it expects an id key with a list of columns in my csv file that uniquely represent uh each record in this case i have a staff id and spaces are important and if you have a spaces you need to put you need to put quotes around it uppercase and lowercase are also important and something that is not obvious here but different version of modules might introduce changes in the configuration so this is the latest version which is 3.x if you were to use version 2.x this value is not ids it's actually keys so be mindful of the versions of the modules drupal everything that you use you should be using the latest version unless you have a very particular reason why not to do that in this case i'm creating notes this is how you say that you want to create notes and this is the content type uh most of the time the migrated api is going to be working with machine names so this is the machine name of my content type and then i am saying uh the title of my note populated with the first name column of my csv file so now that i have my migration files and if i see the structure i can see that i have my custom module i have a migrations folder and that's the name that i define you know i already have my module enabled so i'm i don't need to enable it again i'm going to be running uh the migration with brush just to save some time but you can do it from the from the user interface if it is the finance code and i'm going to show an example of that in a moment so this is executing the migration if i go to the website now i can check that i have this notes migrated uh what i see here that i see more information is because i already have uh migrated other content so for example if i want to start from scratch i go back to the command line and do a rollback operation and i say i want to roll back everything and this is very common when you're doing a migration you are going to often be like importing and reverting importing and reverting and so on so in this case if i just execute my one migration that i defined before i can see that i am getting the the results that i am expecting something weird is happening here and it is that i only said in my migration that i want to migrate the first name but i'm getting something else so the very first thing that you know you do if you are getting a a a result that you're not expecting is like making sure that you are working with the latest version of your files in this case i'm going to rebuild my cache because it's a migration defined as code i'm going to roll back migration sometimes you can use like shortcuts or url aliases for you know rolling back or important and if i refresh i can see that now i'm getting the results from my csv file the reason why i was getting something different is because i hadn't i had a previous version of that file loaded into my drupal site and that's why it was you know showing something else so by doing this i already have a migration working i am already importing content and if i were to roll back the content would disappear so let's say that i want to do something else i want to put together the first name and the last name so how do i do that i do that using process plug-ins remember that if i said i want to combine two columns into one field or one property i use a process plug-in so um for the sake of the example you just go to your search engine of choice and look for drupal migrate process plugins the first results is that documentation from drupal.org and you land into this website there is a list of core migrate process plugins that looks promising so i go there after going there i this everything that is provided by drupal color out of the box one of them says concats and it says concatenates a set of strings in computer parlance concatenation is the same as putting things together so i am going to click here and i come to the uh example or the documentation of this particular process plugin so what i'm going to do is literally copy and paste like i'm doing this just for the sake of the example but i recommend that you actually read the whole thing and i'm going to remove this and paste it here so as you see in the documentation plugin source and delimiters are at the same level but here when i pasted plugin then source and delimiter came after like a sub as a property and this is going to fail if i don't correct this this is going to fail so this is just an example of when you are copying and pasting from the internet make sure that your ide is not doing anything extra in this case i am saying i want to populate that title with whatever is the result of a plugin called concat and then my source what is going to be my source is going to be my first main column and my last name column so again uppercases are important and then you can have a delimiter which is like something that goes in between i don't want to slash as in the example i want to swatch space so if i want a one white space i put you know in quotes and then the white space there so the workflow now to for this to take effect i'm going to roll back the migration i'm going to clear the cache and i'm going to import again and if i go to the website i can see that now i have my first name and my last name um concatenated and if i were to click there i can see the white space in between let's have a look at more examples of the syntax of the migration let's say that i not only want to put them together but i also want to make them uppercase so that is another transformation but there is another transformation that i want to apply on the same uh property in this case the title so how can you do that you can do that using something that is called process blogging change so you do it like this in this case i am going to use a uploading called callback i am not going to go to that documentation for the sake of brevity but basically uh this callback plugin is going to allow me to call any php function if i want and and the way that you configure it you will go to the documentation to know how to do it but basically in a callable configuration you said what what configuration you want to to what function you want to execute so the way that it works is that the result of this is going to be passed to this and the result of everything in the change i can have you know as many of these as i want i could have another one like another plugin in the chain you know they are going to be passing from one to the other to the other and the very last one it is going to be assigned to the property or to the field so in this case i'm going to clear the cache scroll back and import again and if i go to the website and i refresh i can see that everything is in uppercase and that is what that function is actually doing it put it putting things in uppercase uh and let's say that we want if we click on the node itself and we click on edit we can see that we have a reach text field and these are interesting because like in addition to like the text that you will enter here you also get to define the text format so this is uh something that is called a compound field and just like uh what you have to do here is like migrate the source like the value of the field and migrate which text format you want to use so what i'm going to use is i'm going to copy from an existing example so again you need to be very careful with your indentation like any field or property needs to be one level inside of process now what i have here is the machine name of the field and then with the slash this is how you did you access the suffix those are called subfields for the value i am going to use the biography column i have a column in my csv file called biography and that is what i'm going to assign and then for the format the text format there is nothing in my csv file providing that information so what can i do i can provide a default value so there is a plotting called default value and in this case uh that this is the way that you configure it default value and then whatever you define here it's what it's what's going to be used by drupal so what i'm saying is for the field ud biography which is the one that we saw before the value i want to copy from my csv file biography calling and i want you to use the basic html again this is the machine name so you need to be able to find the a machining of anything that you're using if i were to you know come back prepared in my cash import again i can go to the website and verify that i am importing my you know my different biography and if i were to click on one of them i can i can verify that i am using the basic html so uh when this when you have a complex fields like this one that they have two fields that's how you do it you put the machinima for the field and then a slash and then what property you want to set in this case value and then the format let's uh say that we we have a multi-value field how do we do that and again like the syntax is a little bit more involved i'm going to paste and make sure that my indentation is correct so in this case in my csv file i have a column called drupal i have a column called gitlab and calling called twitter and all of them contain urls to different profiles from this person and i have these constants link title so what is that this is something extra that you can define in your source you can define these are called source constants and it's basically some variables or placeholder text that you might want to use later so for any uh url uh from drupal i want to use this text for gitlab i want to use this text and for twitter i want to use this text so the way that this works is like the syntax is the machine name of the field and then because it is a multi-value field you can say zero for the first element set one for the second element two for the third element and so on there are ways to make this dynamic i am i'm showing this like hardcoded zero one and two on purpose so you understand the syntax but it is possible to have you know one process plugin that will read an array of values and populate the fields properly but in this case uh it is machine name of the field delta or the element in in the multivalue field and then the subfield so this is a link text a link field and links have a your uri for the url itself and a title so again like i make my change i roll back i clear my cache i import again by doing so if i go to the website i can see that i have you know my my different links for the different person let's have a look at one of them in this case i am using the as i said before a web server that comes with drupal core so sometimes it it freezes so what i did is just like restarted the server this is like very easy setup you don't need to have many things installed for this to work properly but this is what we saw before we saw a multi-value field where i am populating uh you know with three values the url is the uri subfield and the text is the you know the corresponding subfield and these are the constant that i were using before so that's uh basically what i am doing here one thing uh that i want to show is that you know i can continue the process and you have the examples that you can you know execute everything at once but one thing that i want to show is that so far i i have been executing migrations defined as code because for one they are in my migrations folder uh and two i need to clear the cache for changes to take effect but as i said before you can also do it as configuration entities and execute the same thing from the user interface so in order to do that you put them in config install you define your migrations as you can see it's like almost the same and we're going to see like what are the difference in a moment but once you have those migrations and you enable your module you can go to the website to a structure and then you will have a migrations directory a link you this is a group a migration group that i define and i can list the migrations that are part of that group in this case i there are three of them one for a paragraph one for images and one for nodes so i'm going to execute the node migration and i can say rollback import star preset in this case i want to import and one thing that is going to happen is that i am migrating nodes but those nodes have associated images and paragraphs so the migrate api is able to detect that i have dependencies from my from one migration to another and i am going to get you know all of them executed at once so if i go back to the site i i can see that you know this is what i was doing before like manually one by one and this is what i did from the user interface so i have images this is a paragraph these are dates address fields taxonomy terms and so on so you know you can do it from the user interface if you want as well something that i want to show is the difference uh how different these things are so before i was running from a csv file my migrations and this one are from json files so if i were to compare the two this is what is different i am on my left i see the json migration on my right i see the csv migration so the id is different because as i said before every migration in your project needs to have a different value so that is different the label is a little bit different just to identify that one is coming from json files and another one is coming from csb5 there is some migration group that is something that is provided by the migrate plus module that allows you to share configuration for example if i am going to be migrating from the same json file i can put that configuration to to fetch data from file from that file in a migration group that we can show you in a moment but basically it's just like to get some information but what i want to highlight is actually that the similarities are more common than the differences my source uh constants are the same basically except for json and from csb what is expected to be different is like you know in this case i am running from a csv file and in this case i am reading from a json file so that needs to be different my source definitely has to be different but after that um what is uh in the process section the title is about the same these are some things that i was manually changing for the example but the title is basically the same the mapping of the rich text field is the same of the urls is the same for dates for addresses basically everything in the same like all my process operation is basically the same except for like the machine names of the migrations that i am using this is for power of migrations and then my destination is exactly the same my plugin and my machine name of that content type and the migration dependencies this is what allows you to say okay this node expects images and expects paragraph so you also need to import those before or in advance and because in this case i am using json files in this case i am using images and csb files that's a little bit different but for the most part is the same so this is another benefit of using the micro api you can literally swap either the destination or the source or the or the process and it will be mostly working the same so basically in this case the same content exactly the same content uh it's migrated off from a csv file or from a json file and what i showed before that the migration group this is how you define it you have an id a label a description a source but this is the important part you can have shared configuration so anything that you define here is going to be reused for any migration that is configured to be part of this group so if i go here i can see that i have a node migration an image migration a power of migration and all of them are going to share the same you know configuration to be able to read from my json file so we have about 10 minutes left so i want to make sure i have i leave some time for four questions at the end what i'm going to do is just like for to show like the final version i'm going to remove my migration that i was doing temporarily and i'm going to hold um rollback mr it stands for rollback is an alias i'm going to reveal that the caches for any changes to take effect i'm going to import everything again and after doing that if i go to the website i can see uh again the name is different because this is part of the example that i was showing but everything else is the same and basically i am saying magnet the same information from a csv file or from a json file and with that i just want to switch back a little bit here so this is a summary of the syntax you have your process then you have the machine name of the field the delta if it is a multi-value field then the subfield that you want to populate and then any logins that you want if you want if you have a process flowing change you can have you know many plugins doing transformation for the same piece of data the one important thing to remember when you have a chain like this is that only the source it should be only the first uh plugin that change you need to explain specify a source for anything else in the change what you what uh it is assumed as your source is the value that was returned by the previous process plugins in this case like whatever this produces is going to be the input of this one whatever this produce is going to be the input of this one and because there is nothing else in the change whatever this produces is going to be assigned to my destination field in this case these are some of the modules that were used in the example migrate plus migrate tools my resource csb to be able to read from csv file and in that url there are like 50 plus modules that provide functionality for drupal migrations i would like to say thanks to a lot of the migrant maintainers who have helped me over the years to understand how this works i reminded about the trainings that are happening next week online and the website understand drupal where you can find the 31 days of migration series and the course and we have you know about 10 minutes for questions so i'm going to be reading here but this is uh the url to the slide deck and the url to the feedback form so you know the drupal association would love to hear feedback for all the sessions so i and i personally would like to you know know what you think about this presentation as well so i'm going to leave that on the screen while i uh check for questions in the chat and as i said before all the examples that i show are already available including documentation on how to set up the virtual machine well not the virtual machine like how to set up that quick start command to be able to execute the migrations so let's see if we have some questions here um so there is a question saying how would you recommend exporting from d7 so if you want to go from d7 to either drupal 8 or drupal 9 actually the mitre api offers a complete uh suit of modules specifically for that and there is one called migrate upgrade that can help you automatically generate those jumble definition files that i showed before and with that module you can generate the files if you have any structural changes for example going from a content type to a user or from a content type to a paragraph or if you want to combine two fields or if you want to split a content type so you can use the migrate upgrade module to generate those files make the commodifications execute them and then you have you know your drupal 7 site or drupal 6 site upgraded to drupal 8 or 9. it is important to know that there is no need to to go by triple 8 you can jump directly from 6 to 9 or from 7 to 9. that's that's something that you can do and uh i would highly recommend checking out you know the articles the trainings but in general the migrate upgrade module is a complete suite of modules uh and plugins and everything that you need to go from drupal 6 or 7 to drupal 8 or drupal 9. as of today drupal 8 and 0.9 in terms of migrations are basically exactly the same the only reason why you might want to go with eight and not with nine is if you have a module that is available for drupal 8 but it does it doesn't have a stable release for drupal 9 so you would have to wait for that or find a solution and an alternative for that module in 2.9 but other than that there is really no reason to like go to eight and after go to nine if you have to do that you can do it of course and you know from going from eight to nine when all your modules are ready are available it's going to be you know a lot easier than going from seven to eight or six to eight another question is migrating from one side to another or migrating from d7 to d8 i'm not very clear about the questions but just to to make it clear in the examples that i showed today i was migrating from csv and from jsons but it is also possible to migrate from sql databases and that is what the migrate upgrade does and even in drupal core there is modules that allow you to connect directly to your drupal 6 or drupal 7 database and use that as your source and read directly from the database so you can you can have that as an option like connect directly to the database and upgrade from there um let's have a notice question i use vs code are there plugins to help with spacing i don't think that i know that there is a plugin called prettier that you can configure to automatically format your source code in different languages and different extensions so i will check for that if anything but in general you know just try to be careful as you saw when i was copying and pasting some of the formatting was wrong so you need to be very mindful about that there are um let's see yeah there's people sharing that there are some plugins that might help you with with that let's see if there are any other questions um [Music] so there is a question when changing the text area for matter will it strip anything out of the content that does not need the format basic html for example yes so the micro api is going to do whatever you instruct it to do in my example i said i want to use the basic html text format so you can go modify what html tags what html attributes are allowed in this particular base basic html text format and drupal is going to strip anything that is not allowed if you were to use for example plain html plain text everything is going to be escape for example if you're going to use full html that is probably a bad idea because you might be allowing some insecure html in your in your website coming from your migration but basically whatever you define as your input format drupal is going to respect how you have that configured in your site see if it is basic html restricted html full html plain text you know whatever you define drupal is going to take care of cleaning up and the way that it is defined on the website another question will migrate have better performance over fits module if you need to import thousands of nodes you you can move the thread uh and it might be slow so if you have big so for one uh as far as i understand fitz is not like fully ported out for drupal 8.9 i might be wrong uh but in terms of performance of the migrate api one thing that you need to be uh aware of is that the final api is going to be working with other parts of drupal and in particular if you are creating nodes that is you're going to be executing node shape operations like at the entity level it is possible for example to turn on validation if you want but in general there are you know many things that might happen my recommendation when you have to migrate a lot of content is that you can do it in chunks uh for example migrate a thousand at a time and one way to do that is uh the drug command that i was i was executing you can pass an extra flag called limit to say how many elements you want to import at the same time there are other more advanced concepts one is called high watermark that allows you to say i'm going to be migrating a thousand elements at a time and the next time that you execute you know pick up from when you last left uh in the migration and in that regard you know you can avoid timeout operations from uh just executing a very long-running migration but normally that is what you do if you have you know big data sets you break them down into smaller chunks like uh a thousand for example at a time and using something like the high water mark you can schedule those migrations to happen let's say every five minutes and they will be picking up where the last migration left left let's see if there are any other questions migrating paragraph so if you need to migrate paragraph uh what you need to do it's actually one of the easiest migration to start but things start come to get complicated depending on how your paragraph structure is so this is a part of migration that you can run the example yourself but basically my source is the same uh from before i have a csv file i simply change the the path to the csv file and specify that corresponding id my destination i used entity reference revision column paragraph that's how you do it and then the different bundle is the machine name of the paragraph that you want to migrate and then this is the field mapping so in my paragraph i have a title field and an author field so as you can see it is not that different from a node migration um this is how you can do a paragraph migration when things start to get a little bit complicated is how to connect once you imported the paragraph to the node so the way to do that uh is using a plugin called migration lookup so what you what these plugins allows you to do is say i know that i already migrated something in a different migration and i want to connect that to my to my node migration in this case i am using many things at once i am using the migration lookup plugin to connect to the layer of migration i am using something that is called a pseudo field and then this is my paragraph field so paragraph fields are different to other reference fields in that you need to specify two things one is called the target id and the other one is called target revision id so that's something that you need to keep in mind any other reference field is only the target id so it's easier to manage and this migration lookup is going to return an array of two values and i am basically basically using you know the extract plugin from my pseudo field when you have a zero field and you want to use it as your source you put an adsine at the beginning and single quotes around it and then you know this array i want to extract the first element and i want to extract the second element and this is one way to populate paragraphs but there are many other ways things start to get complicated when you have nested paragraphs when you have paragraphs that might have revisions when you have paragraphs that might have translations when you have multi-value paragraph that's when you need to come up with these like necessary structures sometimes you might have to uh do like recursive uh migrations if you or process plugins that do a recursive operation in your data but like migrating paragraph itself is simple in the sense that um this is how you do it uh the destination you specify it like this your source as you would do otherwise and this is your field mapping once your paragraphs are into the system you connect them to your node in this case uh like this for example and things start to get complicated when you have multi-value you know nested paragraph and so on let's see if there are any other questions uh yeah you can there is a comment saying that you can set dependencies between entity reference and yes you can do that uh when you usually when you have an entity reference you have a separate migration only for that like in this case only for my images and only for my paragraph and this is one way that you can explicitly at the migrate api level say that you want to execute these migrations in advance because your node migration depends on these other two and i think that's it so thank you very much remember that the slide deck the example everything is already available so feel free to try it out yourself and you have my email if you have questions you can reach out later and that is the link also to the feedback form for the drupal association to let them know how i did so i would appreciate your feedback and thank you very much have a good rest of your day or night bye you

Channel: Drupal Association

Views: 163

Rating: 5 out of 5

Keywords: DrupalCon Global 2020

Id: NBhkixY-PuA

Length: 52min 30sec (3150 seconds)

Published: Fri Aug 28 2020