Data Management - Metadata Management

[Music] welcome to introduction to data management course lesson 1 this lesson covers metadata management capability after this lesson you will be able to understand fundamental concepts people process and Technology aspects of metadata management understanding of basic concepts is a very first step in every learning process so let's start with data element as the basic concept in metadata management a data element refers to a unit of data for which the definition identification representation and permissible values are specified by means of a set of attributes even in a small organisations there is a huge number of data elements and not all of them are equally important those which are of special interest for the organization are called critical data elements critical data element is the data element that is critical to success in a specific business area or business process it is very important to tag critical data elements because it is not effective now they're possible to manage all data elements on the same way due to resource limitations what is the criteria for one data element to become critical one data element can be considered as critical it refers to one business facts that are deemed critical to the organization to support critical business processes across an organization and its components three data used to derive values that appear in key reports for unique identifiers of things important to the business for instance customer ID this criteria listed is not all-inclusive as there are many other examples any data element that is required for the execution of a key business process can be considered as critical data element data elements and critical data element are fundamental concepts in data management [Music] now knowing what the data element is let's focus on the big picture of the metadata management capability by definition metadata management involves managing data about other data whereby this other data is generally referred to data models and structures not the content it includes managing information about data structures from different models and their mutual associations for instance metadata management includes information about business terms and glossary attributes and logical data model or tables and columns in the database as well as their associations so if we put critical data element as a center of metadata universe let's see what kind of data we can collect about it we will start with business glossary which contains business term which represents critical data element then its definition and synonyms and acronyms and use in addition we collect information about taxonomy x' also called classifications which actually determine how elements should be classified according to specific criteria for instance if the criteria is gender person can be classified as male or female business rules refers to set of rules applicable to data element [Music] you ownership refers to specific people or roles that are accountable or responsible for the data element [Music] [Music] logical data model refers to entities and attributes that are mapped to data element physical data refers to database schemas tables and fields physical data are spread across several layers like operational sources data warehouse analytical data marts and so on in addition there is an integration process between those layers which refers to data movement from source to target data [Music] finally there are various applications for reporting and analytics so to summarize with metadata management we are collecting information about critical data element in order to understand how the data element is used across the organization once again metadata management it is not about the actual data but about the data models and structures as an analogy if we take an example of water supply metadata is not about the water in our case data but about the infrastructure and pipes in our case the data models and structures [Music] the metadata information about glossary taxonomy business rules and ownership are also called business metadata on the other hand the metadata information about logical data model physical data and data integration are also called technical metadata [Music] business metadata describes data element from the business uses perspective and includes information like business glossary with terms and definitions synonyms acronyms taxonomy business rules and ownership let's move forward with an example that covers business metadata let's say we have data element named customer name and the element is tagged as critical please note that going forward we would use the acronym CDE for critical data element so let's collect business metadata for customer name CDE starting with the glossary part business term the business term used to represent that data element is customer name definition customer name represents a word or a combination of words by which a customer is called or known customer refers to person or entity synonyms customer is often called as client so the client name could be the synonym for customer name taxonomy z' customer name can be referred into content as a legal name or tray name the legal name of a business is the name of the person or entity that owns a business legal name is the name that identifies a customer for legal administrative and other official purposes a trade name is generally considered as the name of business uses for advertising and sales purposes that is different from the legal name in its Articles of Incorporation or other organizing documents a trade name can also be referred to as a fictitious name or a doing business as examples of trade name is the use of the name McDonald's by the company whose legal name is McDonald's Corporation [Music] with business term definition synonyms acronyms and taxonomy we have covered the glossary part of the business metadata [Music] the next piece of business metadata is about the business rules on the screen you can see a set of business rules associated with our CDE name customer name for example we have two rules that provide conformity of the legal customer name the rules are defined as one for limited liability companies LLC's and corporations the business legal name is the one that was registered with the state government these names will often have a legal ending such as LLC in Gore LLP and two if the business is a partnership the legal name is the name given in the partnership agreement or the last names of the partners also we have one rule for trade name conformity defined as a trade name may not include Inc LLC or or similar legal endings although a trade name may sometimes also be a trademark a trade name is not in itself a form of intellectual property at this point please remember that business rules are instrumental for data quality assessment which is a part of data quality management capability data quality capability is covered in one of the lessons that follows [Music] before we move forward with our example of collecting metadata information for customer named CDE let's discuss about roles and responsibilities related to metadata management capability in order to manage critical data elements on an effective way we need to define and assign certain roles and also responsibilities for managing the CDE metadata so here is the description of the roles and responsibilities business owner is ultimately accountable with regard to the definition data quality and value of data in a given subject area the business owner is responsible for confirming that data is used in a fashion consistent with the overall data strategy business owner is also responsible for driving data management processes and activities data Stewart is responsible for operational oversight of assigned data and interactions with subject matter experts also called Smith a Crossing organization as well as identifying the approach to standardize measure and monitor data quality charged with confirming that data standards are defined processes are employed and practices are implemented technical owner is technology specialist who is ultimately accountable that data from particular data system are managed and used according to defined data standards that includes enforcement of business rules data policies and procedures data custodian is technology specialist who is responsible for the secure storage and management of the data for the particular data system charged with confirming that the defined data standards are enforced [Music] you [Music] please note that business owner and data steward are business roles and need to be assigned among business professionals from the organization on the other hand technical owner and data custodian are technical roles and need to be assigned among technical professionals [Music] now let's get back to our customer name CDE example on the screen is shown customer name as critical data element we have assigned metadata role to particular people from the organization along with their names there are information about their position as well as contact information contact information is important so that anybody can reach out to them in case of need for specific information or to submit requests related to customer name CDE [Music] other than role assignment please notice that some roles are exclusive such as for example business owner that ensures that certain aspects of critical data element such as name or definition have to be managed by one person in order to avoid ambiguity on the other side some roles like data custodian can be assigned to multiple people by definition data custodian is responsible to enforce standards for critical data element in this case customer name CDE for a particular system accordingly if customer name has multiple representations across several data systems the data custodian role will be assigned for each system before we move forward please note that metadata roles and responsibilities cover people aspects of metadata management capability as we defined in course objectives [Music] so far we have covered business metadata like glossary business rules and our ownership now it's time to look at technical metadata technical metadata describes data elements from the technical user perspective and includes information like logical data models source and target systems table and field structures as well as cross model dependencies [Music] [Music] besides business and technical metadata there is another category named operational metadata operational metadata includes information about application runs their frequency record accounts and other statistics for auditing purposes operational metadata is important for auditing purposes and technical operations now let's get to the first example of technical metadata one shown on the screen is specifically related to logical data model logical data model or ldm metadata refers to the information about entities attributes and their mutual relationships after collecting metadata about logical data model it is important to establish association between the CDE and all representations of CDE and logical data model on that way you will be able to trace CDE Lima gem business to technology let's look at an example with our customer named CDE let's say we have model with two entities customer and order customer entity contains attributes such as customer ID customer name customer address and so on therefore we can make association between CDE customer name which belongs to business metadata and attribute customer name from the entity customer which belongs to technical metadata here is another example of technical metadata this time with physical data model and data liners physical model metadata includes information about data systems database schemas tables columns constraints transformations and so on also it is important to establish association between CDE and all representations of CDE and physical data model as well as to collect information about data lineage data line which refers to traceable pass for CDE starting from end-user report upstream to the ultimate source that pass includes aggregated sources such as data warehouse and data Mart's operational data stores staging areas and transactional systems let's look at the example let's say we have orders transactional system with two tables CST representing the customer data and Ord representing the orders data in addition we have finance data Mart which contains aggregated information from order system aggregated information refers to the revenue field amount per customer field CST ID CST nm and per period of time field PR DTP then we perform data integration process which aggregates data promoters transactional system into finance data Mart from the metadata perspective we collect information about both systems transactional and data Mart including tables and columns as a last step we make association between customer name CDE and its representations in both systems for orders transactional system that is columns EST NM in table CST for finance data Mart that is columns est nm in table fin please note that with these associations you are able to trace CDE data flow starting from business reports trough data Mart's transactional systems upstream to the ultimate source that is what data lineage is about now we are getting to the point by collecting CDE business metadata such as definition taxonomy rules then determining ownership and finally collect technical metadata such as data systems tables columns and data limit you are getting complete view of CDE and the enterprise also called CDE data standard [Music] [Music] business and technical metadata together represent CDE data standard which means that critical data element is completely described from business and technical perspective CDE data standard is also called 360 view of CDE [Music] by achieving 360 view of your CTE you provide conditions for managed data environment which is ultimate goal of data management once you achieve that you will be able to answer questions such as what is the CDE definition who is business owner of the CDE where CDE data are stored what reports uses CDE what system is ultimate source of the CDE and so on so far we discussed what kind of metadata we should collect about the CDE now let's formalize the process how to perform metadata collection and to enforce of CDE data standards across the organization the metadata management consists of five key activities one identify critical data elements refers to activities performed to analyze business requirements conduct interviews with business stakeholders and identify in skilled CDEs this activity is performed by business owner and data steward to collect CDE business metadata for in scope CDs define business term synonyms acronyms definition and taxonomy z' also define business rules and determine ownership this activity is also performed by business owner and data steward 3 collect CDE technical metadata for in scope CDs identify CDE representations and data systems determine authoritative data source and data lineage also determine associations with logical data models if exist this activity is also performed by technical owner and data custodian for create CDE data standard also called 360 view create associations between CDE business and technical metadata validate CDE data standard this activity is performed by all metadata roles business and technical five enforce CDE data standard enforce CDE data standard to ensure that every CDE is managed and used according to define standard this activity is colored differently because data standard enforcement spreads across metadata management and data governance capabilities this activity is also performed by all metadata roles business and technical last but not least brief reminder about our course objective to cover each capability from people process and technology perspective this slide refers to the process part [Music] there is one more important aspect related to metadata management process system development lifecycle or F DLC F DLC or system development lifecycle refers to the process of planning creating testing and deploying of an information system metadata management process should be a part of SDLC in other words during application design and development you should ensure enforcement of data standards defined as a part of metadata capability in all application segments similar like previous slide this slide refers to the process part of metadata management capability [Music] so far we have covered metadata management fundamental concepts as well as people and process aspects now let's focus on the technology required to support metadata management process in order to establish metadata management capability you need some tole to support the metadata process from technology perspective metadata management tools should have certain set of features in order to support metadata process on an effective way here is the list of some of the key features one ability to establish and deploy centralized metadata repository to ability to define consistent terms valid definitions valid values data domains through a hierarchy of glossaries three ability to extract metadata from the popular database management systems data modeling tools business intelligence tools and ETL for ability to create name and define associations between metadata repository artifacts and five ability to manage versions of metadata from multiple source tools and technologies before we move forward a brief reminder about the course objective to cover each capability from people process and technology perspective this slide refers to the technology part hope this journey through metadata management capability is getting you on the board for remaining capabilities so far we covered fundamental concepts as well as people process and Technology aspects of metadata management before we move to the next capability here are the key takeaways critical data elements also called CDs are foundational components of metadata management business metadata described CDEs from business perspective and includes term definition synonyms acronyms business rules taxonomy and ownership technical metadata describes CDEs from technology perspective includes logical data models physical data models data integration and data lineage metadata management process formalizes activities related to metadata management metadata management roles defines roles and responsibilities in metadata management process includes business owner data steward technical owner and data custodian data standards also called 360 view provides comprehensive view of CDs including business and technical metadata and metadata management tools provide technology support to metadata management process in order to ensure that CDEs are properly managed and used that's all for lesson 1 and metadata management capability before we move forward with the next lesson take some fun and complete the metadata management quiz

Channel: Global Data Store LLC

Views: 31,849

Rating: 4.6935482 out of 5

Keywords: Data Management, Metadata Management, Data Quality, Collibra, Free Course

Id: oAbHuJJTI5Y

Length: 30min 35sec (1835 seconds)

Published: Sat Feb 11 2017