SAP Data Intelligence Metadata Explorer Walkthrough

the metadata management delivered by metadata explorer targets three main use cases the first use case is data discovery and governance you can rapidly leverage any connected system and get a complete view of all your data assets eliminating data silos it allows you to leverage the business glossary across all metadata as well as using user generated data tags to efficiently find the necessary data assets the second use case is data quality monitoring where a data steward can define business rules and apply them on multiple sources to monitor the results across the data landscape and the third use case enables a business user to leverage the assets from the connected systems once connected the solution allows for easy search functionality and powerful refining of selected data through in-flight transformations of data quality and enrichment so that you can go from raw data to usable data as quickly as possible prior to manipulating the data governance of the process needs to be put in place so that everybody's terminology remains consistent across lines of businesses and across different personas for example though users may run the same business and the fabric consumers may be leveraging the same underlying ERP every line of business within a company has different local or legal requirements ASAP data intelligence deliver standard templates that can be derived to enhance the glossary by adding custom attributes or new categories on the left hand side we can see what was created in the system using the predefined templates these templates can be enriched to surface not only s ap specific data repositories but really any needs the templates can include 20 custom attribute groups containing up to 25 per group to meet requirements to define a term for any line of business you can also easily create a new category based on your interests such as supply chain inventory warehouse or a subcategory interacting with the business glossary allows you to filter by alphabet to select terms associated with one or more categories or to see only the terms you created as a user all these capabilities can be combined to retrieve the term you are looking for for example let's show only the term we created then let's select F we can see the term first name let's take a look when reviewing editing or creating a term the term needs a name it can have a list of keywords that are associated with it and can also list synonyms that the term could correspond to the definition of the term which is in a text format can also include images tables and hyperlinks if it requires a diagram or a blueprint this is fully customizable to allow you to define and enrich the terms as needed now we can view this terms relationships relationships amongst terms show how a term relates to some of the datasets or to some fields that are in the data catalog the artifacts listed here give you an entry point to perform other capabilities such as profiling a dataset accessing the profiling information or preparing the data one simple and intuitive way to explore your connected systems is to browse the connections which is especially helpful when you are trying to enrich your data catalog this view allows you to freely explore the data assets while providing additional information when at the root of all of the connected systems you can check which capabilities are available for any of your systems for example you can check if the connected system supports profiling publishing lineage extraction self-service data preparation or content addition additionally you can discover the data and then proceed with the supported capabilities when browsing your connected systems you can decide to publish datasets or entire data repositories in the metadata catalog now let's have a look at our metadata catalog the metadata catalog automatically identifies the content such as the data types or personal information based on the extra did metadata and the profiling information collected from the dataset the solution automatically tags the information for the content types when the metadata is extracted here we can see all our extracted metadata we can use the hierarchy of tags to filter the catalog for example on personal information it shows the tables or flat files that have that information you can create your own tags in the hierarchy and associate them to the data set or columns while looking at profiling information you can also define filters that can be simple or complex depending on your needs the retrieved data assets show their type but you can also directly access and interact with the retrieved assets now let's see how both the business glossary and the data catalog are combined we can search a data set using the glossary terms that were already defined so for this example we will search for given name and now we can see the results using our data catalog that is linked to our business term this means you can discover the data by browsing searching by terms keys or synonyms by directly using the glossary or by using natural language to retrieve the correct data once we think we have retrieved the correct data we can access the associated fact sheet to view more information about the data set the fact sheet provides a quick access point to view all useful information about our data as we can see this data set was published in the catalog but it was also profiled if this was not the case we could choose to profile the data set from here users can also profile the data multiple times at any time this is especially useful to track the evolution and the quality of the data based on the profiling information that is retrieved and surfaced in the fact sheet once computed the profiling information for the data set contains information such as data types unique key identifiers minimum maximum and average length the percentage of blank zero null values the number of distinct values or whether the column consists of only unique values you can interact with the fact sheet by using the graph or defining filters while analyzing the results you can preview the data you can also check if there are any ratings and comments associated with the dataset you the fact she also allows us to view tags from the metadata catalog that are associated with the dataset and to interact with them this is also where lineage would be surfaced if available for a data asset let's find a dataset which contains some lineage information to find this data set we will use the discovery dashboard this is a central location that shows all the activity that is happening in the metadata Explorer including a comprehensive overview of the platform usage the number of data sets and how they are used the number of tasks that were executed the profiling metrics and the latest published data sets in the catalog here is the data set we are looking for let's select one table to view its fact sheet and lineage information as we can see the lineage information is available so we can track that this object has a view restricted to only date datatype columns based on a table view from an original table stored in sa p Hana while reviewing and interacting with the lineage graph we can access all the usual capabilities such as viewing the fact sheet reviewing the asset in the data catalog or preparing the data for other usage data intelligence comes with additional capabilities such as rules rule books and data quality dashboards that can help to ensure that the data can be trusted to drive successful decisions data stewards can create rules regardless of the data set that they are being evaluated on the solution comes with predefined rule categories and allows users to create additional categories to group rules by types or evaluation dimension criteria you can create as many rules as needed per category let's take a look at one pre-existing rule as you can see we know if a rule is already used in a rule book we'll delve into rule books shortly a rule contains parameters and conditions the parameters are the list of attributes for example in a dataset which we want to enforce a condition on these parameters are used in the conditions definition to identify what makes a rule successful the list of conditions operands is dependent on the parameter data types now coming back to rule books rule books enable you to import and gather rules and link them to a data set for evaluation let's view this rule book when creating a rule book and importing the rules you must associate every parameter coming from the rules to the fields available in the data set you wish to evaluate you can associate one rule to multiple data sets simultaneously within a single rule book the rule book allows you to execute the rules and view their associated evaluation results the execution of the rule book can be performed multiple times to track the evolution of the rules evaluation results through time once executed you can see the evaluation result details and isolate failing records to take further action you can also create rule dashboards to create an executive view of the data set quality across your data landscape here in this dashboard we are tracking the quality of different sets of data that are all used together in one analysis to track device usage per customer you