NOTE: This article does not replace documentation.
For specifics of features not discussed in this article please consult http://docs.bmc.com

Points covered in this article:

1. CMDB-Inline-Normalization
2. What triggers Normalization?
3. Normalization CMDB data in AR Server group
4. Why does Normalization matter?
5. Normalization and Multitenancy
6. "- Global -" Company - What does it mean?
7. Unknown Versions and Patches
8. Normalization of objects acquired from discovery sources
9. Alias Mapping
10. Market Version default values
11. Normalization Defects
12. Exclude the PatchNumber attribute in a specific Class
----------------------------------------------------------------------------------------------------------------------------------------------------------------------


1. CMDB-Inline-Normalization

Let's begin by explaining the CMDB-Inline-Normalization setting in ar.cfg/ar.conf (ARDBC).



NOTE: The CMDB-Inline-Normalization setting must always be set to True (T). Please don't confuse this setting with "Dataset Level Inline Normalization".

This setting only effects manual changes by an end user. Typically someone like an Asset Operator or perhaps even CMDB Administrator that is testing Normalization changes with Dataset Level Inline Normalization: It is used with the setting of "Preserve Manual Edits" in the Normalization configuration.



Setting the "Dataset Level Inline Normalization" is what causes CI's (Configuration Items) to be normalized during a manual update or anytime a CI is created or modified.



Typically normalization is done via batch job or job running in Continuous mode, but it can be used with Dataset Level Inline Normalization. Running batch or continuous jobs will also include normalization of Relationships. You can see BMC documentation for listing of available features.

Changing this Dataset setting to Continuous and the Normalization engine will only normalize the CIs in a scheduled batch job or continuous job.




2. What actually triggers Normalization?

1. Normalization is triggered by saving any changes or adding values into the fields of BMC.CORE forms. These are defined as Configuration Item (CI) records where the CMDB class ID is included in the "Normalization Configuration > Class Configurations" form.



e.g. BMC_Person is not normalized because it is not listed in that form (The CI NE Status will be as "Not suitable for Normalization").

2. The fields "Model" and "ManufacturerName" are required for normalization to pass.



+ These fields (CMDB data Attributes) are as follows:

- Category (Field Id 200000003. Owned by BMC.CORE:BMC_BaseElement class)
- Type     (Field Id 200000004. Owned by BMC.CORE:BMC_BaseElement class)
- Item     (Field Id 200000005. Owned by BMC.CORE:BMC_BaseElement class)
- Model    (Field Id 240001002. Owned by BMC.CORE:BMC_BaseElement class)
- ManufacturerName (Field Id 240001003. Owned by BMC.CORE:BMC_BaseElement class)
- VersionNumber    (Field Id 240001005. Owned by BMC.CORE:BMC_BaseElement class)
- PatchNumber      (Field Id 260131107. This one has 64 characters and is owned by BMC.CORE:BMC_ApplicationSystem class . Also Field Id 301137300  with 254 characters and owned by BMC.CORE:BMC_Software)
- MarketVersion    (Field Id 530062400. Owned by BMC.CORE:BMC_BaseElement class)
- Company          (Field ID 1000000001. Owned by BMC.CORE:BMC_BaseElement class)
- AccountId        (Field ID 301186800. Owned by BMC.CORE:BMC_BaseElement class)

VERY Important: If Company is specified then AccountId will also be considered.

3. Normalizing CMDB data using NE plugin running in an AR Server group:

Prior to 9.x release the normalization plugin was triggered and could run on any system in the server group without any control. On those versions Normalization could run on the AR Server "closest" to the call that has received a change to an altered CI. By closest we mean that the CI is processed on the server that is currently accessed via Midtier. The actions requested will be performed on the "In use" server.

AR Server Dispatcher Service has no specific instructions to run the Normalization Jobs on a specific server.
Inline settings discussed above will apply in this context. Removing the NORM plugin from configuration can lead to ARERR 8755 "The specified plug-in does not exist" errors.

After the 9.x release: You can view the Normalization Engine ranking from the  AR System Service Failover Ranking form. The rank in the AR System Service Failover Ranking form is populated by setting the rank of "Service Failover" in the AR System Server Group Operation Ranking form. The Normalization Engines are assigned Rank 1, Rank 2, and so on, where Rank 1 is the highest.



and the "AR System Service Failover Whiteboard" form:



Please see documentation on AR Server and how to implement Normalization Service using these forms.
ref: https://docs.bmc.com/docs/ac2002/normalization-failover-for-scheduled-batch-jobs-908213765.html

4. What impact is there if CIs are not normalized? Why does normalization of data matter?  

Normalization of data makes sure that all records are in a format that every business unit can consume in the same way. It validates approved and normalized products.

"What makes Normalization work?" or "How do I get the 'Normalized and Approved' status?"

• CIs get reconciled by the Reconciliation Engine. CMDB data Reconciliation has a toggle that will only process CIs that are Normalized. This setting is listed as "Process Normalized CIs Only".

 

• Another impact is on running reports on data where CI's have not been normalized. A product might be imported as "Latitude" or "Dell Latitude" and running a report that looks for: "Dell Latitude" laptop would exclude all "Latitude" CIs even though they are the same product. Product Catalog is very effective when data is normalized to common and matching values.
• Software licensing will use and associate "MarketVersion" with the cost of a software package. This helps with the cost analysis of the production environment.

First, let's begin with the CMDB Class and what you can do to see what fields need to have a value to be Normalized. Each class you want normalized must be listed in the Class Configuration panel of the Normalization console. Each of the fields that have a check in them must have a value to pass normalization.

This screen capture shows an example using the BMC.CORE:BMC_Product class.

Normalization Configuration Editor:



In the Normalization Configuration Editor screen capture above you can see that PatchNumber is specified for BMC_Product class. Using the correct class when looking for reasons for Failed Normalization is important. Below you can see that the PatchNumber attribute is visible in CI classes that are children of BMC_BaseElement, but not in BaseElement itself.

PatchNumber in Attribute Definitions:




This means that looking at data from the BaseElement container will not give you the information you need. If PatchNumber has a value and caused the Product CI to fail, then this is not the view that will give you that.
Please use the class container (form, schema, view) that is closest to the ClassID. For example, use BMC.CORE:BMC_Product instead of BMC.CORE:BMC_BaseElement.

PatchNumber, Version, and Model can vary in Associated Company. This is often one that is overlooked by data admins. Please make sure you add this to your investigation of CIs that have failed to normalize. Once you specify one PatchNumber for a Model then all Versions without a Patch will need to have Unknown or BMC_UNKNOWN PatchNumber associated with it. Otherwise, they will fail to normalize.

Again, here the software is doing exactly what it is told. This creates lots of extra work for the CMDB Administrators so please make informed decisions on patch tracking via the CMDB.

 

This knowledge article may contain information that does not apply to version 21.05 or later which runs in a container environment. Please refer to Article Number 000385088 for more information about troubleshooting BMC products in containers.

5. Normalization and Multi-tenancy

Multi-tenancy applies when a service provider uses BMC applications to provide services to multiple clients.
Company example like ACME Co. and Calbro Inc. may both be providing services using the same application managed on the same AR Server instance.
Multi-tenancy setting impacts directly into what is put into the Company field of a CI.
If the AR Server hosted Asset Management application is set to "single tenant" then all values in the Company field should be the same. With that setting all company values default to that company.
A product (or a model) typically has a - Global - association and be used as such by all Companies consuming the CMDB data. However, one or more Companies may have an exclusive version of PatchNumber that is not common for other Companies.
These Version(s)/PatchNumber(s) would then be only associated with these Companies. A Model should almost always be associated with - Global -, but the version can change. Most or all of your Models will be associated with - Global - Manufacturer.

In the Product Catalog this translates into "- Global -" Company and Product association.
If the AR Server hosted Asset Management application is set to multi-tenants as seen here:




6. -Global- Company - What does it mean?

The values of Model and Manufacturer must be found in the Product Catalog (PCT) and associated with the "- Global -" Company:



It is not necessary to have the Normalized Product checked to Yes for Model and Company Association.

The same thing applies to Patches. Once a  patch is specified then from that point on that Model will require a Patch number to be set or set as Unknown.

7. Unknown Versions and Patches

If a Version is being used with a product and no Version is specified in the CI's Version field then a new Version must be added with Unknown.




Conversely if any "known" Versions are associated with a Company then each Company must have this version associated with it. Otherwise it is all a - Global - Company.
Consider that a product model is used by everyone and hence this will usually always be '- Global -'.

Products that are exclusively used by one Company are the only products that would not be Global. Such products would not be visible to anyone else.
Product that are common, such as Microsoft Windows operating system, will always be Global and only the Version could be associated with a non-Global Company but only if the other companies don't use it.
Please keep in mind that you will not be able to use the '- Global -' Company once you associate a Version to a unique company. These two relationships can not coexist. A product is either Global or not.


Note that if a field such as PatchNumber contains null (empty) values but also contains actual values, if the UNKNOWN (or BMC_UNKNOWN) is used as a catch-all to normalize CIs where PatchNumber = null, the CIs where PatchNumber = a value will fail to normalize.  These CIs failing normalization will occur even if the version is defined.  Basically, the UNKNOWN fields and actual values in the field cannot be used together.

If the CI PatchNumber contains a value, then any CI with a PatchNumber = null should fail to normalize and those CIs should be corrected on the source side (such as in BMC Discovery), such that the PatchNumber field would contain a value.  If every CI PatchNumber = null and the UNKNOWN value is used for the PatchNumber, then the CIs would normalize.

The same issue with UNKNOWN can occur with the Model/Version field as well.  This  process is by design in Normalization.

8. Impact on data found by overlapping Discovery

This setting is important to understand because you might find that the data discovery sources are discovering CIs with Version values, but not all the time.
Some of the discovered items may not provide Version and since our goal is to have the data normalized without a value then this does not make logical sense. 
The Version is either important to your Organization or it isn't. To say that we're tracking Versions of a Model but then accept a value that indicates UNKNOWN makes no sense to the program.
Once a version is added to Version and Company association then that Model, Manufacturer and Company (even if - Global -) must have a valid value to normalize. We've added the UNKNOWN keyword to our code to solve this.
Using "Unknown" is a workaround for the software code to be able to accept CIs without the version. We had to do this because Version is not a guaranteed discovered item and the data needs to be sorted. A NULL value does not do well while used in data sort situation. It's a feature.

Best practice would be to allow the Normalization Status to fail and then work a case using Change Management to find out why the Version is missing. It's definition and assignment should be documented for reference.

9. Model and Manufacturer Alias and Alias Mapping

This area tends to confuse quite a lot of users because of the two apparent alias mappings by PCT and Norm. 
Product Catalog Mapping and Normalization Alias Mapping will only work in the following sequence.




Only use the Normalization Alias from the Product Catalog Console for Model or Manufacturer mapping:



Through this form you can setup mapping for Manufacturer values to map to actual manufacturer name. Also Model Name to actual model name.
This works rather well and it populates the CTI values as well. However, the sequence is important.

For example here is a screen capture of what a BMC_Product CI looks like before normalization:




And here it is after Normalization. No Product Catalog Alias Mapping is necessary to have this work.
In fact it actually makes it more complicated than it needs to be. Please only use the Normalization Alias to get the data normalized.




10. Market Version and default values

You may have noticed that Market Version is populated with a value that is the same as Version when Normalization is set to Dataset Inline mode (see point 1) or after a Normalization Job was completed.
This is set automatically be the following rule found in Normalization Features known as "Rulename98". This rule is configured to populate the Market Version field when InstanceId and Version is not empty.
If you've configured your Market Version in the Product Catalog setup to be a different value then this rule could override that. You can change this rule to only show the "Major" part of a Version.
For example 10.1.001 is broken down to 10(Major).1(Minor).001(no designation). Update the rule to show only the Major part of the string value.

Note: Since MarketVersion only matters to Licensing where the cost associated with the product release rather than patch, using Major is most common for all products.



Please note that there is a know defect (SW00520230) while searching the Normalization Features console for rules by their name.
The search feature does not work and you'll have to scroll through the list to find the rule. Also scrolling does not work if you've tried to search once and you will need to reopen the interface to find it.


11. Normalization and code defects

Some product defects do exist. Please review the release notes of each product release and its related Service Pack to see if you're encountering defects that do match the features described in this article.
Some known issues that were released with the 8.1 code line have already been fixed with a hotfix which will normalize data as described above (per design specification).
Additional work is being done to address multiple matches in the PCT. These only impact products where Manufacturer is Unknown.

12. Exclude the PatchNumber attribute in a specific Class

To disable the patch normalization, the PatchNumber attribute has to be excluded from the attribute List of a particular class. E.g. SoftwareServer.

From the CMDB Console, the attribute list can’t be edited and have to use the backend form to do so.




For Normalization process, the information about attribute to be normalized is stored in the "NE:AttributesInfo" form using the AttributeId (not by name).

To simplify the steps, We have extracted the AtrtributeIds of PatchNumber attribute from CMDB metadata (using OBJSTR:AttributeDefinition form).

There are two PatchNumber attributes in CMDB each for two class hierarchy in BMC CMDB.

1. InstanceId for PatchNumber attribute belongs to BMC_ApplicationSystem (parent of BMC_SoftwareServer)

            OB000874CD4B4FoD2cQADD29CwFQMA

2. InstanceId for PatchNumber attribute belongs to BMC_Software (Parent of OperatingSystem)

            OB000D56BA2BA54hfvQAioxBBQ4gAA

Steps to follow -

1. Go to NE:AttributesInfo form
2. Search for the class under ClassID field we want to disable patch normalization for. Once the record id fetched, we will get a list of attributes separated by ';' under the AttributeIdList field
3. OB000874CD4B4FoD2cQADD29CwFQMA and OB000D56BA2BA54hfvQAioxBBQ4gAA are the patch number Attribute IDs.
4. Remove any of the above 2 values, whichever is available in the AttributeIDList for particular class and the ';' that goes with it, save the record and it will disable the patch normalization for the selected ClassID

           

        5. To confirm if the change is applied successfully, verify from the CMDB Console

          

See also: BEST FAQ on BMC CMDB Suite - Normalization