Altering CMDB classes and attributes (aka CDM metadata)


The following information is relevant to all CMDB issues where a data structure component of the CDM is missing, corrupted or generates an ERROR when modifying.
First lets address the composition of the CDM
Common Data Model (CDM) of CMDB Metadata is (in general) made of the following:

1. CMDB Metadata
2. AR Metadata
3. Tables and Columns
4. Indexes
5. Class Hierarchy
6. Conclusion

1. ---------------  CMDB METADATA ----------------

CMDB Metadata - is made out of OBJSTR:Class & Attribute Definitions (these are forms)
There is also a queue that tracks the changes and that form is named OBJSTR:Pending. Records in this form reflect the status of the changes as they are happening.
In some cases the Pending change (Change Pending) can remain in pending status and need to be cleared. However, allowing that change to complete is important.
Class and Attribute changes or additions can take longer than expected because the change has to propagate from the parent class to all its children.
This can take up to 30 minutes in the worst case and few minutes when everything is fine-tuned for performance.

When working through issues you should likely start with a check list and see if this CMDB component is still here. The class manager console shows the state of the CDM as it is "now".
Before making changes you should first check if the attribute already exists. Please don't add unnecessary attributes to BaseElement class. Instead start at the class that actually matches the object you want to have defined for management.

The following image shows the sequence example how one would view the CMDB metadata for field (Attribute Name) "Frequency".



This means that "Frequency" is definitely found in Attribute Definitions form.



Attribute Definitions and OBJSTR:Class together make up the CMDB metadata. Here is an example of the BMC_ComputerSystem class looks like.
Here you can see that the Class Id matches the ForeignKeyID (Class GUID):



Making changes to this form is not recommended for any reason. Only an advanced CMDB Administrator should make changes here although it would be best if that did not happen.
Changes to the CDM should be always done through the Class Manager or CMDBDriver.

The CMDB to AR Metadata Synch request will be tracked in form "OBJSTR:Pending" (after being dispatched through Application Pending) where you can see the general details of the status:



Destination of the request is done based on AR Server Group Ranking. All such requests will be routed to an AR System where CMDB Admin is set as Primary.



2. ------------------------------ AR METADATA --------------------------

- ARSCHEMA & FIELD (these are table names). This is next item on your check list. Is something still referencing the field here?
Use SCHEMA ID to find associated fields in FIELD. It could be missing here, but you have to look.
Best way to look for it is to use SQL query:

SQL: select * from FIELD where SCHEMAID IN ( select SCHEMAID from ARSCHEMA where NAME = 'BMC.CORE:BMC_ComputerSystem')



This report shows useful information about these fields. You can see who created it (owner), who modified it (lastChanged), and the FieldID.
Fields created during the AtriumCore installer with usually show "Action Request Installer Account" or what we know is "RIK" (Remedy Install Kit).
ForeignKeyId (or the Class ID GUI) does not show here because that's part of the CMDB Metadata.

In this case the schemaid is pulled from the associated ForeignKeyId value associated with Frequency and all other CMDB Attributes that are associated with that class.

You will need to run a report from "Attribute Definitions" to find the ForeignKeyId value for these two fields.
Save the SchemaID for next step. It gets even more interesting. :)


3. ------------------------ DB Tables and columns ----------------------

The third and final make up of the CMDB data structures are the tables and columns where the data is stored.
These are the actual tables and structures that hold the CMDB data. Add this to your check list. Usually this is the place where things get "lost".

You can detect problems in this tier of data structures when the SQL errors specifically point to a table or a column.
For example error like "Field Not found T572 C200000035" tells us that the CMDB Attribute "Frequency" metadata reference has a problem with the data structure that was created at some point in time in the past and is now missing.

You can verify this directly in the ARSYSTEM* database.

Using MSSQL SQL client run a query that will show the table design:

SQL query: "exec sp_columns T572"

It will return the table definition for that table. It should include the column id of these fields. 
Below are two  queries shown as Query 1 and Query 2.

SQL Query 1 : "select schemaid,fieldid,fieldname from FIELD where fieldname = 'Frequency' and schemaId = '572'  " (Any database)

SQL Query 2 : "exec sp_columns T572" (MSSQL) or "describe T572" (Oracle)

Here is what it looks like when you run this in a MSSQL client:


If the COLUMN_NAME C200000035 wasn't listed here then that would lead to some SQL error with a message about missing data definition like a column id or table id.

In such case something or someone has already removed it. Trying to figure out what actually caused it or who did it would be very difficult because the record is simply gone and there is no auditing this deep in the database. It's always a good idea to have a vanilla back for reference. If you have no such backup then you can look in the following location:

ATRIUMCORE_HOME/cmdb/en/workflow/Attribute.arx
These are the Out of the box (OOB) attributes AtriumCore installs. Additional attributes are added by Asset Management and the rest are extensions or custom attributes.

If you find data structure (tables and columns) missing at this level then the AR and CMDB METADATA might still refer to this data structure, but the structure itself and the data it contained is gone. To fix this you maybe able to remove the metadata references by removing them from AttributeDefinitions directly or from AR Metadata. However this depends on the situation and should be only attempted by a qualified CMDB Administrator that also has AR Server Administration skill set. BMC Support does have such staff and we recommend that you reach out to us. We will be more than happy to talk about the next step even if you are comfortable with your next step.

4. -------------------- INDEX ------------------

The last component in this make up are the indices. This information that you should when addressing corrupted CDM issues.

An index is composed of a Table ID and a set Column ID's. It helps the database find data quickly by having the table id and the data in the indexed columns listed in a hash table that can dramatically improve job performances. For example in this case if I wanted to create a data transformation job that uses combination of TokenId and Frequency I would create an index that would have a name: I572_530010100_200000035. 

It is important to know that when you need to remove old attributes the CMDB will not delete the attribute if it has been indexed. The Index needs to be removed first before you can remove the attribute!

5. ---------------------- Class Hierarchy ---------------------------

This section may not be necessary for you to fix issues with corrupted CMDB CDM, but I'll add it for complete disclosure.
CMDB CDM is an inheritance model where the top level "base" data structure has the most common attributes.
These common attributes are inherited by all its children classes. Child classes have more specific attributes that fit the purpose of the defined object.

For example BMC.CORE:BMC_Person would have an "Email Address" field and BMC.CORE:BMC_IPEndPoint has an "IP Address".
Each attribute is mutually exclusive for each object. CMDB Class hierarchy is then joined into a view. This view joins the base class and any other class it is defining.
Typically it's just two classes. A join would be like BMC.CORE:BMC_ComputerSystem that has data references in BMC.CORE:BMC_BaseElement and BMC.CORE:BMC_ComputerSystem_
One day you may have a discussion at you water cooler where someone argues that "The BMC.CORE:BMC_ComputerSystem_" is the actual form that "actually" holds the data, but after reading this article you should be able to demonstrate that neither is true. The finalized formatting you see when looking at CMDB forms is just the visualization aspect that lets the data  surface for the end users to interact with it. Destroying the BMC.CORE:BMC_ComputerForm by any other means than through the CMDB Class Manager leads to no data destruction until we actually get into the underlying tables. However that does not mean that fixing such destruction is going to be trivial to restore. We've published another KA that shows how to use the CMDB driver to restore some of the data structures from a backed up CDM.xml. Please see KA ID: "000095846"

Everything you saw described in points 1 through 4 is true even for the BMC.CORE:BMC_ComputerSystem_ form, but if you feel that you need to fix the data structure for yourself then please stop and leave the construction mechanism to CMDBDriver (discussed in Conclusion) rather than attempting to restructure all ~192 CMDB classes on your own.



6. ------------ Conclusion ---------------

The following statements define the course of action that can be taken:

- When all components of the CMDB data structure are present as described above then the CMDBDRIVER should be able to complete the deletion of fields you need gone.

- If the Attribute (field) refuses to be deleted then check for indexes.

- If you find any data structure components missing then CMDBDRIVER can not delete these fields. You can try using a delete CI (di) command with the CMDBDriver first to clear any metadata in-between. This should at least clear the fields from the higher levels of the class hierarchy. It will also truncate the data at the table level! Please see KA ID: "000095846"

CDMB metadata is the interface to data and it is designed to find it by having AR Metadata reference from the matching DB Table by using SchemaID. These table (schema) Id's are not static because they're created during an install on AR System that may have different range of Id's available for table ID assignment. Don't expect all CMDB installs to have the same T tables basically. Conversely all column Ids (CMDB Attributes) will be same Id's from AR system to another. Hence the "common" word in CDM.


One last thing!
Asset Management and other applications also have an interest in the CMDB data and hence there is one more deeper level of views. These are joins of a join of the CMDB Metadata reference from AR Metadata finally getting data from Tables where the table is actually still "maybe" the real table. Many Remedy Administrators that have attempted to overlay AST views have found themselves in a tough spot during upgrades because of this complexity. BMC Software does not allow overlays for BMC.CORE forms up until version 9.1 due to this complexity but that may change at some point in the future. Perhaps by the time you're reading this article that will have changed, but until then please stay away from overlay BMC.CORE forms. It can be done, but you will need to master the above concept and be willing to work with BMC Support or Professional Services to undo data structure corruption that resulted from overlaying it.