Handling PATROL Events of Class 11/41

How to handle PATROL Events of Class 11/41 and more especially how to remove the Events of Class 41 when the thresholds are no longer exceeded.

Related Topics

Description

This article explains how to handle PATROL Events of Class 11/41 and more specifically how to remove the Events of Class 41 from BMC Impact Integration for PATROL (Bii4p) when the thresholds are no longer exceeded.

The Different Types of Events

By default, BMC Impact Integration for PATROL (Bii4p) sends three types of events:

  • Events of Class 11 and 9 that are generated by the PATROL Agent
  • Events of Class 41 that are generated by the KMs developed by Sentry Software.

inline

Events of Class 11

Events of Class 11 are generated by the PATROL Agent when the parameter value has exceeded the alarm range thresholds. This will raise a warning or alarm state for this parameter. These events only contain the bare minimum of information about the fault. Here is an example of the Event Class 11:

Class 11:
PATROL_EV;
                event_handle=2235787;
                mc_ueid='BII4Patrol/fe80::1454:69d3:25a3:58da/3181/1357819736/448436';
                mc_client_address='192.168.7.24';
                adapter_host='sam.internal.sentrysoftware.net';
                mc_location='internal.sentrysoftware.net';
                mc_service='';
                mc_host_class='NT 6.1 Windows Server 2008 R2 Standard Standard Edition AMD64';
                mc_host='morgan.internal.sentrysoftware.net';
                mc_host_address='fe80::1454:69d3:25a3:58da';
                mc_host_id=0;
                mc_account='';
                mc_object_class='MS_HW_FAN';
                mc_object='MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c';
                mc_object_uri='';
                mc_object_owner='';
                mc_tool_class='BII4PATROL 7.3.60';
                mc_tool='morgan.internal.sentrysoftware.net:3181';
                mc_tool_id='';
                mc_tool_rule='';
                mc_tool_key='448436';
                mc_tool_sev='4';
                mc_tool_address='';
                mc_tool_uri='';
                mc_tool_time=0;
                mc_tool_suggestion='';
                mc_origin_class='V3.10';
                mc_origin='morgan.internal.sentrysoftware.net:3181';
                mc_origin_key='448436';
                mc_origin_sev='4';
                mc_parameter='Speed';
                mc_parameter_value='2775.00';
                mc_parameter_unit='';
                mc_parameter_threshold='';
                mc_event_category=;
                mc_event_subcategory=OTHER;
                mc_event_model_version='1.1.00';
                mc_incident_time=1357819736;
                mc_incident_report_time=0;
                mc_arrival_time=1357819734;
                mc_local_reception_time=1357819734;
                date_reception=1357819736;
                date='20130110130856.000000+060';
                status=OPEN;
                severity=CRITICAL;
                mc_original_severity=CRITICAL;
                mc_priority=PRIORITY_5;
                mc_original_priority=PRIORITY_5;
                mc_owner='';
                mc_long_msg='';
                msg='Alarm #1 of global parameter ''Speed'' triggered on ''MS_HW_FAN.MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c''.  0 <= 2775.00 <= 3000';
                duration=0;
                mc_timeout=0;
                repeat_count=0;
                mc_action_count=0;
                administrator='';
                mc_acl=[];
                mc_date_modification=1357819734;
                mc_notes=[];
                mc_operations=[];
                mc_notification_history=[];
                mc_bad_slot_names=[];
                mc_bad_slot_values=[];
                mc_history=[];
                mc_modhist=[];
                mc_propagations=['pnet_bii4p7:199931'];
                mc_collectors=['1.1','2.2.25.1.1','4.24.21','6.1.1'];
                mc_abstraction=[];
                mc_abstracted=[];
                mc_associations=[];
                mc_cause=0;
                mc_effects=[];
                mc_event_relations=[];
                mc_relation_source='';
                mc_smc_id='';
                mc_smc_alias='';
                mc_smc_impact=NOT_ELECTED;
                mc_smc_type='';
                mc_smc_priority=0.000000e+000;
                mc_smc_causes=[];
                mc_smc_effects=[];
                itsm_category='';
                itsm_type='';
                itsm_item='';
                itsm_product_name='';
                itsm_model_version='';
                itsm_manufacturer='';
                itsm_operational_category1='';
                itsm_operational_category2='';
                itsm_operational_category3='';
                itsm_company='';
                itsm_location='';
                pn_detail_diag=0;
                pn_detail_diag_count=0;
                pn_device_name='';
                p_agent='morgan.internal.sentrysoftware.net';
                p_agent_address='fe80::1454:69d3:25a3:58da';
                p_agent_port=3181;
                p_agent_version='V3.10';
                p_application='MS_HW_FAN';
                p_catalog='STD';
                p_class='11';
                p_diary='';
                p_expectancy='';
                p_handler='Patrol';
                p_instance='MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c';
                p_node='morgan';
                p_origin='MS_HW_FAN.MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c.Speed';
                p_owner='';
                p_source_id='120';
                p_status='0';
                p_type='5';
                p_args=['Alarm #1','global','Speed','MS_HW_FAN.MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c','0','2775.00','3000'];
                p_diary_text=[];
                p_diary_time=[];
                p_diary_operator=[];
                p_class_group='param_group';
                pes=0;
                pes_tcp_port=0;
                pes_udp_port=0;
                pes_icon_name='';
                pes_parent_instance='';
                pes_param_status='';
                pes_alert_date='';
                pes_alert_time='';
                pes_tz='';
                pes_last10='';
                pes_ave10='';
                pes_last10_ts='';
                pes_last10_tp=0.000000e+000;
                pes_user_defined='';
                pes_alarm_min='';
                pes_alarm_max='';
                pes_patrol_home='';
                pes_custom_id1='';
                pes_custom_id2='';
END

Events of Class 41

Events of Class 41 are generated by the Sentry KMs each time a threshold is exceeded and only if the parameter returns a critical value. This event will contain the full health report by detailing the fault that has occurred. Events of Class 41 are generated by the KM based on values collected by the KM.

Here is an example of the Event Class 41 generated by Hardware Sentry KM:

PATROL_EV;
                event_handle=2235788;
                mc_ueid='BII4Patrol/fe80::1454:69d3:25a3:58da/3181/1357819736/448446';
                mc_client_address='192.168.7.24';
                adapter_host='sam.internal.sentrysoftware.net';
                mc_location='internal.sentrysoftware.net';
                mc_service='';
                mc_host_class='NT 6.1 Windows Server 2008 R2 Standard Standard Edition AMD64';
                mc_host='morgan.internal.sentrysoftware.net';
                mc_host_address='fe80::1454:69d3:25a3:58da';
                mc_host_id=0;
                mc_account='';
                mc_object_class='MS_HW_FAN';
                mc_object='MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c';
                mc_object_uri='';
                mc_object_owner='';
                mc_tool_class='BII4PATROL 7.3.60';
                mc_tool='morgan.internal.sentrysoftware.net:3181';
                mc_tool_id='';
                mc_tool_rule='';
                mc_tool_key='448446';
                mc_tool_sev='3';
                mc_tool_address='';
                mc_tool_uri='';
                mc_tool_time=0;
                mc_tool_suggestion='';
                mc_origin_class='V3.10';
                mc_origin='morgan.internal.sentrysoftware.net:3181';
                mc_origin_key='448446';
                mc_origin_sev='3';
                mc_parameter='Speed';
                mc_parameter_value='';
                mc_parameter_unit='';
                mc_parameter_threshold='';
                mc_event_category=;
                mc_event_subcategory=OTHER;
                mc_event_model_version='1.1.00';
                mc_incident_time=1357819736;
                mc_incident_report_time=0;
                mc_arrival_time=1357819734;
                mc_local_reception_time=1357819734;
                date_reception=1357819736;
                date='20130110130856.000000+060';
                status=OPEN;
                severity=CRITICAL;
                mc_original_severity=CRITICAL;
                mc_priority=PRIORITY_5;
                mc_original_priority=PRIORITY_5;
                mc_owner='';
                mc_long_msg='';
                msg='Hardware problem with Fan 1A Tach:2f2901f201f4501f5c. The speed of this fan is critically low (2775 rpm).

Hardware health report (Thu Jan 10 13:08:56 2013)
======================

Monitored object       : Fan 1A Tach:2f2901f201f4501f5c
Type                   : Fan
On host                : localhost
PATROL object ID       : /MS_HW_FAN/MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c
Internal device ID     : Fan 1A Tach:2f2901f201f4501f5c
Connector used         : MS_HW_Director61NT.hdf
Identifying Information:
- Fan 1A

This object is attached to:
Computer: IBM System x3650 M4 -[7915C2G]-

============================================================
Parameter: Speed (Currently in ALARM state)
------------------------------------------------------------
Current value: 2775 rpm
Unit         : RPM (Revolution Per Minute)
Current state: ALARM

Thresholds (manually set):
  - If Speed is in the [0 rpm ; 3000 rpm] range: Trigger an ALARM
  - If Speed is in the [3000 rpm ; 4000 rpm] range: Trigger a WARNING

Problem: The speed of this fan is critically low (2775 rpm).

Consequence: The temperature of the chip, component or device that was cooled  
down by this fan, may rise rapidly. This could lead to severe hardware damage  
and system crashes.                                                             

Recommended action: Check if the fan no longer cools down the system. If so,   
replace the fan.

 

.';
                duration=0;
                mc_timeout=0;
                repeat_count=0;
                mc_action_count=0;
                administrator='';
                mc_acl=[];
                mc_date_modification=1357819734;
                mc_notes=[];
                mc_operations=[];
                mc_notification_history=[];
                mc_bad_slot_names=[];
                mc_bad_slot_values=[];
                mc_history=[];
                mc_modhist=[];
                mc_propagations=['pnet_bii4p7:199932'];
                mc_collectors=['1.1','2.2.25.1.1','4.24.21','6.1.1'];
                mc_abstraction=[];
                mc_abstracted=[];
                mc_associations=[];
                mc_cause=0;
                mc_effects=[];
                mc_event_relations=[];
                mc_relation_source='';
                mc_smc_id='';
                mc_smc_alias='';
                mc_smc_impact=NOT_ELECTED;
                mc_smc_type='';
                mc_smc_priority=0.000000e+000;
                mc_smc_causes=[];
                mc_smc_effects=[];
                itsm_category='';
                itsm_type='';
                itsm_item='';
                itsm_product_name='';
                itsm_model_version='';
                itsm_manufacturer='';
                itsm_operational_category1='';
                itsm_operational_category2='';
                itsm_operational_category3='';
                itsm_company='';
                itsm_location='';
                pn_detail_diag=0;
                pn_detail_diag_count=0;
                pn_device_name='';
                p_agent='morgan.internal.sentrysoftware.net';
                p_agent_address='fe80::1454:69d3:25a3:58da';
                p_agent_port=3181;
                p_agent_version='V3.10';
                p_application='MS_HW_FAN';
                p_catalog='STD';
                p_class='41';
                p_diary='';
                p_expectancy='';
                p_handler='Patrol';
                p_instance='MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c';
                p_node='morgan';
                p_origin='MS_HW_FAN.MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c.Speed';
                p_owner='';
                p_source_id='120';
                p_status='0';
                p_type='5';
                p_args=['Hardware problem with Fan 1A Tach:2f2901f201f4501f5c. The speed of this fan is critically low (2775 rpm).

Hardware health report (Thu Jan 10 13:08:56 2013)
======================

Monitored object       : Fan 1A Tach:2f2901f201f4501f5c
Type                   : Fan
On host                : localhost
PATROL object ID       : /MS_HW_FAN/MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c
Internal device ID     : Fan 1A Tach:2f2901f201f4501f5c
Connector used         : MS_HW_Director61NT.hdf
Identifying Information:
- Fan 1A

This object is attached to:
Computer: IBM System x3650 M4 -[7915C2G]-

============================================================
Parameter: Speed (Currently in ALARM state)
------------------------------------------------------------
Current value: 2775 rpm
Unit         : RPM (Revolution Per Minute)
Current state: ALARM

Thresholds (manually set):
  - If Speed is in the [0 rpm ; 3000 rpm] range: Trigger an ALARM
  - If Speed is in the [3000 rpm ; 4000 rpm] range: Trigger a WARNING

Problem: The speed of this fan is critically low (2775 rpm).                   

Consequence: The temperature of the chip, component or device that was cooled  
down by this fan, may rise rapidly. This could lead to severe hardware damage  
and system crashes.

Recommended action: Check if the fan no longer cools down the system. If so,   
replace the fan.

 

'];
                p_diary_text=[];
                p_diary_time=[];
                p_diary_operator=[];
                p_class_group='normal_group';
                pes=0;
                pes_tcp_port=0;
                pes_udp_port=0;
                pes_icon_name='';
                pes_parent_instance='';
                pes_param_status='';
                pes_alert_date='';
                pes_alert_time='';
                pes_tz='';
                pes_last10='';
                pes_ave10='';
                pes_last10_ts='';
                pes_last10_tp=0.000000e+000;
                pes_user_defined='';
                pes_alarm_min='';
                pes_alarm_max='';
                pes_patrol_home='';
                pes_custom_id1='';
                pes_custom_id2='';
END

Events of Class 9

When the alarm no longer exists, the PATROL Agent generates a new event of class 9. This event only closes the event of class 11; the event of class 41 remains open with the same severity.

inline

PATROL_EV;
                event_handle=2235846;
                mc_ueid='BII4Patrol/fe80::1454:69d3:25a3:58da/3181/1357819955/448451';
                mc_client_address='192.168.7.24';
                adapter_host='sam.internal.sentrysoftware.net';
                mc_location='internal.sentrysoftware.net';
                mc_service='';
                mc_host_class='NT 6.1 Windows Server 2008 R2 Standard Standard Edition AMD64';
                mc_host='morgan.internal.sentrysoftware.net';
                mc_host_address='fe80::1454:69d3:25a3:58da';
                mc_host_id=0;
                mc_account='';
                mc_object_class='MS_HW_FAN';
                mc_object='MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c';
                mc_object_uri='';
                mc_object_owner='';
                mc_tool_class='BII4PATROL 7.3.60';
                mc_tool='morgan.internal.sentrysoftware.net:3181';
                mc_tool_id='';
                mc_tool_rule='';
                mc_tool_key='448451';
                mc_tool_sev='3';
                mc_tool_address='';
                mc_tool_uri='';
                mc_tool_time=0;
                mc_tool_suggestion='';
                mc_origin_class='V3.10';
                mc_origin='morgan.internal.sentrysoftware.net:3181';
                mc_origin_key='448451';
                mc_origin_sev='3';
                mc_parameter='Speed';
                mc_parameter_value='';
                mc_parameter_unit='';
                mc_parameter_threshold='';
                mc_event_category=;
                mc_event_subcategory=OTHER;
                mc_event_model_version='1.1.00';
                mc_incident_time=1357819955;
                mc_incident_report_time=0;
                mc_arrival_time=1357819953;
                mc_local_reception_time=1357819953;
                date_reception=1357819955;
                date='20130110131235.000000+060';
                status=OPEN;
                severity=INFO;
                mc_original_severity=INFO;
                mc_priority=PRIORITY_5;
                mc_original_priority=PRIORITY_5;
                mc_owner='';
                mc_long_msg='';
                msg='Alert on ''MS_HW_FAN.MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c.Speed'' from global parameter ''Speed'' cancelled; exception no longer exists.';
                duration=0;
                mc_timeout=0;
                repeat_count=0;
                mc_action_count=0;
                administrator='';
                mc_acl=[];
                mc_date_modification=1357819953;
                mc_notes=[];
                mc_operations=[];
                mc_notification_history=[];
                mc_bad_slot_names=[];
                mc_bad_slot_values=[];
                mc_history=[];
                mc_modhist=[];
                mc_propagations=['pnet_bii4p7:199934'];
                mc_collectors=['1.1','2.2.25.1.1','4.24.21','6.1.1'];
                mc_abstraction=[];
                mc_abstracted=[];
                mc_associations=[];
                mc_cause=0;
                mc_effects=[];
                mc_event_relations=[];
                mc_relation_source='';
                mc_smc_id='';
                mc_smc_alias='';
                mc_smc_impact=NOT_ELECTED;
                mc_smc_type='';
                mc_smc_priority=0.000000e+000;
                mc_smc_causes=[];
                mc_smc_effects=[];
                itsm_category='';
                itsm_type='';
                itsm_item='';
                itsm_product_name='';
                itsm_model_version='';
                itsm_manufacturer='';
                itsm_operational_category1='';
                itsm_operational_category2='';
                itsm_operational_category3='';
                itsm_company='';
                itsm_location='';
                pn_detail_diag=0;
                pn_detail_diag_count=0;
                pn_device_name='';
                p_agent='morgan.internal.sentrysoftware.net';
                p_agent_address='fe80::1454:69d3:25a3:58da';
                p_agent_port=3181;
                p_agent_version='V3.10';
                p_application='MS_HW_FAN';
                p_catalog='STD';
                p_class='9';
                p_diary='';
                p_expectancy='';
                p_handler='Patrol';
                p_instance='MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c';
                p_node='morgan';
                p_origin='MS_HW_FAN.MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c.Speed';
                p_owner='';
                p_source_id='120';
                p_status='0';
                p_type='2';
                p_args=['MS_HW_FAN.MS_HW_Director61NThdflocalhost_Fan1ATach2f2901f201f4501f5c.Speed','global','Speed'];
                p_diary_text=[];
                p_diary_time=[];
                p_diary_operator=[];
                p_class_group='param_group';
                pes=0;
                pes_tcp_port=0;
                pes_udp_port=0;
                pes_icon_name='';
                pes_parent_instance='';
                pes_param_status='';
                pes_alert_date='';
                pes_alert_time='';
                pes_tz='';
                pes_last10='';
                pes_ave10='';
                pes_last10_ts='';
                pes_last10_tp=0.000000e+000;
                pes_user_defined='';
                pes_alarm_min='';
                pes_alarm_max='';
                pes_patrol_home='';
                pes_custom_id1='';
                pes_custom_id2='';
END

Removing the Events of Class 41

Several options exist to remove the events of Class 11 from bii4p7. One of them consists in writing a new 'rule' that will update the event of class 11 with the information available in the event of Class 11 and close the event of Class 41.

To create this "new" rule:

  • Create a file called sentry.mrl which contains the following code:

new sentry_close_std :
    PATROL_EV ($NEW)
        where [ $NEW.status != CLOSED AND $NEW.status != BLACKOUT AND $NEW.p_class == 41]
                updates PATROL_EV ($OLD)
        where [ $OLD.status != CLOSED AND $OLD.status != BLACKOUT AND
                                       $OLD.adapter_host == $NEW.adapter_host AND
                                       $OLD.mc_object == $NEW.mc_object AND
                                       $OLD.mc_tool_class == $NEW.mc_tool_class AND
                                       $OLD.mc_origin == $NEW.mc_origin]
                                       within 60
                {
                               $NEW.status = CLOSED;
                               $OLD.msg = $NEW.msg
                }
END

  • Save this file in the MCELL_HOME/etc/<cellname>/kb/rules directory and update your .load file accordingly, e.g.:

#
# File name: .load
# Version: 7.4.00
# Copyright 1998-2010 BMC Software, Inc. All Rights Reserved
#
kbversem
kbverssim
mc_startup
im_internal
mc_intevt
impact_admin_server
ips
mc_sm_start
mc_sm_associate
mc_ci_policies
mc_sm_maintenance
mc_sm_elect
mc_sm_attach
mc_sm_shadow
mc_sm_slm
bii4p
sentry
admin_propagate
patrol_portal
ibrsd_event_association
ibrsd_event_incident_info
ibrsd_error_event_association
ibrsd_bem_categorization
ibrsd_sim_remote_action
ibrsd_orphaned_root_cause
ibrsd_root_cause
ibrsd_causal_priority
ibrsd_propagate_causal_events
propagate_bppm

  • Recompile the cell:

mccomp -n <cellname>

  • Restart the cell or run this command to reload kb:

mcontrol -n <cellname> reload kb

The result will be as follow:

inline

When the PATROL alarm no longer exists, the CRITICAL event is closed in the cell:

inline