SSH-Based Connectors Stopped Working

How to diagnose SSH issues when SSH-based connectors stop working.

Related Topics

Downloads

Introduction

If your SNMP-based connectors suddenly stop working, you can use the diagnosis tool installed with Hardware Sentry to troubleshoot the issue. Depending on your version of Hardware Sentry, you will either leverage the MS_HW_SSHRemoteUnixCommand.jar or MS_HW_matsya-module-sshclient.jar tool which can be found in:

  • %PATROL_HOME%\bin (Windows)
  • $PATROL_HOME/../bin (Linux/Unix)

The diagnosis tool being Java-based, it requires Java to be installed. A package with a suitable Java Runtime Environment is available to download on Sentry’s Website.

To launch the tool, you will have to run the java -jar <jar file name> command:

Usage:
java -jar MS_HW_SSHClient.jar --host <hostname> --login <username> [options]
Where:
--host <hostname> specifies the hostname to connect to
--port <port> specifies the ssh port to connect to
--login <username> specifies the user to log in as
--key <privateKey file> specify the path of the privatekey

Example

Let’s assume that the HP BladeSystem (SSH) connector has stopped working and that the following error message appears in the Test Report parameter of the connector:

TELNETINTERACTIVE TEST on fermat-oa: FAILED
----------------------------------
Disconnected or timeout while waiting for ">" from fermat-oa through SSH

A PATROL_EV event should also been generated and the msg slot should contain the following information:

PATROL_EV event triggered when SSH connector fails

In this example:

  • fermat-oa is the HP Blade chassis monitored
  • Java is installed in %PATROL_HOME%\jre64\bin (Windows) or $PATROL_HOME/../jre64/bin (Linux/Unix).

To troubleshoot the issue:

  1. We run the following command against the monitored server:
  • When using MS_HW_SSHRemoteUnixCommand.jar or MS_HW_matsya-module-sshclient.jar:

    • On Windows:

      java -jar %PATROL_HOME%\bin\MS_HW_matsya-module-sshclient.jar -hostname:fermat-oa -username:monitor -password:xxxxxx
      
      Result:
      Authenticating as monitor...
      MS_HW_matsya-module-sshclient.jar: Error: Failed to authenticate as monitor on fermat-oa
      
    • For Linux/UNIX:

      java -jar $PATROL_HOME/../bin/MS_HW_matsya-module-sshclient.jar -hostname:fermat-oa -username:monitor -password:xxxxxx
      
      Result:
      Authenticating as monitor...
      MS_HW_matsya-module-sshclient.jar: Error: Failed to authenticate as monitor on fermat-oa
      

The error message above indicates that the connector was not able to connect to the HP Blade chassis using the “monitor” account. With a different account, like root for example, we are able to connect to the chassis:

java -jar MS_HW_matsya-module-sshclient.jar -hostname:fermat-oa -username:root -password:xxxxxx
Authenticating as root...

HP BladeSystem Onboard Administrator
(C) Copyright 2006-2015 Hewlett-Packard Development Company, L.P.


Type 'HELP' to display a list of valid commands.
Type 'HELP <command>' to display detailed information about a specific command.
Type 'HELP HELP' to display more detailed information about the help system.


fermat-oa>

The show user “monitor” command indicates this account has been disabled:

fermat-oa> show user "monitor"
show user "monitor"

Local User "monitor" Information:
       Full name:
       Contact Info:
       User Rights: User
       Account Status: Disabled
       Server Bay Access List:
       Interconnect Bay Access List:
       OA Access: No

If the “monitor” account has been accidentally disabled, we can run the following command to enable it:

fermat-oa> enable user "monitor"
enable user "monitor"

User "monitor" has been enabled.

The connector’s Test Report now displays:

TELNETINTERACTIVE TEST on fermat-oa: SUCCEEDED
----------------------------------------------
Obtained "HP.* BladeSystem Onboard Administrator" in the SSH session with fermat-oa

And we are now able to use the “monitor” account to connect to the HP Blade chassis:

java -jar MS_HW_matsya-module-sshclient.jar -hostname:fermat-oa -username:monitor -password:xxxxx
Authenticating as monitor...

HP BladeSystem Onboard Administrator
(C) Copyright 2006-2015 Hewlett-Packard Development Company, L.P.


Type 'HELP' to display a list of valid commands.
Type 'HELP <command>' to display detailed information about a specific command.
Type 'HELP HELP' to display more detailed information about the help system.

fermat-oa>
Note:We could also have achieved the verifications above using an 3rd-party SSH client such as Putty.