SCOM Dashboard for your whole infrastructure

Hello SCOM funs! 

Last months I was very busy with developing very nice visual dashboard which can in one big picture show overall status of your whole monitored environment. This goal was not so easy if you imagine how much of valuable data SCOM has and how difficult it can be to display them in one picture. 

Off course you can have more dashboards, but the best option is to have only one big dashboard permanently projecting on one big screen in your office, so everybody can see it in one quick glance and understand that everything is all right or if there is some issue, you will see where you need to start your remediation job.

For visualization layer I used Visio, where I can draw anything without any limitations. The problem was an amount of SCOM data that I needed to visualize. For that purpose, I developed thousand-line PowerShell script, which is reading SCOM health and performance data and making aggregations and storing those data in temporary SQL tables. Visio dashboard has its shapes linked to this SQL tables, and if there is any change in monitoring data, Visio dashboard immediately render this health status in graphical way.

Look at one of many possibilities you can get from this:

The lower part in dashboard is about VMware infrastructure. It is rendering health information about Datacenters, Clusters, Hosts, Datastores, Networks and individual VMs. All health data are gathered via free VMware MP which is described in my blog post http://wininframaster.blogspot.sk/2017/08/monitoring-vmware-with-scom.html

In the middle you can see Windows Servers layers. All servers are grouped in logical areas, such as AD, Exchange, DB and so on. You can even display your Unix servers here as well. It is totally up to SCOM admin. Only thing he needs to do is to create in SCOM console normal group, call it according to name convention (eg: Dashboard Service AD) and add servers he want to aggregate in one logical group. 

Then in SQL Table you will see new line for your logical group of servers with all aggregated health statuses. And in Visio diagram you can simply copy template for logical group and link it in one click to new SQL Table line and that’s it.

Visio Template linked to one SQL table line is looking like this:

You can simply remove not relevant tiles and rearrange the rest of them. 

If you have SharePoint, you can store your Visio picture directly inside it. But if you want, you can use bare IIS and render Dashboard picture in simple web page. For this purpose, I created second PowerShell script, which is creating invisible Visio COM object, refreshing linked data sources and exporting dashboard to jpeg file on IIS server. On IIS I have nice web page with intelligent javascript refresh function which is reloading actual picture without any blink. It works simply great!

If you are interested in this very elegant solution, do not hesitate to send me an email at wininframaster@gmail.com. I already made a few SCOM guys and their managers very happy with this overall picture of their monitored infrastructure :) Thanks.

Prepare Orchestrator for PowerShell activities

Maybe you hit already various problems with .NET activity in Orchestrator for calling PowerShell scripts. The problem is that this activity is using only PowerShell version 2.


This is very limiting in your automation scenarios. I sow many workarounds by using PowerShell Remoting but those scripts were very complex and not so easy to use.


Fortunately there exists better and simpler way how to avoid this limitation. It is registry hack on Orchestrator server. Simply edit your registry path HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework. Add hier a new DWORD entry and value for OnlyUseLatestCLR = 1.


That is all. Now you can without any limitation use latest available PowerShell version on your Orchestrator server.


Second very often limitation in your automation effort can be calling web request with Invoke-WebRequest command-let. The problem is that it is using IE engine for actual request and if your automation script is running under service account, very often this account never used IE before. When you open IE for the first time you can see this dialog:

It is the classical First Run customization wizard. Many organization are disabling this wizard by GPO but very often this GPO is not linked to OU with service accounts.

So if you see in your PowerShell runbooks this error:

Invoke-WebRequest : The response content cannot be parsed because the Internet Explorer engine is not available, or Internet Explorer’s first-launch configuration is not complete. Specify the UseBasicParsing parameter and try again.

it is because your account never went through IE First Run Customization Wizard and Invoke-WebRequest cannot make it's job.

To avoid this error, simply run gpedit on your Orchestrator server and open this setting:

Computer Configuration > Policies > Administrative Templates > Windows Components > Internet Explorer. Set the policy Prevent running First Run Wizard to Enabled.

With this settings you should be able to accomplish majority of your automation without bigger issue :)



Combine arrays with hash tables in PowerShell

Very often when I'm scripting something more complex in PowerShell, I found very powerful to combine arrays with hash tables.
I like hash tables because of the relationship between key and their values, but what if you need to have more values bound to one key? No problem! Insert your array directly to hash table. Lets see how:


1) Declare your empty hash table:
$Hash = @{}


2) Prepare your array variables:
$Group1 = "a", "b", "c"


$Group2 = "d", "e", "f"


3) Add array variables directly to hash table:
$Hash.Add("Group1", $Group1)


$Hash.Add("Group2", $Group2)


Of course my recommendation is to at first check if key does not exists in hash and only then add it, or if it already exists, you need to set existing key-value pair inside hash table. You can use this approach:
If($Hash.ContainsKey($key))
{
 $Hash.Set_Item($Key, $Value)
}
Else
{
  $Hash.Add($Key, $Value)
}


4) See the result:
$Hash

You should see this output:

Now you can use your array values in any way you need:
$Values = $Hash.Get_Item("Group1")
$Value1 = $Values[0]
$Value2 = $Values[1]
$Value3 = $Values[2]


Hope this short tutorial was helpful at least for me when I will need combine arrays with hash next time :)

Monitoring VMware with SCOM

Are you using SCOM for monitoring your Windows servers? Are you heavily virtualizing your servers with VMware? If yes, it will be nice to have one big picture of your whole Windows infrastructure together with virtualization platform.


In my last SCOM projects, customers where seeking some free solution how to get health status of their VMWare infrastructure into SCOM. Fortunately on Internet you can find free community VMWare management pack for SCOM: https://github.com/Mitch-Luedy/Community.VMware


It is very simply solution, but for visualizing health state of your VMware world is sufficient.


Lets look how to implement it:


1) Install SCOM agent on your VMware vCenter server. If you have more vCenter servers in your environment, no problem, install it on all.


2) Install VMWare vSphere PowerCLI on SCOM management server. You can download it directly from VMWare web site. Do not install latest version because it is not using PowerShell modules any more and they are required by MP workflows.
In my last project I used successfully version VMware-PowerCLI-6.3.0-3737840. You can install it on one SCOM management server or on more. You are controlling which management server will be used by Community - VMware Monitoring Resource Pool after importing MPs.


3) Now you need to grant rights to VMware infrastructure. You can use SCOM Default Action Account and delegate read only access in VMware or you can use dedicated domain service account. This service account you need to specify in Community - VMware Monitoring Profile after importing MPs.


From network perspective, you need open TCP port 443 from your SCOM management servers to VMware vCenter server/s. Of course, SCOM agent needs TCP port 5723 from vCenter servers to SCOM management server.


4) Import MPs Community.VMware.mpb and Community.VMware.Unsealed.xml

And that's it. Wait until your VMware vCenter servers will be discovered. Then PowerShell workflows will run automatically on them and populate SCOM object classes for VMware Datacenters, Clusters, Hosts, Networks, Data stores and Virtual Machines.

I will show you in upcoming posts how you can use those classes for visualizing health data in nice Visio based dashboards.

Thanks Community for this wonderful MP!

SCOM alerts in Zabbix console

In previous post I was describing possible simple approach how to get single point of view of two independent monitoring solutions - Zabbix and SCOM - in one SCOM console. But what if you need opposite direction of integration?

Of course you can accomplish it and what is beautiful you can do it also only with SC Orchestrator in the middle. Let's see how.

In Orchestrator I have two Runbooks:

1) Send Alert to Zabbix

This Runbook monitors new alerts in SCOM. Then for each new alert I'm writing event message to the custom event log which is located on Orchestrator server. You can create it easily with:

$LogName = “Zabbix SCOM Sync”

$Source = “ZabbixSCOMSync”

New-EventLog -LogName $LogName -Source $Source

I'm using for new event the same error levels as they have in SCOM (as severity). The rest of magic of creating new Zabbix alerts is configured directly in Zabbix. The reason is that I didn't find any way in JSON-RPC how to do that via Zabbix's API. So for creation of alerts I'm utilizing Zabbix agent on Orchestrator server.

At first Install Zabbix agent on your Orchestrator server. Then go to Zabbix console and add it as host. After that I created new Item with following configuration:

This item will be responsible for catching all event written to the event log.

As final step you need to create three triggers in Zabbix. Each one for particular SCOM's severity level:

Now you are ready to run you new Runbook. Wen new alert will be created in SCOM, it will be captured with Runbook and written to the synchronization Event log from which it is immediately picked up by Zabbix agent and written to the Zabbix console as new Alert. 

Note: I'm adding to the message also ID of SCOM alert.

2) Resolve Zabbix's alert when it close in SCOM

Second Runbook is for acknowledging and closing Zabbix's alerts when the original SCOM's alert is closed.

First activity is triggered only when SCOM alert is closed. Then second activity is doing all the magic:

• Looking in the event log to find exact time it was written to it. It is the time when Zabbix created this alert in it's own console.
• Searching (with JSON-RPC) all open trigger's events in Zabbix and get the one with the same time as event in event log was created
• Acknowledge and close (with JSON-RPC) identified Zabbix event

And that's it. Try to close your SCOM alert and check in Zabbix console. It should be something like this:

Issues

The biggest issue was limiting on Zabbix's API. It is not possible to directly create new alert so I have to use Zabbix's agent for that.

The more serious issue is that it is not possible to identify events for acknowledging by searching in their messages. I spend a few hours with finding way but without luck. Finally I hade to use time comparison, but fortunately it seems to be working very well.

Another problem was that Zabbix is cutting text in messages to only 20 characters if they are from log monitoring. Fortunately you can change this behavior directly in Zabbix backend:

In my environment I changed it to 200 characters.

And that's it. I hope it was useful also for you and maybe it can open new integration scenarios for you as well.