Tesco value network monitor #2

Slightly modified version of the test-connection script. After a weekend-long server outage which resulted in a few thousand emails, I’ve built some logic in to count the number of failures a server has, and only send emails periodically. In testing, this script would generate an email about every 3 minutes against a list  of 50-odd devices. This bit of code- $numberDetector = $serverArray[$rewriteValue,1] / 10– and the subsequent If… -ge 5 code can be modified to increase/ decrease the email frequency.

It took me ages to work out the logic, but eventually I figured out how to create and manipulate the array at the core of the script. It’s pretty well documented in terms of explanation (that’s why it’s so long, really. Again, apologies for the non-tabbing of the code.

Of course, I could just use System Center OpsMan…

======================================================================================================================

#Tesco value server monitor 🙂
#Repeats a test-connection sequence until 01:00am. Then stops.
#Then starts up again at 03:00.
#This enables modification of the server list to be picked up,
#but also ensure false failures don’t get picked up when the
#servers are being automatically rebooted (01:00-03:00).

#Initialise a group of variables[

#This will be used to populate the array of servers initially and also
#to increment a counter to keep track of the number of test-connection failures
[int]$counter = 0

#This will be used to provide the index to each server in the array incrementally
[int]$index = 0

#This will be used to find the count value when a specific index is found.
#This is important; the count will always be one higher than the array index.
[int]$indexFound = 0

#This will be used to re-wrtie the array element.
#It will always be one lower than $indexFound, because the count will start at 1 whereas arrays start from 0
[int]$rewriteValue = 0

#This is a string with the list of servers in it
[string]$serverList = get-content c:\Users\xxxx\desktop\pingList.txt

#This is an array of servers, comprising cells in a (number of servers) x 3 matrix.
#Each server has: an index, failure counter number and name.
#The array takes its vertical size dimension from the number of items in $serverArray, so it will expand as needed
$serverArray = New-Object ‘object[,]’ $serverList.Count,3

#This loop identifies each individual server from the $serverList string, and
#Populates each element of the array in sequence. Arrays always count from 0, so a 5-row array will contain elements 0-4.
#This is a 2-dimensional array, so in the below example the [$index part refers to the “row” of the array. the ,x] part refers to the “column” part.
#This means the first server in the list will have element 0 (index) set to 0, element 1 (counter) to 0 and element 3 (name) to the server name
Foreach($server in $serverList)
{
#These lines write:
#the value of $index to $serverArray row $index, element 0. This will be used to provide a unique index number for the array row.
#the value of $counter to $serverArray row $index, element 1. This will be use to keep track of the number of test-connection failures.
#the value of $server to $serverArray row $index, element 2. This will be used to store the server name
$serverArray[$index,0] = $index
$serverArray[$index,1] = $counter
$serverArray[$index,2] = $server

#increments the $index variable by 1
$index++
}

#Entering Do loop
do {

#Sets $theTime variable to the current time.
$theTime = Get-Date
#Sets $now variable to the hour.
$now = $theTime.Hour

#Reads from a list of servers
get-content c:\Users\xxxx\desktop\pingList.txt | Foreach-Object {

#Sets $ComputerName variable to the most recent item ($_) in memory
$computerName=$_

#Sets $Result variable to output of test-connection
$Result = test-connection $_ -Quiet

#The purpose of this loop is to stop the failed counter of a server remaining
#at the threshold level even after it’s started passing the test-connection process.
#Without this, a server would gradully build up enough failed test-connections to trigger the email alerts.
#If $Result is true (i.e. this machine can connect to the remote server),
#Sets the value of the “counter” element of the given array row to be 0.
#It also writes some stuff to screen, but this is for testing really.
If($result -eq $true)
{
#These lines are commented out, for testing only
#Write-Host “”
#Write-Host $_ “The server is fine”

#This sets $indexFound to the value of the count at which the server name was found.
#It looks for the array row which contains a match for $computerName- i.e. the server name.
#It then finds how many rows it’s read in order to get there.
#This will be one higher than the actual array index.
$indexFound = $serverArray -match $computerName | foreach { ‘{0}’ -f $_.ReadCount}

#This sets the $rewriteValue variable to the value of $indexFound -1 so that we get the correct array row.
$rewriteValue = $indexFound – 1

#This sets the 2nd element of the array row containing the server name to be 0.
#Otherwise, the server would gradually build up enough missed test-connections to trigger the alert emails,
#even tho’ it had only been through a few routine reboots, for example.
$serverArray[$rewriteValue,1] = 0

}
#The purpose of this loop is to deal with failed test-connection processes.
#
ElseIf ($Result -eq $False)
{
#These lines are commented out, for testing only
#Write-Host “”
#Write-Host $_ “Failed”

#This sets $indexFound to the value of the count at which the server name was found.
#It looks for the array row which contains a match for $computerName- i.e. the server name.
#It then finds how many rows it’s read in order to get there.
#This will be one higher than the actual array index.
$indexFound = $serverArray -match $computerName | foreach { ‘{0}’ -f $_.ReadCount}

#This sets the $rewriteValue variable to the value of $indexFound -1 so that we get the correct array row.
$rewriteValue = $indexFound – 1

#This sets the 2nd element of the array row containing the server name to be itself plus .
#Otherwise, the server would gradually build up enough missed test-connections to trigger the alert emails,
#even tho’ it had only been through a few routine reboots.
$serverArray[$rewriteValue,1] = $serverArray[$rewriteValue,1] + $counter

#This variable is purely to slow down the number of emails sent.
#It divides the number of failures by 10, which will either give an Int32 (whole number)
#or a Double (number with decimal places). This is used by the If loop below.
$numberDetector = $serverArray[$rewriteValue,1] / 10

#These lines just output stuff to screen for testing, so are commented out.
$serverArray[$rewriteValue,1]
#$numberDetector.GetType()

#The purpose of this loop is to send emails, but only if the $numberDetector variable is in an Int32 state.
#This only happens on every 10th iteration of the failed test-connection attempts, so will slow the generation of emails down significantly.
If(($serverArray[$rewriteValue,1] -ge 5) -and ($numberDetector.GetType().Name -eq ‘Int32′))
{
#Define hub transport server
$smtp_server = “yourmailserver.yourdomain.com”

#Define email sender and recipient
$sender = “GI Joe <g.i.joe@yourdomain.com>”
$recipient = “A N Other <a.n.other@yourdomain.com>”

#Define email subject and body
$msg_subject = “Important! Server $computerName is not responding to ping requests”
$msg_body_text = “Server $computerName has failed to respond to numerous ping requests. Please investigate urgently.The script that generated this alert is \\someserver\support\scripts\ping.ps1”

#Send it
Send-MailMessage -to $recipient -from $sender -subject $msg_subject -body $msg_body_text -smtpserver $smtp_server
}
$counter++
}
}

#As soon as the script hits 01:00am, this script will stop.
} While ($now -ne ’01’)

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s