Simple SSL Certificate Expiry Monitoring in Zabbix

To monitor SSL certificate expiry dates in Zabbix, a simple SSH script will execute the SSL certificate check and update the date. SSL expires at various times and it can be quite hard to manage. Zabbix HTTPS Certificate Monitoring is available with zabbix-agent2 that works without any external scripts but if you want to continue with our existing zabbix agent, the below simple script is advisable with single items.

SSL Certificate Expiry Monitoring

Login to your Zabbix agent host SSH.

Go to /etc/zabbix/zabbix_agentd.conf.d directory which is the common for zabbix and named the file checkssl.sh

$sudo vim  checkssl.sh
data=`echo | openssl s_client -servername $1 -connect $1:443 2>/dev/null | openssl x509 -noout -enddate | sed -e 's#notAfter=##'`

ssldate=`date -d "${data}" '+%s'`
nowdate=`date '+%s'`
diff="$((${ssldate}-${nowdate}))"

echo $((${diff}/86400))

then give it execute permissions.

$ sudo chmod 755 checkssl.sh

then test SSL certificate expiry for various websites we manage, and also many others. 

For example,

$ ./checkssl.sh cloudkb.net
$ ./checkssl.sh github.com

The command should return a number indicating how many days are left before the SSL certificate expires.

Check SSL Expire date

This script can be called any way you like for the particular use case. Now, Let us enable “EnableRemoteCommands=1” in /etc/zabbix/zabbix_agentd.conf

$ sudo vim /etc/zabbix/zabbix_agentd.conf

And set EnableRemoteCommands=1

Restart the Zabbix agent

$ sudo systemctl restart zabbix-agent

How to configure in Zabbix Server?

Let us open zabbix server web page and configure the host items.

Configuration -> Select Hosts -> Click Items, and then press the Create Item button to get the new item configuration.

And fill in the details as seen in this image.

Zabbix SSL Check Item

For example, System run settings.

Key : system.run[/etc/zabbix/zabbix_agentd.conf.d/checkssl.sh localhost]

Or 

Key : system.run[/etc/zabbix/zabbix_agentd.conf.d/checkssl.sh <your_website>]

Type of information: Numeric (float)

After saving, select the new item you created, and press the Test button.

Then Goto, Monitoring -> Host -> Latest Data and filter for the host you added the item to, and after “Update Interval” which you configured should see a new property appear somewhere in the list titled SSL Check.

Now you can configure the triggers to alert when the expiry days remain below 30 days or whatever you decide is important.

Example,

Zabbix SSL check Triggers

You can copy the existing trigger and modify the date that you want. Update your comments if you have any troubles.

Use the below SSL Certificate Expire check in Zabbix Template.

 

Integrate Pagerduty with Zabbix Monitoring

This guide describes how to integrate your Zabbix 4.4 installation with PagerDuty using the Zabbix webhook feature. This guide will provide instructions on setting up a media type, a user, and an action in Zabbix.

Why PagerDuty

  • You can send notifications through the various integrated collaboration tools that PagerDuty supports, including SMS, push notifications, phone calls, and email.
  • Monitoring teams can reach out to the subject matter expert, who can help resolve customer critical infra issues.
  • PagerDuty provides flexibility by having rotating on-call schedules based on business hours shift, overnight on-call shift, and weekend shift so reaching out to the appropriate engineer can be achieved seamlessly. 

In PagerDuty

1. From the Configuration menu, select Services.

2. On your Services page:

  • If you are creating a new service for your integration, click +New Service.

  • If you are adding your integration to an existing service, click the name of the service you want to add the integration to. Then click the Integrations tab and click the +New Integration button.

3. Select Use our API directly and Events API v2 from the Integration Type menu and enter an Integration Name. If you are creating a new service for your integration, in General Settings, enter a Name for your new service.

4. Click the Add Service or Add Integration button to save your new integration. You will be redirected to the Integrations page for your service.

5. Copy the Integration Key for your new integration:

In Zabbix

The configuration consists of a media type in Zabbix, which will invoke webhook to send alerts to PagerDuty through the PagerDuty Event API v2. To utilize the media type, we will create a Zabbix user to represent PagerDuty. We will then create an alert action to notify the user via this media type whenever there is a problem detected.

Create Global Macro

1. Go to the Administration tab.

2. Under Administration, go to the General page and choose the Macros from drop-down list.

3. Add the macro {$ZABBIX.URL} with Zabbix frontend URL.

4. Click the Update button to save the global macros.

Create the PagerDuty media type

1. Go to the Administration tab.

2. Under Administration, go to the Media types page and click the Import button.

3. Select Import file media_pagerduty.yaml and click the Import button at the bottom to import the PagerDuty media type.

4. Change the value of the variable token

Create the PagerDuty user for alerting

1. Go to the Administration tab.

2. Under Administration, go to the Users page and click the Create user button.

3. Fill in the details of this new user, and call it “PagerDuty User”. The default settings for PagerDuty User should suffice as this user will not be logging into Zabbix.

4. Click the Select button next to Groups.

  • Please note, that in order to notify of problems with the host this user must have at least read permissions for the such host.

5. Click on the Media tab and, inside of the Media box, click the Add button.

6. In the new window that appears, configure the media for the user as follows:

  • For the Type, select PagerDuty (the new media type that was created).
  • For Send to: enter any text, as this value is not used, but is required.
  • Make sure the Enabled box is checked.
  • Click the Add button when done.

7. Click the Add button at the bottom of the user page to save the user.

8. Use the PagerDuty User in any Actions of your choice.

For more information, use the Zabbix and PagerDuty documentation.

 

vCenter /storage/archive storage partition 100% full

First, understand that this issue does not affect any operations of vCenter Server as the /storage/archive partition can be full by design. This volume stores as much WAL history as possible, and is automatically cleaned up by the archiver service by automatically removing the oldest WAL segments. We often getting the below error vCenter /storage/archive storage partition 100% full.

But if you still want to fix it to avoid any monitoring alerts or anything in the security. 

First, SSH into the vCenter serve log in as root. after successful login type shell to enable the shell

Refer the below command to see more details about the disk mounts.

lsblk; lsscsi
vCenter disk usage

As you can see from the images the drive /storage/archive is at 96% capacity. The /storage/archive mount point is named sdm. The sdm is located on disk 13. So lets expand disk 13.

Log into the vSphere client and find the vCenter appliance then edit settings, in this case, located disk 13 and add additional space.

vCenter Virtual Hardware

Fix for the vCenter /storage/archive storage partition full error

Once increased the partition size in the vCenter used the above steps.

In the SSH session to the vCSA, run the autogrow script “/usr/lib/applmgmt/support/scripts/autogrow.sh”

/usr/lib/applmgmt/support/scripts/autogrow.sh

Run the “df -h” command and verify the “/storage/archive” mount.

After all the process completed, verify the vSphere Client System Configuration Node Health is “Good” and verify the vCenter Server Appliance VAMI Health Status for Storage is “Good” .

This issue is an erroneous alarm that does not affect the operations of vCenter Server. But there is a permanent fix after installing the new version 6.7 Update VMware Downloads, there will no longer be warnings in the Health Status portion. There will be no alarm even if partition is 100% full – as this is by design, and has no impact on the running of the vCenter.