Integrate Pagerduty with Zabbix

This guide describes how to integrate your Zabbix 4.4 installation with PagerDuty using the Zabbix webhook feature. This guide will provide instructions on setting up a media type, a user, and an action in Zabbix.

In PagerDuty

1. From the Configuration menu, select Services.

2. On your Services page:

  • If you are creating a new service for your integration, click +New Service.

  • If you are adding your integration to an existing service, click the name of the service you want to add the integration to. Then click the Integrations tab and click the +New Integration button.

3. Select Use our API directly and Events API v2 from the Integration Type menu and enter an Integration Name. If you are creating a new service for your integration, in General Settings, enter a Name for your new service.

4. Click the Add Service or Add Integration button to save your new integration. You will be redirected to the Integrations page for your service.

5. Copy the Integration Key for your new integration:

In Zabbix

The configuration consists of a media type in Zabbix, which will invoke webhook to send alerts to PagerDuty through the PagerDuty Event API v2. To utilize the media type, we will create a Zabbix user to represent PagerDuty. We will then create an alert action to notify the user via this media type whenever there is a problem detected.

Create Global Macro

1. Go to the Administration tab.

2. Under Administration, go to the General page and choose the Macros from drop-down list.

3. Add the macro {$ZABBIX.URL} with Zabbix frontend URL.

4. Click the Update button to save the global macros.

Create the PagerDuty media type

1. Go to the Administration tab.

2. Under Administration, go to the Media types page and click the Import button.

3. Select Import file media_pagerduty.yaml and click the Import button at the bottom to import the PagerDuty media type.

4. Change the value of the variable token

Create the PagerDuty user for alerting

1. Go to the Administration tab.

2. Under Administration, go to the Users page and click the Create user button.

3. Fill in the details of this new user, and call it “PagerDuty User”. The default settings for PagerDuty User should suffice as this user will not be logging into Zabbix.

4. Click the Select button next to Groups.

  • Please note, that in order to notify of problems with the host this user must have at least read permissions for the such host.

5. Click on the Media tab and, inside of the Media box, click the Add button.

6. In the new window that appears, configure the media for the user as follows:

  • For the Type, select PagerDuty (the new media type that was created).
  • For Send to: enter any text, as this value is not used, but is required.
  • Make sure the Enabled box is checked.
  • Click the Add button when done.

7. Click the Add button at the bottom of the user page to save the user.

8. Use the PagerDuty User in any Actions of your choice.

For more information, use the Zabbix and PagerDuty documentation.

 

vCenter /storage/archive storage partition 100% full

First, understand that this issue does not affect any operations of vCenter Server as the /storage/archive partition can be full by design. This volume stores as much WAL history as possible, and is automatically cleaned up by the archiver service by automatically removing the oldest WAL segments. We often getting the below error vCenter /storage/archive storage partition 100% full.

But if you still want to fix it to avoid any monitoring alerts or anything in the security. 

First, SSH into the vCenter serve log in as root. after successful login type shell to enable the shell

Refer the below command to see more details about the disk mounts.

lsblk; lsscsi
vCenter disk usage

As you can see from the images the drive /storage/archive is at 96% capacity. The /storage/archive mount point is named sdm. The sdm is located on disk 13. So lets expand disk 13.

Log into the vSphere client and find the vCenter appliance then edit settings, in this case, located disk 13 and add additional space.

vCenter Virtual Hardware

Fix for the vCenter /storage/archive storage partition full error

Once increased the partition size in the vCenter used the above steps.

In the SSH session to the vCSA, run the autogrow script “/usr/lib/applmgmt/support/scripts/autogrow.sh”

/usr/lib/applmgmt/support/scripts/autogrow.sh

Run the “df -h” command and verify the “/storage/archive” mount.

After all the process completed, verify the vSphere Client System Configuration Node Health is “Good” and verify the vCenter Server Appliance VAMI Health Status for Storage is “Good” .

This issue is an erroneous alarm that does not affect the operations of vCenter Server. But there is a permanent fix after installing the new version 6.7 Update VMware Downloads, there will no longer be warnings in the Health Status portion. There will be no alarm even if partition is 100% full – as this is by design, and has no impact on the running of the vCenter.

 

Install and setup own gitolite server in ubuntu

Gitolite is small, simple, and powerful. the backend application managing the fiddly bits that you configure using Git.  You can easily set up your own git server with as many users and as many repositories as can be stored in the space on your server. 

Gitolite client and server

First, let’s log in on the remote machine; the one you want to set up Gitolite on.

ssh [email protected]

Then install git:

sudo apt-get install git
sudo adduser git
sudo passwd -l git

It’s time to create the $HOME directory for the git user; then we will create SSH public key and adjust some permissions and directory ownership:

sudo ssh-keygen
sudo chmod 700 /home/git/.ssh
sudo chown -R git:git /home/git/

Let’s impersonate the git user, then clone and install Gitolite:

sudo su git -l
git clone https://github.com/sitaramc/gitolite ~/gitolite
mkdir ~/bin
~/gitolite/install -to ~/bin

Exit the git user shell and re-login to make the commands we just installed in ~/bin available to us.

Then setup Gitolite with your public key.

exit
sudo su git -l
gitolite setup -pk ~/.ssh/id_rsa.pub
exit
exit

Now let’s clone the gitolite-admin master repo:

git clone [email protected]:gitolite-admin ~/gitolite-admin

open the ~/gitolite-admin/conf/gitolite.conf file and put these lines to create a new repo and permissions,

repo my-repo
      RW+ = user1

Move your public key to the below location.

cd ~/gitolite-admin/keydir
vi user1.pub

And paste your public key details.

After that you’ll need to commit and push those changes to the remote gitolite-admin repo:

cd ~/gitolite-admin/
git add .
git commit -m "Add new repo, keys, access"
git push

That’s it

Cloning the new repo

Log in to your local machine and cloning the new repo (make sure your key permission in gitolite.conf to pull repo from remote machine)

git clone [email protected]:my-repo

Now you can add content to it and use git as usual.

Git Daemon


Next, we’ll set up a daemon serving repositories using the “Git” protocol. This is a common choice for fast, unauthenticated access to your Git data. Remember that since this is not an authenticated service, anything you serve over this protocol is public within its network.

In any case, the Git protocol is relatively easy to set up. Basically, you need to run this command in a daemonized manner:

$ git daemon --reuseaddr --base-path=/home/git/repositories/ /home/git/repositories/

The –reuseaddr option allows the server to restart without waiting for old connections to time out.

If you’re running a firewall, you’ll also need to punch a hole in it at port 9418 on the box you’re setting this up on.

Since systemd is the most common init system among modern Linux distributions, you can use it for that purpose. Simply place a file in /etc/systemd/system/git-daemon.service with these contents:

[Unit]
Description=Start Git Daemon
 
[Service]
ExecStart=/usr/bin/git daemon --reuseaddr --base-path=/home/git/repositories/ /home/git/repositories/
 
Restart=always
RestartSec=500ms
 
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=git-daemon
 
User=git
Group=git
 
[Install]
WantedBy=multi-user.target

Finally, you’ll run systemctl enable git-daemon to automatically start the service on boot, and can start and stop the service with, respectively, systemctl start git-daemon and systemctl stop git-daemon.

Next, you have to tell Git which repositories to allow unauthenticated Git server-based access to. You can do this in each repository by creating a file named git-daemon-export-ok.

$ cd /home/git/repositories/<your repo>
$ touch git-daemon-export-ok

The presence of that file tells Git that it’s OK to serve this project without authentication.

Done, Access the repo via the below command.

$ git clone git://10.14.48.30/toolchain.git