Difference between revisions of "Basestation Monitoring with Zabbix"

From YateBTS
Jump to: navigation, search
(Monitoring Unmet Demands)
(Monitoring Unmet Demands)
 
(10 intermediate revisions by 2 users not shown)
Line 18: Line 18:
 
In order to have the full functionality as per this guide, you need to have:
 
In order to have the full functionality as per this guide, you need to have:
 
# Yate-enb 7825 or newer
 
# Yate-enb 7825 or newer
# Yate-common 1.6.1 or newer
+
# Yate-common 1.7.1 or newer  
 +
 
 +
Note: Yate-common 1.7.1 is not currently part of the public release yet but its released for labkits.
  
 
== Zabbix server Installation ==
 
== Zabbix server Installation ==
Line 54: Line 56:
  
 
Step1: Add the template to monitor the basestation
 
Step1: Add the template to monitor the basestation
 +
 +
All Radio Access Network Templates can be downloaded from '''[https://wiki.yatebts.com/images/a/a8/zbx_export_RAN_templates.xml Zabbix YateRAN Templates]'''
 +
 +
For more details about what the templates contain and items detail, please refer to [https://wiki.yatebts.com/index.php/Zabbix_Monitoring_Template_Predefined_Items Zabbix Monitoring Template Predefined Items]
  
 
* Save the template template.xml
 
* Save the template template.xml
Line 61: Line 67:
 
<br />
 
<br />
 
* You should get a message with import successfuly and you can find the template when you search for (sdr)
 
* You should get a message with import successfuly and you can find the template when you search for (sdr)
[[File: zabbix-imported-template.png |800px]]
+
[[File: zabbix-imported-template.png |800px]]<br />
 
[[File:zabbix-search-for-template.png |800px]]
 
[[File:zabbix-search-for-template.png |800px]]
  
Line 83: Line 89:
 
* Option 2: Link for the group of hosts:  Configuration -> Templates -> select (Template YateSDR) -> add the group of your hosts in the filed groups -> update.
 
* Option 2: Link for the group of hosts:  Configuration -> Templates -> select (Template YateSDR) -> add the group of your hosts in the filed groups -> update.
 
[[File:zabbix-link-template-group.png |800px]]
 
[[File:zabbix-link-template-group.png |800px]]
 
  
 
== Monitoring Triggers ==
 
== Monitoring Triggers ==
Line 104: Line 109:
  
 
=== Application Restarted  ===
 
=== Application Restarted  ===
When Yate-sdr service restarts it will show a trigger named Yate-sdr service restarted with average severity.  
+
When Yate-sdr service restarts it will show a trigger named Yate-sdr service restarted with average severity. <br />
This trigger automatically clears itself after successful restart of the service and mostly doesn't require any action from the admin.  
+
This trigger automatically clears itself after successful restart of the service and mostly doesn't require any action from the admin. <br />
 
[[File:zabbix-guide-restate-yate-sdr.png |800px]]
 
[[File:zabbix-guide-restate-yate-sdr.png |800px]]
 
  
 
== Monitoring Unmet Demands ==
 
== Monitoring Unmet Demands ==
 +
Unmet demands shows some indications that UEs are failing to connect or to use the network resources.
 +
It meant to give an idea where the problem is happening and how dangerous its affecting the network.
 +
This graph is to give a clue of the problem and cannot be used for troubleshooting.
  
 
From Zabbix frontend, navigate to Monitor -> Graphs -> Select Group/ host -> Select Basestation Unmet Demands graph from the drop down menu.  
 
From Zabbix frontend, navigate to Monitor -> Graphs -> Select Group/ host -> Select Basestation Unmet Demands graph from the drop down menu.  
 
The graph will show:
 
The graph will show:
  
# Unanswered paging
+
# Unanswered paging: Shows the number of paging messages dropped without transmission by the eNodeB
# E-RAP failed attempts
+
# E-RAP failed attempts: Shows the number of failed attempts while establishing Random  Access Procedure.
# low PHRdB: TB.UlPhrAvg
+
# low PHRdB: Low power headroom which indicates the UEs are using too much power.  
# Percentage of the difference between msg3 and prachs
+
# Percentage of the difference between msg3 and prachs: The percentage of uncompleted connection due to not answering RRC Connection Request (msg3)
# Loss rate for uplink and downlink  
+
# Loss rate for uplink and downlink: Lost packets for uplink and downlink  
 
[[File:zabbix-guide-unmet-demands.png|800px]]
 
[[File:zabbix-guide-unmet-demands.png|800px]]
  
Line 129: Line 136:
 
=== Speed ===
 
=== Speed ===
 
Speed report is represented with a graph, showing the bit rate.
 
Speed report is represented with a graph, showing the bit rate.
 +
 
[[File:zabbix-guide-speed.png|800px]]
 
[[File:zabbix-guide-speed.png|800px]]
  
 
=== Throughput ===
 
=== Throughput ===
 
Speed report is represented with a graph, showing uplink and downlink throughput
 
Speed report is represented with a graph, showing uplink and downlink throughput
[[File:zabbix-guide-throughput.png|800px]]
 
  
 +
[[File:zabbix-guide-throughput.png|800px]]
  
 
== Receive emails for triggers  ==  
 
== Receive emails for triggers  ==  

Latest revision as of 15:01, 16 January 2019

Introduction

This guide will explain how to use Zabbix for monitoring Labkits/Satsite

Critical 3 triggers:

  1. When S1AP link is down
  2. When the application restarted
  3. When the Basestation is down

Unmet demands Showing when the network was unable to meet the demands of UEs/MSs with an indication of the cause.

Performance report Showing the speed and the throughput of the network

Minimum Requirements

In order to have the full functionality as per this guide, you need to have:

  1. Yate-enb 7825 or newer
  2. Yate-common 1.7.1 or newer

Note: Yate-common 1.7.1 is not currently part of the public release yet but its released for labkits.

Zabbix server Installation

Zabbix is an enterprise open source monitoring software, is used to monitor and track the status of various network services, servers, and other network hardware.

There are various methods for installing Zabbix server, we are going to use Zabbix appliance on Virtual Box.

Steps:

  1. Install VirtualBox on your server
  2. Download Zabbix Appliance in the ovf extension
  3. From the VirtualBox -> File -> Import appliance -> browse for Zabbix appliance -> import
  4. Make sure you are running the desired network setting: Settings -> Network (For our server we are using Bridged Adapter)
  5. start the Appliance and login with appliance/ zabbix
  6. apply the command (ifconfig) to know Zabbix IP address
  7. From any device with reachability to the Zabbix server, open the browser and enter http://<zabbix-server-ip>/zabbix
  8. You should be able to login with Admin / zabbix


Basestation configurations

We need to configure Zabbix (TM) PHP active agent for Yate based products. It uses the Zabbix version 1 JSON to push requested data to a Zabbix server. Zabbix.php is an active agent and is included in yate-common rpm package.

  1. Login to the Labkits/Satsite via ssh (ssh yatebts@<Basestation-IP> -p 54321) and user the credentials yatebts/<unit serial number>
  2. Open (or create, if it doesn't exist) the configurations file ( /etc/yate/sdr/extmodule.conf) and add the following
    1. [scripts]
    2. zabbix.php=server OR zabbix.php=server:port
    3. for multiple servers: zabbix.php=server:port,server:port,...
  3. systemctl restart yate-sdr.service


Zabbix server configurations

Step1: Add the template to monitor the basestation

All Radio Access Network Templates can be downloaded from Zabbix YateRAN Templates

For more details about what the templates contain and items detail, please refer to Zabbix Monitoring Template Predefined Items

  • Save the template template.xml
  • Open Zabbix frontend http://<zabbix-server-ip>/zabbix and login with (Admin / zabbix )
  • Navigate to configurations -> Template -> Import -> select the file <file>.xml and keep the rules to the default -> Import

zabbix-import-template.png

  • You should get a message with import successfuly and you can find the template when you search for (sdr)

zabbix-imported-template.png
zabbix-search-for-template.png


Step 2: Add Basestation as host

  • From Zabbix frontend, navigate to configuration -> hosts -> create host
  • In the create host page:
  • From Host tab
    • set Hostname to yate-sdr-ybts-<serial number of the labkit/Satsite>
    • Optionally set visible name for how you will see the host
    • Create new group for basestations or use any predefined group
    • Set the Labkit/Satsite IP address
    • Make sure the host is enable


Step 3: Link the host(s) to the template

  • Option 1: Link each host by itself: Configuration -> Hosts -> Select the host -> Template tab -> search for Template YateSDR and link it -> press update.

zabbix-host-link-template.png

  • Option 2: Link for the group of hosts: Configuration -> Templates -> select (Template YateSDR) -> add the group of your hosts in the filed groups -> update.

zabbix-link-template-group.png

Monitoring Triggers

From Zabbix frontend, navigate to Monitor -> problems to see list of the triggers.

S1AP Link Down

  • When the link is detected to be down a trigger called S1AP Disconnected will appear with high severity

zabbix-guide-s1ap-down.png

  • When the link is detected to be recovered , the trigger will be marked as resolved and auto clear the severity

zabbix-guide-s1ap-resolved.png

Basestation Stopped

  • When the Basestation stops, will show a trigger named Basestation Stopped with high severity.

zabbix-guide-basestation-stopped.png

  • when the Basestation is up again, the trigger will be auto resolved and the severity will be cleared

zabbix-guide-basestation-resolved.png

Application Restarted

When Yate-sdr service restarts it will show a trigger named Yate-sdr service restarted with average severity.
This trigger automatically clears itself after successful restart of the service and mostly doesn't require any action from the admin.
zabbix-guide-restate-yate-sdr.png

Monitoring Unmet Demands

Unmet demands shows some indications that UEs are failing to connect or to use the network resources. It meant to give an idea where the problem is happening and how dangerous its affecting the network. This graph is to give a clue of the problem and cannot be used for troubleshooting.

From Zabbix frontend, navigate to Monitor -> Graphs -> Select Group/ host -> Select Basestation Unmet Demands graph from the drop down menu. The graph will show:

  1. Unanswered paging: Shows the number of paging messages dropped without transmission by the eNodeB
  2. E-RAP failed attempts: Shows the number of failed attempts while establishing Random Access Procedure.
  3. low PHRdB: Low power headroom which indicates the UEs are using too much power.
  4. Percentage of the difference between msg3 and prachs: The percentage of uncompleted connection due to not answering RRC Connection Request (msg3)
  5. Loss rate for uplink and downlink: Lost packets for uplink and downlink

zabbix-guide-unmet-demands.png

Note: The UL loss rate item is initially returned as percentage (UL packet loss rate in percent x100) according to 3GPP Section 4.4.4.2. In the template the item is divided by 100 for compatibility with the other items in the graph

Monitoring Performance reports

From Zabbix frontend, navigate to Monitor -> Graphs -> Select Group/ host -> Select Basestation Speed or Basestation Throughput from the drop down menu

Speed

Speed report is represented with a graph, showing the bit rate.

zabbix-guide-speed.png

Throughput

Speed report is represented with a graph, showing uplink and downlink throughput

zabbix-guide-throughput.png

Receive emails for triggers

Zabbix offer sending email notifications when triggers happen.

To configure it:

  1. Configure the email SMTP settings under, from Administration -> media types -> email
  2. Create a user (if not already exist) from Administration -> Users -> User
  3. Link the media to the user from Administration -> Users -> Media
  4. Create Action to link sending email with the trigger , from Configuration -> Actions