BE log support within log-indexer

Purpose and scope

Internal project which purpose is to replace current BE logging facilities based on own components with generic logging system currently used by PortaSIP components.

Benefits:

• Decrease of different components number involved in logging.
• Unification of logging system used for PortaSwitch components.
• Decreasing of required knowledges volume regarding logging and involved components.
• Join logs location.
• Simplification of logging process for BE etc.

This document covers the following questions:

• Requirements, limitations and specifics related to BE logging.
• BE PERL API of logging system.
• Generic sip.log format changes required for BE logs support.
• Configuration files changes required for BE logs support.
• Log-indexer Interface changes to cover log-displaying PortaAdmin needs.

Specifications

Definitions, acronyms and abbreviation

BE - billing engine, is a PERL application that acts as a source of LogMessages and requires an API for pushing them into generic logging system.

LogMessage - one-line or multi-line plain-text message (string) pushed by BE into logging system using appropriate API method. Every LogMessage must be tied to LogLevel and be one of two types: either DebugType or BillingType.

Example 1. One-line LogMessage.

Verification of password using method 'digest_response': success

Example 2. Multi-line LogMessage.

Mod(voice): Call schema: <Voice::Party id="11165400002"/> --O--> <Voice::Party id="11165400001"/>   <Voice::Party id="11165400002"/> --I--> <Voice::Party id="11165400001"/>

 

MessageType - particular king of LogMessage or LogEvent by its nature, either DebugType or BillingType.

DebugType - BE LogMessage (or LogEvent) type that is required for only internal, generic or non-session related component debugging purposes (e.g. modules initialization, assert checks etc.). In general, it does not require indexing and long-time storing.

Example 3. DebugType LogMessage.

Logfile opening, server time: 2016-01-21 05:05:02 GMT +0000 (UTC)

 

BillingType - BE LogMessage (or LogEvent) type that contains session-related billing-sensitive information. Requires all-range logging system functionality.

Example 4. BillingType LogMessage.

Verification of password using method 'digest_response': success

 

LogEvent - LogMessage together with additional information like level, date and time etc. It usually represented in some specific format depending on place where it occurs.

Example 5. LogEvent stored in sip.log file.

2014-02-04T03:48:34.371Z|rtpproxy[549]||IR|65| Global statistics: ext_sessions=0 int_sessions=0 rtp_nservers=0

 

EventsBlock - set of LogEvents that belong to some logical stage of call processing, it is based on LogMessages that are tied using the same BlockId.

BlockId - unique identifier that determines EventsBlock, it ties separately carried LogEvents.

CallSession - set of EventsBlock determined by H323-Conf-Id.

H323-Conf-Id - unique CallSession identifier.

LoggingLibrary - PERL module that provides API to logging system.

LogAppender - log4cplus library LogEvents processor for particular output destination (e.g. local file or network socket with buffering functionality).

LogMaster - central daemon of logging system that processes LogEvents as well as REST API requests from PortaAdmin, communicates with permanent storage to store and retrieve DBMessages, converts data representation etc.

DBMessage - LogEvent and related data represented in format suitable for permanent storage.

SearchQuery - JSON based query string from PortaAdmin to LogMaster using RestAPI.

RelationTypes - type of relations between GroupId and H323-Conf-Id, between two different H323-Conf-Ids.

PortaAdmin - here is a part of PortaSwitch web interface that through WebForms provides user access to BE logs.

WebForms

RestAPI

Basic assumptions

• Each EventsBlock is a separate RADIUS-request/response.
• For one H323-Conf-Id there are few EventsBlocks.
• Each EventsBlock belongs (relates) to only one H323-Conf-ID.
• Each H323-Conf-Id represents exactly one CallSession.
• CallSession may have relations between each other. Type of relations are "next/previous" and "parent/child".
• CallSession relations are represented as their H323-Conf-Id relations.
• H323-Conf-Id relations have relation type (next/previous and parent/child) with n-to-n type.