System Control

The System Control node comprises configuration and administration pages that affects the operation of nJAMS Server. nJAMS Administrators and nJAMS Server Operators can control Message Processing, Jobs, Plugin Management and many more within this category.

Client Communication

This is an overview of the state of client connections.

The Client Communication component communicates with nJAMS Clients over a dedicated JMS connection.

This list represents:

Client Name: the name of the nJAMS Client.

Client Version: the last known version number of the nJAMS Client

SDK Version: last known sdk version number the nJAMS Client is based on

Connection Status: current state of the JMS connection used to communicate with the nJAMS Client. State can be STARTING, STARTED, RESTARTING, STOPPING, STOPPED.

Connection Name: name of the JMS connection nJAMS Server uses to communicate with the nJAMS Client.

Send, Received, Errors, ErrorsHandled: number messages sent, received, and number of errors occurred during uptime of this component

You can explicitly request a project message from an nJAMS Client. Select one or more nJAMS Clients from the list and press REQUEST PROJECT MESSAGE.

Data Provider

A Data Provider is the nJAMS Server component that processes log messages. In addition, a Data Provider maintains a communication channel from nJAMS Server to nJAMS Clients. Once a Data Provider is started, log messages will be parsed and processed. You have to configure at least one Data Provider to enable nJAMS Server to process log messages.

Overview:

In a standard setup nJAMS Server connects to a single JMS provider and receives log messages from a single JMS destination. However, if you want to monitor multiple environments, you may have to connect to multiple JMS servers to receive log messages from al environments. Or you may have to collect log messages from more than one JMS destination.

Depending on the scenario, you may configure one or more Data Providers.

1. In this example there are already 4 Data Providers configured.

2. Press ADD to create an additional Data Provider.

Configuration:

The list is blank in case there are no Data Providers configured. Otherwise select a Data Provider from the list to see its configuration details on the right hand side:

3. Click ADD to create a new Data Provider. Navigate to System Control > Message Processing page and stop the Data Provider. You can then modify it by selecting an entry from the list and clicking EDIT. The following dialog:

The following parameters are mandatory:

Setting	Description
Name	Name of the Data Provider
Type	The type specifies whether the Data Provider is designated to process log messages or Argos metrics.
Thread Count	The number of threads this Data Provider can use to process inbound log messages. The default value is 8. Increase this number, if you experience backlogs on message processing and the nJAMS Server is not CPU bound.
Autostart	This setting determines whether a Data Provider should be autostarted during startup of nJAMS Server. In most cases a Data Provider should start processing messages immediately when nJAMS Server is started. However, you can also choose to start a Data Provider manually (NEVER), or consider the last state (PREVIOUS STATE). Select from combo box: ALWAYS: auto start of Data Provider enabled NEVER: auto start off PREVIOUS STATE: Data Provider will start according to its last state. Assume you started a Data Provider manually and nJAMS Server is stopped unexpectedly, the Data Provider will be started accordingly, when nJAMS Server gets restarted successfully. In case the Data Provider was intentionally stopped by the user, the Data Provider will not start automatically.
Connection	Select an existing connection you created before from the combo box
Destination prefix	This is the prefix of the destination, where log messages or metrics are coming in
Error handler	This setting specifies how to deal with incoming messages that could not be processed by nJAMS Server: Use this connection’s queue/topic: sends the failing message to the specified error destination of this Data Provider. This is the same behavior of previous versions of nJAMS Server. Write error files”: writes the failing message into files. The location of the error files can be configured at “Error handlers” on the Message Processing page. Print to log: writes a log entry and, if applicable, writes the message into nJAMS log file. Discard silently: ignores failing messages. In addition, you will be provided with a list of additional Data Providers, whose connections can be used to send failing messages to.

Click on SAVE to save the configuration. You can now start the Data Provider.

Deployment

Deployment allows nJAMS Administrators to deploy a new version of nJAMS Server or to add various extensions to nJAMS Server. An extension can be nJAMS Plugins or 3rd party libraries that are required to communicate to other external systems.

For example, the nJAMS Replay is an extension to nJAMS Server. A plugin like nJAMS Replay has to be deployed on nJAMS Server first before it can be used. The deployment procedure basically means to upload the plugin distributable and to restart nJAMS Server.

On page Deployments nJAMS Administrators can upload extensions as well as restart or shutdown nJAMS Server.

1. List of deployed extensions. The nJAMS Server itself is always part of this list and cannot be removed.

2. UPLOAD allows you to select and upload an extension for deployment. Example of files are .jar, .war. An uploaded file resides on the WildFly machine and waits for deployment.

   For example, when you upload the library tibcrypt.jar, this file will be uploaded to folder <wildfly_dir>/standalone/njams-work. Current state is ‘not installed’:

   

   Please note: uploading larger WAR files in Internet Explorer may take several minutes.

3. RESTART allows to restart nJAMS Server. If there are uploaded extensions that are not yet deployed, you have to restart nJAMS Server in order to get this extension deployed:

   

   Confirm by clicking YES to restart nJAMS Server. Otherwise click on NO, if you want to performa a restart later.

4. SHUTDOWN stops nJAMS Server and the WildFly Application Server. You have to restart WildFly Application Server manually.

5. This is to RESET nJAMS Server. The work folder will be restored to currently deployed artifacts.

6. You can plan a DOWNTIME of nJAMS Server here. Enabling a Downtime will create a Job that can be scheduled for executing a planned Downtime.

7. FAIL-SAFE off allows you to individually disable components in nJAMS Server. In case of an erroneous extension, which prevents nJAMS Server from working properly, you can disable the failing extension and perform a restart.

   The following options are available:

   

   Disable all custom reports: You can disable all custom reports, if they are malfunctioning and prevent nJAMS Server from working properly

   Disable all plugins on restart: All nJAMS plugins will be disabled. Requires a restart of nJAMS Server.

   Deploy nJAMS Server core artifact only: This option ensures only the nJAMS Server core system is started. Requires a redeployment and restart of nJAMS Server application.

   Reset database on restart: initializes the nJAMS instance and allows to start nJAMS Server from scratch. Caution: database will be reset, all data gets lost. Requires a restart.

Feature Management

The Feature Management allows nJAMS Administrators to unlock and configure extensions to nJAMS Server. For example, nJAMS Replay extends nJAMS Server with regards to recording and replaying any data that is monitored by nJAMS.

Overview:

Feature Management provides a list of features available on nJAMS Server.

The example below shows nJAMS Replay is available:

1. See the list of available Features including the status. Pressing ‘config’ button allows you to edit the Feature’s settings.

Features can be started and stopped:

2. Select a Features and the start/stop button is available. Press STOP to stop the selected Feature.

3. To configure a Feature, click on the adjacent icon. Each Feature has its own configuration page. Refer to the Feature manual for more information.

Activate Feature:

To activate a Feature, an nJAMS Administrator has to enter a corresponding license key. The license key is provided at Integration Matters Download Portal according to your contract agreement.

nJAMS Replay:

For more information about nJAMS Replay please visit nJAMS Replay documentation.

IM Flows:

For more information about IM Flows please visit IM Flows documentation.

nJAMS Hawk Console:

Before you can use nJAMS Hawk Console feature, you have to make sure the following TIBCO libraries are available for nJAMS Server:

Library (jar file):	Located at:
tibrvjms.jar	<TIBCO_HOME>/ems/<version>/lib
askit.jar	<TIBCO_HOME>/hawk/<version>/lib
console.jar	<TIBCO_HOME>/hawk/<version>/lib
console_agent_shared.jar	<TIBCO_HOME>/hawk/<version>/lib
talon.jar	<TIBCO_HOME>/hawk/<version>/lib
util.jar	<TIBCO_HOME>/hawk/<version>/lib
tibrvj.jar	<TIBCO_HOME>/tibrv/<version>/lib
TIBCrypt.jar	<TIBCO_HOME>/tra/<version>/lib

You can upload the required libs at Administration / System control / Deployment.

After restarting nJAMS Server you can configure nJAMS Hawk Console feature accordingly:

Purpose:

The nJAMS Hawk Console Plugin subscribes to TIBCO Hawk events from 1-n Hawk Domains and can be used to monitor such events.

Prerequisites:

The TIBCO Hawk Console Plugin requires that TIBCO Hawk is setup using JMS/EMS as the transport layer. TIBCO RV is NOT supported. An nJAMS Agent must be configured to receive events from the plugin by enabling input plugin njams_subagent.

Usage:

The nJAMS Hawk Console Plugin collects TIBCO Hawk Events, such as a alerts (and their clearance) and Hawk Agent start or stop events. Any recorded event is not directly added to nJAMS Server, but forward to an nJAMS Agent instance, which in turn sends it to the server.

TIBCO AppNode events can be collected, when “appnode.collection.enabled” is set to true.

Note

It is not recommended to use this feature (it is deprecated). Use the nJAMS Agent BWAGENT Input Plugin instead!

TIBCO EMS metrics can be collected, when “ems.collection.enabled” is set true.

Note

It is not recommended to use this feature (it is deprecated). Use the nJAMS Agent EMS Input Plugin instead!

The data produced by this plugin can be used inside the Argos feature of nJAMS Server to build dashboard and create rules.

nJAMS Event Emitter for IM Flows:

This feature is available starting with nJAMS Server release 5.1.8.

nJAMS Event Emitter for Flows NT sends nJAMS events to a designated REST endpoint of a Flows NT instance. Flows NT refers to a new product from Integration Matters, which will replace the existing IM Flows product.

Flows NT is a managed service that provides visibility and measurability of your business operations. You can monitor your business flows in real time and measure the performance against your targets. With Flows NT, you better control and manage your business transactions in terms of various Flow Performance Metrics. More information about Flows NT will be available with the release of Flows NT.

Once enabled nJAMS Event Emitter for Flows NT provides a Flows NT instance with nJAMS events. You can limit the events to specific domain objects. Go to tab “Properties” in the main menu and select a domain object from Tree. The option Flows NT EVENT EMITTER allows to enable sending events of this domain object to Flows NT. By default, this option is turned off for all domain objects.

To enter the “Ingest URL” of the REST endpoint of your Flows NT instance click on the edit button regarding this feature.

Jobs

nJAMS Server offers a job controller to execute jobs for periodic tasks. This includes tasks, which are necessary for proper function of nJAMS Server and tasks which provide optional functionality. Most jobs must be enabled all the time.

1. The list of jobs shows status, name, last and next run date, status, interval and information about the execution time.

On tab ‘Job Executions’ you can see a list of last executions of the jobs:

2. The list contains last start/end time, name, result code / message, scheduler time, and the current execution state of the job.

Working with Jobs:

Select the Job you want to work with:

3. RUN JOB NOW executes the selected Job. This is useful, if you want to run a job straightaway.

   paused: the Job is temporarily paused

   suspended: the Job is suspended permanently, respectively the Job is unscheduled

   schedule: the Job is scheduled:

   

Available Jobs:

The following Jobs are available and can be modified by nJAMS Administrator:

DatabaseRetention:

The DatabaseRetention Job maintains the data of the RDBMS according to the retention settings of Notifications and Job History. Please refer to ‘nJAMS Server User Guide’ for configuring data retention on Domain Objects. This Job is responsible for deleting expired record sets from the database. Typically this Job runs once a day.

DataProviderStatisticsUpdate:

The DataProviderStatisticsUpdate Job is responsible for updating the DP statistics and typically runs every 10 seconds. This Job calculates the statistic data you see on the Message Processing page, including Indexer and Cluster statistics.

IdexRetention:

The IndexRetention Job maintains the data of Elasticsearch according to the retention settings of Domain Objects. Please refer to ‘nJAMS Server User Guide’ for configuring data retention on Domain Objects. This Job is responsible for deleting expired indexes, documents from Elasticsearch. Typically this Job runs once a day.

IndexStatisticsUpdate:

The IndexStatisticsUpdate Job frequently aggregates statistic data for the Reports pages. This Job typically runs every minute. In case this Job is stopped, the Report diagrams are outdated.

IndexWarmer:

The job IndexWarmer is frequently executed and runs standard queries against Elasticsearch. This is to speed up searching.

LDAPSynchronization:

The LDAPSynchronization Job retrieves roles and/or users from a directory service via a LDAP connection. To use this job you have to configure a LDAP connection and a LDAP query for roles or users to be imported. Once the connection has been defined you can schedule this Job. By default this Job is not scheduled.

MetricsMuteCheck:

This job is frequently executed to check whether Metrics collection is muted. Muting is limited in time and this jobs checks whether the limit has been reached. If the time limit is reached, muting will be turned off.

Lifecycle

The Lifecycle page allows nJAMS Administrators to check the state of all internal nJAMS components at a glance. You can also see the dependencies of the components.

Furthermore an nJAMS Administrator can start/stop components on this page. For example, if the ClientCommunication component has to be reset, click on its component state button to stop/start the component.

List of components:

This is the list of nJAMS Server components:

Component	Description
Database	The database component connect to RDBMS via JDBC
JC	Job Controller is the scheduler (Quartz Jobs) for internal Jobs. See chapter ‘Jobs’ above.
FM	FeatureManager is responsible for working with nJAMS features. FM is dependent on the Job Controller.
RES	RulesEngineStatistic component calculates the statistic for the Rule executions
RE	RulesEngine executes Rule instances on matching incoming log messages
RS	Request Store component for Indexer.
I	Indexer component is connected to Elasticsearch and is responsible for Indexing and Searching.
AC	ArgosCache component.
MC	Metrics component processes metrics data required for Argos.
DPC	DataProviderComponent manages DataProvider instances: starting/stopping, check state, etc.

Dependencies of components:

You can see the state and the dependencies of the components on Lifecycle page.

Component is started

Component is stopped

In case all components are started, the state is all green:

In case one of the components is failing, you can identify which dependent component is affected. In the following example, the DataProviderStatistics (DPS) component is stopped which causes the DataProviderController (DPC) component to stop also:

The Database component is the basic component on which all others depend. If the Database component is stopped, all other components will stop as well:

Logging

Select Logging from System Control tree to check nJAMS Server log file(s) and configure log level individually.

View Logs:

1. On tab “View Logs” you can see the current server log file.

2. Select a previous server log file from pull down menu. You can also find the server log files on the file system of the machine, where WildFly Application Server is installed. The particular log files are in <wildfly_directory>/standalone/log/.

3. By clicking on Download you can download the selected server log file.

4. The content has to be manually refreshed by clicking on Update.

Configure logging:

5. Tab Configure Logging allows the nJAMS Administrator to set the log level in detail.

Note

The more granular the log level is, the less the overall performance of the nJAMS Server will be. nJAMS Server will produce a lot of log data, if the log level is set to DEBUG or TRACE. Make sure to use these log levels only when absolutely required and revert them back to INFO or WARNING as soon as possible.

Message Processing

The Message Processing page represents an overview of how nJAMS Server is processing log messages:

1. Overview of all Data Provider instances and their states

2. Bulk Processing Statistics provides statistics of Indexer’s performance.

3. Cluster Nodes Status represents a list of all Elasticsearch Nodes combined to an Elasticsearch cluster.

Data Provider:

The example below shows 4 Data Providers, 3 DPs are started and 1 DP is stopped:

4. Select a Data Provider in order to START/STOP the Data Provider. Furthermore you can edit the Data Provider configuration by clicking on CONFIGURE. Press RETRIGGER ERRORS to resend messages from error queue.

   The DP list provides the following information:

   Column	Description
   State	Running: Data Provider is processing log messages Stopped: Data Provider does not process log messages
   Name	Name of the Data Provider
   Configuration	Type of the Data Provider and its used type of connection
   Threads	Number of threads used by the Data Provider
   Uptime	Uptime since last start of Data Provider
   Msg / sec	Number of log messages being processed per second
   Processed	Number of log messages being processed since last start of Data Provider
   Errors	Number of errors occurred since last start
   Destination Size	Number of current log messages on the event destination that are pending
   Error Destination	Number of log messages in error queue. Log messages will be forwarded to error queue, if
   Size	nJAMS Server cannot parse or process them successfully

Note

When a Data Provider is stopped, the communication channel to nJAMS Clients is still active. You are still able to enable Tracing on nJAMS Client, for example.

Bulk processing:

The indexing status depends on data provider activity. As long as at least one data provider is running, the indexing of new messages is running also. This is not to be mixed up with the indexer connection, which is also responsible for searching and providing Log Messages.

5. Shows the state of Indexing documents.

   Column	Description
   Utilization	Average number of data processing threads feeding data into each indexing thead.
   Wait time	Accumulated time in [ms] that requests for a bulk-processor to become available.
   Error	Total number of errors that occurred on indexing documents since indexing was started.
   Requests	Number of indexing requests per bulk.
   Rejections	Number of rejected indexing requests per bulk.
   Size	Approximated bulk request size.
   Age	Approximated bulk request age in [ms].
   Duration	Overall bulk request processing duration in [ms].
   Throttle time	Summarized time bulk processing was suspended since indexing was started.

Cluster Node Status:

This section represents the condition of the Elasticsearch cluster:

6. Shows the condition of the entire cluster.

   Column	Description
   Node Type	Data Node Client Node Master Node Eligible Master Node
   Node Name	Name of the Elasticsearch Node
   OS Load	Operating System load
   OS Memory Used	Operating System memory usage in percentage
   CPU	CPU utilization in percentage
   JVM Heap Unit	JVM Heap usage in percentage
   File System Used	Storage used on file system of machine where Node is running

Data Processing Statistics:

The Data Processing Statistics provide additional details per domain object. Collecting Data Processing Statistics is disabled by default. In case you need a deeper insight of nJAMS message processing you can turn on the statistics. The Data Processing Statistics can be downloaded for better analysis.

Logmessage Statistics:

The Logmessage Statistics provide detailed information about the log messages nJAMS Server receives per domain object: The Logmessage Statistics can be downloaded for better analysis.

System Config

The System Config page allows nJAMS Administrators to edit all configuration parameters available in nJAMS Server. For a better overview the parameters are categorized; each category has its own SAVE button.

Note

Changing system configuration parameters may cause damage to your nJAMS instance. Please change system configuration only on advise of Integration Matters Support Team or if you are an experienced nJAMS Administrator.

Statistics:

Enabling / disabling log message statistics. Default is on.

Retention:

Default data retention of log messages is 7 days. The default retention for job history and notifications is 30 days.

System:

You can specify the basic setting for the nJAMS instance here.

Jobs:

Basic settings for the job scheduler can be specified here.

User Management:

Enter password retention here.

Processing:

Include SVG when comparing process models specifies how updates of process models of the monitored system (BusinessWorks, Mule, etc.) should be handled in nJAMS Server. This version of nJAMS Server introduces a switch to decide, if updates in the layout of process models should lead to store an additional version of the process model in nJAMS Server.

enabled: process model in nJAMS Server will be updated, when the layout of the process graph has changed.

disabled (default): process model will not be updated in nJAMS Server, when only the layout of the process graph has changed. For any other change in the process model (changing activity names, config, etc.), the process model will be updated, respectively a new version of the process model will be stored.

Max processing time configures a timeout in seconds for processing messages in nJAMS Server, default is 20 secs. If you observe a longer processing of messages from time to time in your nJAMS instance, you can increase this value now. This setting was introduced with version 5.2.2.

Background:

Process models of TIBCO BusinessWorks 5 are modified each time, when a BW engine is re-deployed or even re-started without changes in the process design. This behavior of TIBCO BusinessWorks 5 is due to the system. Changes in the process model cause nJAMS Server to add a new version of the process model. This could lead to a huge number of historicized process models, which in turn can cause to slow down processing of project messages significantly in nJAMS Server.

Other technologies, such as TIBCO BusinessWorks 6 or MuleSoft Runtime Engine, do not generate changes in the process model at restart and for that reason do not lead to that many versions of the process model.

It is recommended to disable this setting, if you predominantly monitor TIBCO BusinessWorks 5 processes. However, enabling this setting has a small catch for all other technologies. Changes in the layout of the process model do not result in a new version. In practical terms, this means that moving activities in the process model to a different location is only a change to the layout and is therefore not considered in nJAMS Server, when this setting is enabled.

If you monitor mainly MuleSoft Runtime Engine or TIBCO BusinessWorks 6, then it is recommended to enable this setting.