HowTo use MoSKito producers

Where to use MoSKito?

MoSKito can be embedded into any application layer (see Chart above).
The goal of using MoSKito is to have a complete picture of "What's going on inside the application?" at this very moment or any time ago, within any supported interval.

NOTE: This documentation describes only how to use log files as the MoSKito Statistics Storage. About another variants, please see MoSKito Central page.

Built-in MoSKito producers

Built-in MoSKito producers are producers that are already implemented and ready to monitor the most essential internal system processes, like memory, threads, etc. They are launched by ProducerRegistryFactory static initializing block.

The distinction of the built-in producers is that they return IStatsProducer.SUBSYSTEM_BUILTIN as the SubSystem value (IStatsProducer.getSubsystem()).

Below is the list of some built-in producers:

Producer

Description

Stats Log Examples

BuiltInMemoryProducer

Built-in memory producer, use Runtime.get... memory methods.
There are 3 supported subtipes of producer: FREE, MAX, TOTAL.

JavaRuntimeFree  CUR: 7822895672 MIN: 0 MAX: 8211983208
JavaRuntimeMax  CUR: 10379526144 MIN: 0 MAX: 10379526144
JavaRuntimeTotal  CUR: 8303738880 MIN: 0 MAX: 8303738880

BuiltInMemoryPoolProducer

Built-in producer for memory pool monitoring.
Each producer monitors one memory pool.
Memory pools are garbage collector dependent
and are determined via jmx on start.
Monitoring for all pools from ManagementFactory.getMemoryPoolMXBeans()

MemoryPool-Eden Space-Heap  INIT: 2863333376 MIN USED: 91624888 USED: 47647
6880 MAX USED: 1069037200 MIN COMMITED: 2147483647 COMMITED: 2290745344 MAX COMMITED: 2290745344 MAX: 0
MemoryPool-Tenured Gen-Heap  INIT: 7158300672 MIN USED: 0 USED: 4366328 MAX USED: 4887952 MIN COMMITED: 2147483647 COMMITED: 5726666752 MAX COMMITED: 5726666752 MAX: 0
MemoryPool-Perm Gen-NonHeap  ...
MemoryPool-Survivor Space-Heap  ...
MemoryPool-Code Cache-NonHeap ...
...

BuiltInMemoryPoolVirtualProducer

Built-in producer for monitoring of virtual memory pools,
which are combination of pools based on memory type,
like: all heap pools, all non-heap pools.

Heap memory  INIT: 10379526144 MIN USED: 91624888 USED: 480843208 MAX USED:
 1073925152 MIN COMMITED: 4581294078 COMMITED: 8303738880 MAX COMMITED: 8303738880 MAX: 0
Non-heap memory  INIT: 136314880 MIN USED: 8222096 USED: 16439040 MAX USED:
 16545736 MIN COMMITED: 24313856 COMMITED: 24313856 MAX COMMITED: 24313856 MAX: 0

BuiltInThreadCountProducer

Built-in producer that counts threads based on jmx provided beans.

Sessions  Current: 20 Min: 10 Max: 24 Started: 302 Daemon: 17

See also Logging for Built-in producers chapter.

Predefined MoSKito producers

There are some predefined (ready to use) producers already implemented:

Producer

Description

Stats Log Examples

CacheProducerWrapper

Stats producer for the ano-prise caches, that implements
net.anotheria.anoprise.cache.Cache<K,V> interface.

userobjects  REQ: 6 HIT: 6 HR: 1.0 WR: 3 GC: 0 RO: 0 FU: 0 EX: 0 FI: 0
useremails  REQ: 2 HIT: 0 HR: 0.0 WR: 2 GC: 0 RO: 0 FU: 0 EX: 0 FI: 0

OnDemandStatsProducer

This producer is used when the different method producing stats aren't known at compile time
(or you don't want to use them).
It is used by the InvocationProxy to add methods dynamically as they being called,
but also by the filters, like
RequestURIFilter, who are dynamically add stats for every new URI.
Used for example in the MoskitoInvokationProxy.

cumulated TR: 200 TT: 27532 CR: 0 MCR: 2 ERR: 86 Last: 0 Min: 0 Max: 27004 Avg: 137.66
getUsersByProperty TR: 2 TT: 27020 CR: 0 MCR: 1 ERR: 0 Last: 27004 Min: 16 Max: 27004 Avg: 13510.0
getUser TR: 124 TT: 215 CR: 0 MCR: 1 ERR: 86 Last: 0 Min: 0 Max: 11 Avg: 1.7338709677419355
getUserByName TR: 6 TT: 0 CR: 0 MCR: 1 ERR: 0 Last: 0 Min: 0 Max: 0 Avg: 0.0
updateUser TR: 24 TT: 126 CR: 0 MCR: 1 ERR: 0 Last: 10 Min: 0 Max: 21 Avg: 5.25
existEmail TR: 34 TT: 69 CR: 0 MCR: 1 ERR: 0 Last: 1 Min: 0 Max: 15 Avg: 2.0294117647058822
createUser TR: 10 TT: 101 CR: 0 MCR: 1 ERR: 0 Last: 5 Min: 5 Max: 22 Avg: 10.1

Custom MoSKito producers

Custom moskito producers should implements net.java.dev.moskito.core.producers.IStatsProducer interface and

use custom, or one of the predefined, net.java.dev.moskito.core.producers.IStats interface implementation to count stats.

See an example of the custom stats producer for some custom queue below:

ActionsQueueStatsProducer

ActionsQueueStatsProducer.java Code

AQS

ActionsQueueStats.java Code

Logging MoSKito statistics

MoSKito comes with embedded support for major Logging frameworks, in particular log4j, java.util.logging and, for lazy people, System.outs. In order to attach a logger to a StatsProducer you have to perform following steps:

1. Create a logger

Moskito comes with two loggers: the IntervalStatsLogger and the DefaultStatsLogger (Interval for default logger is 60 sec). The IntervalStatsLogger can be attached to an Interval, the DefaultStatsLogger attaches himself to the unresetable StatsProducer. Both are connecting themself to the internal IntervalUpdateService and listens to the interval updates. As soon as the IntervalUpdate is fired they read the current state of the Producer they are attached to and send it to the LogOutput. Here's an example of how to configure and attach a log4j logger to any producer:

import org.apache.log4j.Logger;
import net.java.dev.moskito.core.logging.IntervalStatsLogger;
import net.java.dev.moskito.core.logging.DefaultStatsLogger;
import net.java.dev.moskito.core.logging.Log4JOutput;
import net.java.dev.moskito.core.stats.DefaultIntervals;

....
IStatsProducer producer = ...;
// add moskito logger
new DefaultStatsLogger(producer, new Log4JOutput(Logger.getLogger("MoskitoDefault")));
new IntervalStatsLogger(producer, DefaultIntervals.FIVE_MINUTES, new Log4JOutput(Logger.getLogger("Moskito5m")));
new IntervalStatsLogger(producer, DefaultIntervals.FIFTEEN_MINUTES, new Log4JOutput(Logger.getLogger("Moskito15m")));
new IntervalStatsLogger(producer, DefaultIntervals.ONE_HOUR, new Log4JOutput(Logger.getLogger("Moskito1h")));
new IntervalStatsLogger(producer, DefaultIntervals.ONE_DAY, new Log4JOutput(Logger.getLogger("Moskito1d")));
//end moskito logger

2. Configuring Log4J

Attaching a log4j logger to a moskito producer is only half of the work, you still need to tell log4j where to log it, here is an example which creates local files for each interval and logs into them:

2.1 Define Appenders
log4j.xml
  	<!-- MOSKITO APPENDER DECLARATION START -->

	<appender name="MoskitoDefaultFileAppender" class="org.apache.log4j.RollingFileAppender">
    	<param name="File" value="logs/monitoring_msk.log" />
    	<param name="MaxFileSize" value="100MB" />
    	<param name="MaxBackupIndex" value="5" />
    	<param name="Threshold" value="INFO" />
    	<layout class="org.apache.log4j.PatternLayout">
      		<param name="ConversionPattern"	value="%r %d{ISO8601} %m%n"/>
    	</layout>
  	</appender>

	<appender name="Moskito5mFileAppender" class="org.apache.log4j.RollingFileAppender">
    	<param name="File" value="logs/monitoring_msk_5m.log" />
    	<param name="MaxFileSize" value="100MB" />
    	<param name="MaxBackupIndex" value="5" />
    	<param name="Threshold" value="INFO" />
    	<layout class="org.apache.log4j.PatternLayout">
      		<param name="ConversionPattern"	value="%r %d{ISO8601} %m%n"/>
    	</layout>
  	</appender>

	<appender name="Moskito15mFileAppender" class="org.apache.log4j.RollingFileAppender">
    	<param name="File" value="logs/monitoring_msk_15m.log" />
    	<param name="MaxFileSize" value="100MB" />
    	<param name="MaxBackupIndex" value="5" />
    	<param name="Threshold" value="DEBUG" />
    	<layout class="org.apache.log4j.PatternLayout">
      		<param name="ConversionPattern"	value="%r %d{ISO8601} %m%n"/>
    	</layout>
  	</appender>

	<appender name="Moskito1hFileAppender" class="org.apache.log4j.RollingFileAppender">
    	<param name="File" value="logs/monitoring_msk_1h.log" />
    	<param name="MaxFileSize" value="100MB" />
    	<param name="MaxBackupIndex" value="5" />
    	<param name="Threshold" value="INFO" />
    	<layout class="org.apache.log4j.PatternLayout">
      		<param name="ConversionPattern"	value="%r %d{ISO8601} %m%n"/>
    	</layout>
  	</appender>

	<appender name="Moskito1dFileAppender" class="org.apache.log4j.RollingFileAppender">
    	<param name="File" value="logs/monitoring_msk_1d.log" />
    	<param name="MaxFileSize" value="100MB" />
    	<param name="MaxBackupIndex" value="5" />
    	<param name="Threshold" value="INFO" />
    	<layout class="org.apache.log4j.PatternLayout">
      		<param name="ConversionPattern"	value="%r %d{ISO8601} %m%n"/>
    	</layout>
  	</appender>

  	<!-- MOSKITO APPENDLER DECLARATION END -->
2.2 Define Loggers

NOTE: The LogLevel should be INFO.

log4j.xml
  	<!-- MOSKITO LOGGER DECLARATION START -->

	<logger name="MoskitoDefault" additivity="false">
		<level value="INFO"/>
		<appender-ref ref="MoskitoDefaultFileAppender"/>
	</logger>

	<logger name="Moskito5m" additivity="false">
		<level value="INFO"/>
		<appender-ref ref="Moskito5mFileAppender"/>
	</logger>

	<logger name="Moskito15m" additivity="false">
		<level value="INFO"/>
		<appender-ref ref="Moskito15mFileAppender"/>
	</logger>

	<logger name="Moskito1h" additivity="false">
		<level value="INFO"/>
		<appender-ref ref="Moskito1hFileAppender"/>
	</logger>

	<logger name="Moskito1d" additivity="false">
		<level value="INFO"/>
		<appender-ref ref="Moskito1dFileAppender"/>
	</logger>

  	<!-- MOSKITO LOGGER DECLARATION END -->

Logging for Built-in stats producers

To have built-in producers stats logged, the next code should be executed once, while initializing (Plus there should be MoskitoBIXXAppendler appendlers and MoskitoBiXX loggers configured in the log4j.xml):

Configure loggers for Built-in Producers
IProducerRegistryAPI api = new ProducerRegistryAPIFactory().createProducerRegistryAPI();
List<IStatsProducer> stats = api.getAllProducersBySubsystem(IStatsProducer.SUBSYSTEM_BUILTIN);

for (IStatsProducer producer : stats) {
	new DefaultStatsLogger(producer, new Log4JOutput(Logger.getLogger("MoskitoBIDefault")));
	new IntervalStatsLogger(producer, DefaultIntervals.FIVE_MINUTES, new Log4JOutput(Logger.getLogger("MoskitoBI5m")));
	new IntervalStatsLogger(producer, DefaultIntervals.FIFTEEN_MINUTES, new Log4JOutput(Logger.getLogger("MoskitoBI15m")));
	new IntervalStatsLogger(producer, DefaultIntervals.ONE_HOUR, new Log4JOutput(Logger.getLogger("MoskitoBI1h")));
	new IntervalStatsLogger(producer, DefaultIntervals.ONE_DAY, new Log4JOutput(Logger.getLogger("MoskitoBI1d")));
}