JMXTrans and Graphite: getting started

You can achieve very decent monitoring of a Java application - including the JVM itself - with jmxtrans and graphite .

No more horrible hand-crafted tool to extract and chart JMS data.

A TRANS in Italian is a transexual, typically Brazilian - more knows as VIADOS - so hearing JMXTrans always puts me in a good mood because Italians like to make jokes about sex.

OK, let's be serious.
First read the excellent jmxtrans Wiki here.
As suggested by jmstrans site, I listened to the presentation by Coda Hale but frankly it didn't say much new, he speaks well but not very informative

Getting started:
download the tar.gz here

tar xvzf jmxtrans-242.tar.gz
cd jmxtrans-242/rpm
.... mmm when I run build.sh it asks me a release and version?????

I will download a rpm from here (instructions on the site are incomplete... not good...)

rpm -i jmxtrans-20121016.145842.6a28c97fbb-0.noarch.rpm

I verify the installation parameters:
less /etc/sysconfig/jmxtrans

# configuration file for package jmxtrans
export JAR_FILE="/usr/share/jmxtrans/jmxtrans-all.jar"
export LOG_DIR="/var/log/jmxtrans"
export SECONDS_BETWEEN_RUNS=60
export JSON_DIR="/var/lib/jmxtrans"
export HEAP_SIZE=512
export NEW_SIZE=64
export CPU_CORES=2
export NEW_RATIO=8
export LOG_LEVEL=debug

cd /usr/share/jmxtrans
./jmxtrans.sh
Usage: ./jmxtrans.sh {start|stop|restart|status} [filename.json]

I look into an example.json

{
  "servers" : [ {
    "port" : "1099",
    "host" : "w2",
    "queries" : [ {
      "outputWriters" : [ {
        "@class" : "com.googlecode.jmxtrans.model.output.StdOutWriter",
        "settings" : {
        }
      } ],
      "obj" : "java.lang:type=Memory",
      "attr" : [ "HeapMemoryUsage", "NonHeapMemoryUsage" ]
    }, {
      "outputWriters" : [ {
        "@class" : "com.googlecode.jmxtrans.model.output.StdOutWriter",
        "settings" : {
        }
      } ],
      "obj" : "java.lang:name=CMS Old Gen,type=MemoryPool",
      "attr" : [ "Usage" ]
    }, {
      "outputWriters" : [ {
        "@class" : "com.googlecode.jmxtrans.model.output.StdOutWriter",
        "settings" : {
        }
      } ],
      "obj" : "java.lang:name=ConcurrentMarkSweep,type=GarbageCollector",
      "attr" : [ "LastGcInfo" ]
    } ],
    "numQueryThreads" : 2
  } ]
}

(I still need to decide if I hate more JSON or XML... they both hurt the eye...)

I write a sample Java class to monitor:
vi Sample.java

public class Sample {
        public static void main(String[] args) {
            try {
                Thread.sleep(100000000);
            }
            catch (Exception e) {};
        }
}

javac Sample.java
java -Dcom.sun.management.jmxremote.port=1105 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false Sample

cp example.json sample.json
I edit port number and host to match 1105 and my localhost

./jmxtrans.sh start sample.json

this runs in background :
ps -ef | grep jmxtrans
root 2240 1 6 07:47 pts/1 00:00:15 /usr/bin/java -server -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Djmxtrans.log.level=debug -Djmxtrans.log.dir=. -Xms512M -Xmx512M -XX:+UseConcMarkSweepGC -XX:NewRatio=8 -XX:NewSize=64m -XX:MaxNewSize=64m -XX:MaxTenuringThreshold=16 -XX:GCTimeRatio=9 -XX:PermSize=384m -XX:MaxPermSize=384m -XX:+UseTLAB -XX:CMSInitiatingOccupancyFraction=85 -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing -XX:ParallelGCThreads=1 -Dsun.rmi.dgc.server.gcInterval=28800000 -Dsun.rmi.dgc.client.gcInterval=28800000 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=2101 -jar jmxtrans-all.jar -e -f sample.json -s 60

and if I do
tail -f jmxtrans.log
I get a line per minute:
[02 Nov 2013 07:47:22] [main] 0 INFO (com.googlecode.jmxtrans.JmxTransformer:134) - Starting Jmxtrans on : sample.json
[02 Nov 2013 07:47:23] [main] 1125 DEBUG (com.googlecode.jmxtrans.JmxTransformer:354) - Loaded file: /usr/share/jmxtrans/sample.json
[02 Nov 2013 07:47:23] [main] 1154 DEBUG (com.googlecode.jmxtrans.JmxTransformer:429) - Scheduled job: osb-vagrant.acme.com:1105-1383374843215-0158449004 for server: Server [host=osb-vagrant.acme.com, port=1105, url=null, cronExpression=null, numQueryThreads=2]
[02 Nov 2013 07:47:23] [ServerScheduler_Worker-1] 1167 DEBUG (com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job: Server [host=osb-vagrant.acme.com, port=1105, url=null, cronExpression=null, numQueryThreads=2]
[02 Nov 2013 07:47:23] [ServerScheduler_Worker-1] 1690 DEBUG (com.googlecode.jmxtrans.util.JmxUtils:102) - ----- Creating 3 query threads
[02 Nov 2013 07:47:24] [pool-1-thread-1] 1940 DEBUG (com.googlecode.jmxtrans.util.JmxUtils:195) - Executing queryName: java.lang:type=Memory from query: Query [obj=java.lang:type=Memory, resultAlias=null, attr=[HeapMemoryUsage, NonHeapMemoryUsage]]
[02 Nov 2013 07:47:24] [pool-1-thread-1] 2002 DEBUG (com.googlecode.jmxtrans.util.JmxUtils:209) - Finished running outputWriters for query: Query [obj=java.lang:type=Memory, resultAlias=null, attr=[HeapMemoryUsage, NonHeapMemoryUsage]]
[02 Nov 2013 07:47:24] [ServerScheduler_Worker-1] 2006 DEBUG (com.googlecode.jmxtrans.jobs.ServerJob:50) - +++++ Finished server job: Server [host=osb-vagrant.acme.com, port=1105, url=service:jmx:rmi:///jndi/rmi://osb-vagrant.acme.com:1105/jmxrmi, cronExpression=null, numQueryThreads=2]

Next step: install grafite and plot these metrics....