萌生debug OpenTSDB源码的想法是在之前的博文Query the last data point in the set from OpenTSDB中产生并实践的，期间一直没有详细地记录下来操作的步骤，现在得空了重新梳理一下。
官网上本来就有介绍如何在Eclipse中运行OpenTSDB的文章，就是步骤繁琐了些，本文没有按照这种方式来做。
首先是去https://github.com/OpenTSDB/opentsdb/releases上下载某个发行版的源码，我玩的版本是2.3.0的。注意，要选择Source code哦~
不然会碰到/bin/sh: ./build-aux/create-src-dir-overlay.sh: No such file or directory的问题。
解压后，cd到目录里，然后执行sh build.sh pom.xml，大致的输出如下
+ test -f configure
+ ./bootstrap
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I build-aux
......
autoreconf: Leaving directory `.'
+ test -d build
+ mkdir build
+ cd build
+ test -f Makefile
+ ../configure pom.xml
......
+ MAKE=make
++ uname -s
+ '[' Darwin = FreeBSD ']'
+ exec make pom.xml
(cd .. ; ./build-aux/create-src-dir-overlay.sh)
{ \
echo '<!-- Generated by Makefile on '`date`' -->'; \
sed <../pom.xml.in \
-e 's/@ASYNCHBASE_VERSION@/1.7.2/' \
-e 's/@ASYNCBIGTABLE_VERSION@/0.0/' \
-e 's/@ASYNCCASSANDRA_VERSION@/0.0/' \
-e 's/@GUAVA_VERSION@/18.0/' \
-e 's/@GWT_VERSION@/2.6.0/' \
-e 's/@GWT_THEME_VERSION@/1.0.0/' \
-e 's/@HAMCREST_VERSION@/1.3/' \
-e 's/@JACKSON_VERSION@/2.4.3/' \
-e 's/@JAVASSIST_VERSION@/3.18.1-GA/' \
-e 's/@JUNIT_VERSION@/4.11/' \
-e 's/@LOG4J_OVER_SLF4J_VERSION@/1.7.7/' \
-e 's/@LOGBACK_CLASSIC_VERSION@/1.0.13/' \
-e 's/@LOGBACK_CORE_VERSION@/1.0.13/' \
-e 's/@MOCKITO_VERSION@/1.9.5/' \
-e 's/@NETTY_VERSION@/3.9.4.Final/' \
-e 's/@OBJENESIS_VERSION@/1.3/' \
-e 's/@POWERMOCK_MOCKITO_VERSION@/1.5.4/' \
-e 's/@SLF4J_API_VERSION@/1.7.7/' \
-e 's/@SUASYNC_VERSION@/1.4.0/' \
-e 's/@ZOOKEEPER_VERSION@/3.4.6/' \
-e 's/@APACHE_MATH_VERSION@/3.4.1/' \
-e 's/@JEXL_VERSION@/2.1.1/' \
-e 's/@JGRAPHT_VERSION@/0.9.1/' \
-e 's/@spec_title@/OpenTSDB/' \
-e 's/@spec_vendor@/The OpenTSDB Authors/' \
-e 's/@spec_version@/2.3.0/' \
-e 's/@maven_profile_hbase@/true/' \
-e 's/@maven_profile_bigtable@/false/' \
-e 's/@maven_profile_cassandrae@/false/' \
; \
} >pom.xml-t
mv pom.xml-t ../pom.xml
现在有了pom.xml、src-main和src-test，我们把项目导入到IDEA里。
我选择的是hbase
我选择的是net.opentsdb:opentsdb:2.3.0
Build一下project，发现在package tsd.client中有些duplicate classes。
简单瞅一下duplicate class的内容就知道这个package net.opentsdb.tsd.client是需要从sources directory中exclude掉的，因为package name和实际的不符。
然后还有些classes such as BuildData,SyntaxChecker not found，它们是由代码生成的。
我们来看下pom.xml中的片段
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<executions>
<execution>
<id>generate-build-data</id>
<configuration>
<executable>build-aux/gen_build_data.sh</executable>
<!-- optional -->
<arguments>
<argument>target/generated-sources/net/opentsdb/tools/BuildData.java</argument>
<argument>net.opentsdb.tools</argument>
<argument>BuildData</argument>
</arguments>
</configuration>
<phase>generate-sources</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
<execution>
......
</execution>
</executions>
</plugin>
<plugin>
<groupId>com.helger.maven</groupId>
<artifactId>ph-javacc-maven-plugin</artifactId>
<version>2.8.0</version>
<executions>
<execution>
<id>jjc</id>
<phase>generate-sources</phase>
<goals>
<goal>javacc</goal>
</goals>
<configuration>
<jdkVersion>1.6</jdkVersion>
<javadocFriendlyComments>true</javadocFriendlyComments>
<packageName>net.opentsdb.query.expression.parser</packageName>
<sourceDirectory>${basedir}/src/</sourceDirectory>
<outputDirectory>${project.build.directory}/generated-sources/</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
执行mvn compile，大致的输出如下
......
[INFO] --- exec-maven-plugin:1.2.1:exec (generate-build-data) @ opentsdb ---
......
Generating target/generated-sources/net/opentsdb/tools/BuildData.java
fatal: Not a git repository (or any of the parent directories): .git
fatal: Not a git repository (or any of the parent directories): .git
fatal: Not a git repository (or any of the parent directories): .git
[INFO]
[INFO] --- build-helper-maven-plugin:1.7:add-source (add-source) @ opentsdb ---
[INFO] Source directory: /Users/echo/Documents/project/opentsdb-2.3.0/target/generated-sources added.
[INFO]
[INFO] --- ph-javacc-maven-plugin:2.8.0:javacc (jjc) @ opentsdb ---
Java Compiler Compiler Version 6.1_2 (Parser Generator)
(type "javacc" with no arguments for help)
Reading from file /Users/echo/Documents/project/opentsdb-2.3.0/src/parser.jj . . .
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
File "TokenMgrError.java" does not exist. Will create one.
File "ParseException.java" does not exist. Will create one.
File "Token.java" does not exist. Will create one.
File "SimpleCharStream.java" does not exist. Will create one.
Parser generated with 0 errors and 1 warnings.
......
我们需要生成的类都有了
把parser包和BuildData类拷贝到src-main相应的地方，重新build下，现在没有任何error出来了。
直接run下net.opentsdb.tools.TSDMain先看看(试错)效果
10:35:28.466 [main] INFO net.opentsdb.tools.TSDMain - Starting.
10:35:28.475 [main] INFO net.opentsdb.tools.TSDMain - net.opentsdb.tools BuildData built at revision (MODIFIED)
10:35:28.475 [main] INFO net.opentsdb.tools.TSDMain - Built on 2017/08/09 01:45:47 +0000 by echo@Echos-MacBook-Pro.local:/Users/echo/Documents/project/opentsdb-2.3.0
10:35:28.505 [main] DEBUG net.opentsdb.utils.Config - Unable to find or load opentsdb.conf
java.io.FileNotFoundException: opentsdb.conf (No such file or directory)
at java.io.FileInputStream.open(Native Method) ~[na:1.7.0_79]
at java.io.FileInputStream.<init>(FileInputStream.java:146) ~[na:1.7.0_79]
at java.io.FileInputStream.<init>(FileInputStream.java:101) ~[na:1.7.0_79]
at net.opentsdb.utils.Config.loadConfig(Config.java:589) [classes/:na]
at net.opentsdb.utils.Config.<init>(Config.java:132) [classes/:na]
at net.opentsdb.tools.CliOptions.getConfig(CliOptions.java:101) [classes/:na]
at net.opentsdb.tools.TSDMain.main(TSDMain.java:114) [classes/:na]
10:35:28.506 [main] DEBUG net.opentsdb.utils.Config - Unable to find or load /etc/opentsdb.conf
java.io.FileNotFoundException: /etc/opentsdb.conf (No such file or directory)
......
10:35:28.506 [main] DEBUG net.opentsdb.utils.Config - Unable to find or load /etc/opentsdb/opentsdb.conf
java.io.FileNotFoundException: /etc/opentsdb/opentsdb.conf (No such file or directory)
......
10:35:28.507 [main] DEBUG net.opentsdb.utils.Config - Unable to find or load /opt/opentsdb/opentsdb.conf
java.io.FileNotFoundException: /opt/opentsdb/opentsdb.conf (No such file or directory)
......
10:35:28.507 [main] INFO net.opentsdb.utils.Config - No configuration found, will use defaults
Missing static root directory
Usage: tsd --port=PORT --staticroot=PATH --cachedir=PATH
Starts the TSD, the Time Series Daemon
Process finished with exit code 1
啊哦！在四个地方都没找到opentsdb.conf。虽然在src-main/net.opentsdb下有个opentsdb.conf，但程序明显不是从那加载的。
那是从哪里加载的呢？我们去net.opentsdb.utils.Config.loadConfig()里加点代码来测试下吧~
输出结果是/Users/echo/Documents/project/opentsdb-2.3.0/test.txt，哦~ 原来是项目的根目录呀。
那现在可以选择将opentsdb.conf拷贝到项目的根目录下，也可以在TSDMain的运行参数中通过--config=absolute-path-to-opentsdb-conf-file来指定其他的某个位置。
无论你使用哪种方式，我们都需要对opentsdb.conf做下修改
tsd.network.port = 4242
tsd.http.staticroot = /Users/echo/Documents/project/opentsdb-2.3.0/target/opentsdb-2.3.0/queryui
tsd.http.cachedir = /tmp
tsd.core.auto_create_metrics = true
其中，staticroot引用的那个目录是在我们执行mvn compile后生成的，它是OpenTSDB的web query ui所在的地方。
好了，现在解决了配置文件的问题，让我们再次run一下TSDMain。
Exception in thread "main" java.lang.RuntimeException: Initialization failed
at net.opentsdb.tools.TSDMain.main(TSDMain.java:237)
Caused by: java.lang.ExceptionInInitializerError
at net.opentsdb.tsd.RpcManager.initializeBuiltinRpcs(RpcManager.java:290)
at net.opentsdb.tsd.RpcManager.instance(RpcManager.java:148)
at net.opentsdb.tools.TSDMain.main(TSDMain.java:202)
Caused by: java.lang.RuntimeException: Couldn't find mygnuplot.sh on the CLASSPATH: ......
at net.opentsdb.tsd.GraphHandler.findGnuplotHelperScript(GraphHandler.java:913)
at net.opentsdb.tsd.GraphHandler.<clinit>(GraphHandler.java:903)
... 3 more
一个新问题，在classpath中找不到mygnuplot.sh文件。
先来看pom.xml中关于这个文件的配置
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<phase>process-resources</phase>
<configuration>
<target>
<copy file="${basedir}/src/mygnuplot.sh" todir="${basedir}/target/classes"/>
<chmod file="${basedir}/target/classes/mygnuplot.sh" perm="ugo+rx"/>
<copy file="${basedir}/src/mygnuplot.bat" todir="${basedir}/target/classes"/>
<chmod file="${basedir}/target/classes/mygnuplot.bat" perm="ugo+rx"/>
</target>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
如果你有细心观察的话，在我们前面mvn compile后classes下面其实是有mygnuplot.sh、mygnuplot.bat这两个文件的，但是因为我们在IDEA中重新build过项目，所以原来classes下面的东西都没了，类文件也是IDE重新编译出来的。
既然是在classpath下找不到mygnuplot.sh文件，那好办，在项目根目录下创建个src-main-resources文件夹，然后将src-main/net.opentsdb下的mygnuplot.sh copy过来，并在IDE中将src-main-resources mark directory as Resources root，并Rebuild project，现在target/classes下就有这个文件了。
Re-run TSDMain when you have had a single-node standalone HBase started，good luck！
11:34:52.165 [main] INFO net.opentsdb.tools.TSDMain - Starting.
11:34:52.170 [main] INFO net.opentsdb.tools.TSDMain - net.opentsdb.tools BuildData built at revision (MODIFIED)
11:34:52.170 [main] INFO net.opentsdb.tools.TSDMain - Built on 2017/08/09 01:45:47 +0000 by echo@Echos-MacBook-Pro.local:/Users/echo/Documents/project/opentsdb-2.3.0
11:34:52.174 [main] INFO net.opentsdb.utils.Config - Successfully loaded configuration file: /Users/echo/Documents/project/opentsdb-2.3.0/src-main/net/opentsdb/opentsdb.conf
11:34:52.186 [main] DEBUG o.j.n.c.socket.nio.SelectorUtil - Using select timeout of 500
11:34:52.186 [main] DEBUG o.j.n.c.socket.nio.SelectorUtil - Epoll-bug workaround enabled = false
11:34:52.219 [main] INFO org.hbase.async.Config - Successfully loaded configuration file: /Users/echo/Documents/project/opentsdb-2.3.0/src-main/net/opentsdb/opentsdb.conf
11:34:52.316 [main] DEBUG net.opentsdb.core.TSDB - TSD Configuration:
File [/Users/echo/Documents/project/opentsdb-2.3.0/src-main/net/opentsdb/opentsdb.conf]
Key [tsd.storage.compaction.max_concurrent_flushes] Value [10000]
Key [tsd.network.tcp_no_delay] Value [true]
.....
11:34:52.349 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost sessionTimeout=5000 watcher=org.hbase.async.HBaseClient$ZKClient@5462a19d
.....
11:34:52.555 [main] INFO net.opentsdb.tsd.RpcManager - Mode: rw, HTTP UI Enabled: true, HTTP API Enabled: true
11:34:52.564 [main] DEBUG net.opentsdb.tsd.GraphHandler - Using Gnuplot wrapper at /Users/echo/Documents/project/opentsdb-2.3.0/target/classes/mygnuplot.sh
11:34:52.581 [main] INFO net.opentsdb.tsd.RpcHandler - TSD is in rw mode
11:34:52.581 [main] INFO net.opentsdb.tsd.RpcHandler - CORS domain list was empty, CORS will not be enabled
11:34:52.582 [main] INFO net.opentsdb.tsd.RpcHandler - Loaded CORS headers (Authorization, Content-Type, Accept, Origin, User-Agent, DNT, Cache-Control, X-Mx-ReqToken, Keep-Alive, X-Requested-With, If-Modified-Since)
11:34:52.585 [main] INFO net.opentsdb.tsd.ConnectionManager - TSD concurrent connection limit set to: 0
11:34:52.587 [main] WARN net.opentsdb.utils.PluginLoader - Unable to locate any plugins of the type: net.opentsdb.tsd.HttpSerializer
11:34:52.605 [main] INFO net.opentsdb.tools.TSDMain - Ready to serve on /0.0.0.0:4242
我们通过Postman搞点测试数据
然后在web query ui上查询一下
很好很棒！