﻿<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:trackback="http://madskills.com/public/xml/rss/module/trackback/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/"><channel><title>BlogJava-庄周梦蝶-随笔分类-java</title><link>http://www.blogjava.net/killme2008/category/19800.html</link><description>生活、程序、未来</description><language>zh-cn</language><lastBuildDate>Wed, 09 May 2012 17:47:56 GMT</lastBuildDate><pubDate>Wed, 09 May 2012 17:47:56 GMT</pubDate><ttl>60</ttl><item><title>分布式消息中间件Metaq发布1.4.2</title><link>http://www.blogjava.net/killme2008/archive/2012/05/09/377748.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Wed, 09 May 2012 14:47:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2012/05/09/377748.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/377748.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2012/05/09/377748.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/377748.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/377748.html</trackback:ping><description><![CDATA[<br />
&nbsp; &nbsp; <a href="https://github.com/killme2008/Metamorphosis/wiki/For_Developer">我们</a>在维护的淘宝开源消息中间件的<a href="https://github.com/killme2008/Metamorphosis">metaq</a>的github分支，今天发布了1.4.2版本，主要做了如下改进：<br />
<br />&nbsp; &nbsp; 1.添加了大量的使用和原理文档，参见<a href="https://github.com/killme2008/Metamorphosis/wiki">Wiki</a>。<br />&nbsp; &nbsp; 2.合并tools和server-wrapper工程，提供统一的脚本来管理Broker，管理Broker的工作变得非常容易，全部工作都可以通过metaServer.sh的脚本来执行。同时提供了bat启动脚本，用于在windows上启动Broker做测试。<br />&nbsp; &nbsp; 3.新功能：<br />&nbsp; &nbsp;（1）新的客户端API用来获取topic的分区列表<br />&nbsp; &nbsp;（2）新的客户端API用来获取Broker的统计信息<br />&nbsp; &nbsp;（3）异步复制的Slave可以自动获取Master的配置变更，例如Master在配置文件中新增或者删除了topic并顺利reload热加载成功后，slave可自动复制或者移除变更的topic，无需重启。<br />&nbsp; &nbsp;（4）新的统计项目，可以通过'stats config'协议获取Broker的配置文件。<br />&nbsp; &nbsp; 4.添加meta-python项目，一个python的客户端，暂时仅支持发送消息功能。<br />&nbsp; &nbsp; 5.其他小改进，如统计信息的优化、构建工具的整合等。<br />
<br />&nbsp; &nbsp; 更详细的发行日志请看<a href="https://github.com/killme2008/Metamorphosis/wiki/ReleaseNotes">RelaseNotes</a>。<br />
<br />&nbsp; &nbsp; 下载地址： &nbsp;<a href="https://github.com/killme2008/Metamorphosis/downloads">https://github.com/killme2008/Metamorphosis/downloads<br /></a>&nbsp; &nbsp; 入门指南： &nbsp;《<a href="https://github.com/killme2008/Metamorphosis/wiki/%E5%A6%82%E4%BD%95%E5%BC%80%E5%A7%8B">如何开始</a>》<br />&nbsp; &nbsp; 更多文档请看<a href="https://github.com/killme2008/Metamorphosis/wiki">Wiki</a>。<img src ="http://www.blogjava.net/killme2008/aggbug/377748.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2012-05-09 22:47 <a href="http://www.blogjava.net/killme2008/archive/2012/05/09/377748.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Java程序员常用工具集</title><link>http://www.blogjava.net/killme2008/archive/2012/04/17/374936.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Tue, 17 Apr 2012 09:05:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2012/04/17/374936.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/374936.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2012/04/17/374936.html#Feedback</comments><slash:comments>13</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/374936.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/374936.html</trackback:ping><description><![CDATA[<br />&nbsp; &nbsp; 我发现很多人没办法高效地解决问题的关键原因是不熟悉工具，不熟悉工具也还罢了，甚至还不知道怎么去找工具，这个问题就大条了。我想列下我能想到的一个Java程序员会用到的常用工具。<br />
<br />
一、编码工具<br />
<br />
1.IDE：<a href="http://www.eclipse.org/">Eclipse</a>或者<a href="http://www.jetbrains.com/idea/">IDEA</a>，熟悉尽可能多的快捷键，《<a href="https://www.google.com/#hl=zh-CN&amp;site=&amp;source=hp&amp;q=Eclipse+%E5%B8%B8%E7%94%A8%E5%BF%AB%E6%8D%B7%E9%94%AE&amp;oq=Eclipse+%E5%B8%B8%E7%94%A8%E5%BF%AB%E6%8D%B7%E9%94%AE&amp;aq=f&amp;aqi=g-l1&amp;aql=&amp;gs_l=hp.3..0i13.449l7739l0l7948l53l49l9l4l4l9l278l4511l14j15j7l36l0.&amp;bav=on.2,or.r_gc.r_pw.r_cp.,cf.osb&amp;fp=eef1a08d3fa904e4&amp;biw=1594&amp;bih=858">Eclipse常见快捷键列表</a>》<br />
2.插件：&nbsp;<br />
(1) <a href="http://findbugs.sourceforge.net/">Findbugs</a>，在release之前进行一次静态代码检查是必须的<br />
(2) <a href="http://www.atlassian.com/software/clover/overview">Clover</a>，关心你的单元测试覆盖率<br />
(3)&nbsp;<a href="http://checkstyle.sourceforge.net/">Checkstyle</a>&nbsp;代码风格检查<br />
<br />
3.构建和部署工具:<a href="http://ant.apache.org/">ant</a>或者<a href="http://maven.apache.org/">maven</a>，现在主流都是maven了吧，<a href="http://www.sonatype.org/nexus/">使用nexus搭建maven私服</a>，再加上持续集成<a href="http://jenkins-ci.org/">jenkins</a>。代码质量不用愁。<br />
<br />
4.版本管理工具： <a href="http://subversion.tigris.org/">svn</a>或者<a href="http://git-scm.com/">git</a><br />
<br />
5.<a href="http://www.linuxsky.org/doc/admin/200712/213.html">diff和patch</a><br />
<br />
6.设置你的eclipse或者IDEA，如formatter,<a href="http://mestreota.blogspot.jp/2007/11/save-action-on-eclipse.html">save actions</a>以及code template等。代码风格，直接用google的也可以啊。《<a href="http://code.google.com/p/google-styleguide/">Google style guide</a>》<br />
<br />
7.掌握一个文本编辑器，Emacs或者VIM，熟悉常用快捷键。这在你需要在线编辑代码，或者编写其他语言代码时候特别有用。《<a href="http://linuxtoy.org/archives/why-emacs-vim-good.html">神器圣战</a>》<br />
<br />
二、JDK相关<br />
<br />
1.jstat : 观察GC情况，如：<br />
<br />
<div style="font-size: 13px; border-top-width: 1px; border-right-width: 1px; border-bottom-width: 1px; border-left-width: 1px; border-top-style: solid; border-right-style: solid; border-bottom-style: solid; border-left-style: solid; border-top-color: #cccccc; border-right-color: #cccccc; border-bottom-color: #cccccc; border-left-color: #cccccc; border-image: initial; padding-right: 5px; padding-bottom: 4px; padding-left: 4px; padding-top: 4px; width: 98%; word-break: break-all; background-color: #eeeeee; "><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
-->jstat&nbsp;-gcutil&nbsp;pid&nbsp;2000</div>
<br />
2.jmap，查看heap情况，如查看存活对象列表：<br />
<div style="font-size: 13px; border-top-width: 1px; border-right-width: 1px; border-bottom-width: 1px; border-left-width: 1px; border-top-style: solid; border-right-style: solid; border-bottom-style: solid; border-left-style: solid; border-top-color: #cccccc; border-right-color: #cccccc; border-bottom-color: #cccccc; border-left-color: #cccccc; border-image: initial; padding-right: 5px; padding-bottom: 4px; padding-left: 4px; padding-top: 4px; width: 98%; word-break: break-all; background-color: #eeeeee; "><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
-->jmap&nbsp;-histo:live&nbsp;pid&nbsp;|grep&nbsp;com.company&nbsp;|less&nbsp;</div>
<br />
或者dump内存用来分析：<br />
<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
-->jmap&nbsp;-dump:file=test.bin&nbsp;pid</div>
<br />
3.分析dump的堆文件，可以用jhat:<br />
<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
-->jhat&nbsp;test.bin</div>
<br />
&nbsp; 分析完成后可以用浏览器查看堆的情况。这个工具的分析结果还比较原始，你还可以用<a href="http://www.eclipse.org/mat/">Eclipse MAT</a>插件进行图形化分析，或者IBM的<a href="https://www.ibm.com/developerworks/community/groups/service/html/communityview?communityUuid=4544bafe-c7a2-455f-9d43-eb866ea60091">Heap Analyzer</a>.<br />
<br />
4.jvisualvm和jconsole： JVM自带的性能分析和监控工具，怎么用？<a href="http://docs.oracle.com/javase/6/docs/technotes/guides/visualvm/index.html">请自己看文档。</a><br />
<br />
5.jstack：分析线程堆栈，如<br />
<br />
<div style="font-size: 13px; border-top-width: 1px; border-right-width: 1px; border-bottom-width: 1px; border-left-width: 1px; border-top-style: solid; border-right-style: solid; border-bottom-style: solid; border-left-style: solid; border-top-color: #cccccc; border-right-color: #cccccc; border-bottom-color: #cccccc; border-left-color: #cccccc; border-image: initial; padding-right: 5px; padding-bottom: 4px; padding-left: 4px; padding-top: 4px; width: 98%; word-break: break-all; background-color: #eeeeee; "><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
-->jstack&nbsp;pid&nbsp;&gt;&nbsp;thread_dump</div>
<br />
&nbsp; &nbsp; 查看CPU最高的线程在干什么的方法结合top和jstack：<a href="http://www.iteye.com/topic/1114219">http://www.iteye.com/topic/1114219</a><br />
<br />
6.更多JVM工具，参见官方文档：<a href="http://docs.oracle.com/javase/6/docs/technotes/tools/">http://docs.oracle.com/javase/6/docs/technotes/tools/<br />
<br />
</a>7.学习使用btrace分析java运行时问题。《<a href="http://rdc.taobao.com/team/jm/archives/509">Btrace使用简介</a>》<br />
<br />
8.GC日志分析工具：<a href="http://www.tagtraum.com/gcviewer.html">GC viewer</a>、<a href="http://code.google.com/p/gc-console/">GC-console</a>或者<a href="http://stackoverflow.com/questions/1839599/analyze-gc-logs-for-sun-hotspots-jvm-6">自己挑吧。</a><br />
<br />
9.性能分析工具，除了自带的jvisualvm外，还可以用商业的<a href="http://www.ej-technologies.com/products/jprofiler/overview.html">jprofiler</a>。<br />
<br />
<a href="http://kenwublog.com/docs/java6-jvm-options-chinese-edition.htm">10.JVM参数大全<br />
<br />
</a>11.《<a href="http://hllvm.group.iteye.com/group/topic/27945">JVM调优标准参数陷阱</a>》，iteye神贴。<br />
<br />
三、Linux工具<br />
<br />
1.<a href="http://www.commandlinefu.com/commands/browse/sort-by-votes">熟悉常用的shell命令，</a><br />
<div style="display: inline-block; "><br />
2.<a href="http://paulkeck.com/ssh/">设置ssh免登陆<br />
</a></div>
<br />
<br />
3.使用<a href="http://htop.sourceforge.net/">htop</a>替换top。<br />
<br />
4.熟悉下<a href="http://www.ibm.com/developerworks/aix/library/au-unix-strace.html">strace,gdb</a>甚至<a href="http://sourceware.org/systemtap/" title="systemtap"></a><a href="http://sourceware.org/systemtap/"></a><a href="http://sourceware.org/systemtap/" title="systemtap">systemta</a>p来分析问题。<br />
<br />
<a href="http://techgurulive.com/2009/01/29/performance-tuning-tools-ps-top-sar-iostat-and-vmstat/">5.熟悉vmstat,iostat,sar等性能统计工具。</a><br />
<br />
5.自动化部署脚本，<a href="http://docs.fabfile.org/en/1.4.1/index.html">py-fabric</a>或者自荐下我的<a href="https://github.com/killme2008/clojure-control">clojure-control</a>。<br />
<br />
四、其他<br />
<br />
1.掌握一门脚本语言，<a href="http://python.org">Python</a>或者<a href="http://www.ruby-lang.org">Ruby</a>，高效解决一些需要quick and dirty的任务：比如读写文件、导入导出数据库、网页爬虫等。注意不是python.com，咔咔。<br />
<br />
2.使用Linux或者Mac os系统作为你的开发环境。<br />
<br />
3.升级你的&#8220;硬件工具&#8221;，双屏大屏显示器、SSD、8G内存甚至更多。<br /><br />4.你懂的：<a href="https://code.google.com/p/goagent/">https://code.google.com/p/goagent/</a><br />
<br />
五、如何查找工具？<br />
<br />
1.搜索引擎，google或者baidu，《<a href="http://www.williamlong.info/archives/728.html">搜索技巧</a>》<br />
<br />
2.万能的stack overflow：<a href="http://stackoverflow.com/">http://stackoverflow.com/<br />
</a>
<br />
3.虚心问牛人。<br /><br />六、最重要的是&#8943;&#8943;<br /><br />一颗永不停止学习的心。<img src ="http://www.blogjava.net/killme2008/aggbug/374936.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2012-04-17 17:05 <a href="http://www.blogjava.net/killme2008/archive/2012/04/17/374936.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>淘宝开源metaq的python客户端</title><link>http://www.blogjava.net/killme2008/archive/2012/03/21/372405.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Wed, 21 Mar 2012 11:08:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2012/03/21/372405.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/372405.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2012/03/21/372405.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/372405.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/372405.html</trackback:ping><description><![CDATA[&nbsp; &nbsp; 前面一篇博客介绍了我在github上的一个<a href="https://github.com/killme2008/Metamorphosis">metaq</a>分支，今天下午写了个metaq的python客户端，目前仅支持发送消息功能，不过麻雀虽小，五脏俱全，客户端和zookeeper的交互和连接管理之类都还具备，不出意外，我们会首先用上。第一次正儿八经地写python代码，写的不好的地方请尽管拍砖，多谢。<br />&nbsp; &nbsp; 项目叫meta-python，仍然放在github上：<a href="https://github.com/killme2008/meta-python">https://github.com/killme2008/meta-python<br /><br /></a>&nbsp; &nbsp; 使用需要先安装zkpython这个库，具体安装<a href="http://www.zlovezl.cn/articles/40/" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; border-image: initial; font: inherit; color: #4183c4; text-decoration: none; font-family: Helvetica, arial, freesans, clean, sans-serif; line-height: 22px; background-color: #ffffff; ">这篇博客</a>，使用很简单，发送消息：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />-->&nbsp; &nbsp;&nbsp;<span style="color: #0000FF; ">from</span>&nbsp;metamorphosis&nbsp;<span style="color: #0000FF; ">import</span>&nbsp;Message,MessageProducer,SendResult<br />&nbsp;&nbsp;&nbsp;&nbsp;p=MessageProducer(<span style="color: #800000; ">"</span><span style="color: #800000; ">topic</span><span style="color: #800000; ">"</span>)<br />&nbsp;&nbsp;&nbsp;&nbsp;message=Message(<span style="color: #800000; ">"</span><span style="color: #800000; ">topic</span><span style="color: #800000; ">"</span>,<span style="color: #800000; ">"</span><span style="color: #800000; ">message&nbsp;body</span><span style="color: #800000; ">"</span>)<br />&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">print</span>&nbsp;p.send(message)<br />&nbsp;&nbsp;&nbsp;&nbsp;p.close()</div><br />&nbsp; &nbsp;&nbsp;<div style="display: inline-block; ">MessageProducer就是消息发送者，它的构造函数接受至少一个topic，默认的zk_servers为localhost:2181，可以通过zk_servers参数指定你的zookeeper集群:<br /><br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />-->p=MessageProducer(<span style="color: #800000; ">"</span><span style="color: #800000; ">topic</span><span style="color: #800000; ">"</span>,zk_servers=<span style="color: #800000; ">"</span><span style="color: #800000; ">192.168.1.100:2191,192.168.1.101:2181</span><span style="color: #800000; ">"</span>)</div><span style="background-color: #eeeeee; "><br /></span><span style="background-color: #eeeeee; ">更多参数请直接看源码吧。一个本机的性能测试（meta和客户端都跑在我的机器上，机器是Mac MC700，osx 10.7，磁盘没有升级过）：<br /></span><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #0000FF; ">from</span>&nbsp;metamorphosis&nbsp;<span style="color: #0000FF; ">import</span>&nbsp;Message,MessageProducer<br /><span style="color: #0000FF; ">from</span>&nbsp;time&nbsp;<span style="color: #0000FF; ">import</span>&nbsp;time<br />p=MessageProducer(<span style="color: #800000; ">"</span><span style="color: #800000; ">avos-fetch-tasks</span><span style="color: #800000; ">"</span>)<br />message=Message(<span style="color: #800000; ">"</span><span style="color: #800000; ">avos-fetch-tasks</span><span style="color: #800000; ">"</span>,<span style="color: #800000; ">"</span><span style="color: #800000; ">http://www.taobao.com</span><span style="color: #800000; ">"</span>)<br />start=time()<br /><span style="color: #0000FF; ">for</span>&nbsp;i&nbsp;<span style="color: #0000FF; ">in</span>&nbsp;range(0,10000):<br />&nbsp;&nbsp;&nbsp;&nbsp;sent=p.send(message)<br />&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">if</span>&nbsp;<span style="color: #0000FF; ">not</span>&nbsp;sent.success:<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">print</span>&nbsp;<span style="color: #800000; ">"</span><span style="color: #800000; ">send&nbsp;failed</span><span style="color: #800000; ">"</span><br />finish=time()<br />secs=finish-start<br /><span style="color: #0000FF; ">print</span>&nbsp;<span style="color: #800000; ">"</span><span style="color: #800000; ">duration:%s&nbsp;seconds</span><span style="color: #800000; ">"</span>&nbsp;%&nbsp;(secs)<br /><span style="color: #0000FF; ">print</span>&nbsp;<span style="color: #800000; ">"</span><span style="color: #800000; ">tps:%s&nbsp;msgs/second</span><span style="color: #800000; ">"</span>&nbsp;%&nbsp;(10000/secs)<br />p.close()</div><span style="background-color: #eeeeee; "><br />&nbsp;结果：<br /><br /><br /></span><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />-->duration:1.85962295532&nbsp;seconds<br />tps:5377.43415749&nbsp;msgs/second</div></div><img src ="http://www.blogjava.net/killme2008/aggbug/372405.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2012-03-21 19:08 <a href="http://www.blogjava.net/killme2008/archive/2012/03/21/372405.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>xmemcached发布1.3.6</title><link>http://www.blogjava.net/killme2008/archive/2012/03/19/372179.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Mon, 19 Mar 2012 02:51:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2012/03/19/372179.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/372179.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2012/03/19/372179.html#Feedback</comments><slash:comments>2</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/372179.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/372179.html</trackback:ping><description><![CDATA[<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">&nbsp; &nbsp; 开源的memcached Java客户端&#8212;&#8212;<a href="http://code.google.com/p/xmemcached/" target="_blank" style="color: #006699; ">xmemcached</a>发布1.3.6版本。</p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; "><strong style="font-weight: bold; ">&nbsp; &nbsp; 主要改进如下：</strong>&nbsp;</p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">1. &nbsp;为MemcachedClientBuilder添加两个新方法用于配置：<br />
</p><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #0000FF; ">public</span>&nbsp;<span style="color: #0000FF; ">void</span>&nbsp;setConnectTimeout(<span style="color: #0000FF; ">long</span>&nbsp;connectTimeout);&nbsp;&nbsp;<br />
<span style="color: #0000FF; ">public</span>&nbsp;<span style="color: #0000FF; ">void</span>&nbsp;setSanitizeKeys(<span style="color: #0000FF; ">boolean</span>&nbsp;sanitizeKeys);</div>
<p>&nbsp;</p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">2. &nbsp;用于hibernate的XmemcachedClientFactoryd添加了connectTimeout属性，感谢网友<strong style="font-weight: bold; font-size: 13px; ">&nbsp;</strong><span style="font-size: 13px; ">Boli.Jiang的贡献。</span></p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">3. &nbsp;添加新的枚举类型&nbsp;net.rubyeye.xmemcached.transcoders.CompressionMode，用于指定Transcoder的压缩类型，默认是ZIP压缩，可选择GZIP压缩。Transcoder接口添加setCompressionMode方法。</p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">4. &nbsp;修改心跳规则，原来是在连接空闲的时候发起心跳，现在变成固定每隔5秒发起一次心跳检测连接。</p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">5. &nbsp;修改默认参数，默认禁用nagle算法，默认将批量get的合并因子下降到50。</p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">6. &nbsp;修复bug和改进，包括：<a href="http://code.google.com/p/xmemcached/issues/detail?id=161" target="_blank" style="color: #006699; ">161</a>、<a href="http://code.google.com/p/xmemcached/issues/detail?id=163" target="_blank" style="color: #006699; ">163</a>、<a href="http://code.google.com/p/xmemcached/issues/detail?id=165" target="_blank" style="color: #006699; ">165</a>、<a href="http://code.google.com/p/xmemcached/issues/detail?id=169" target="_blank" style="color: #006699; ">169</a>、<a href="http://code.google.com/p/xmemcached/issues/detail?id=172" target="_blank" style="color: #006699; ">172</a><a href="http://code.google.com/p/xmemcached/issues/detail?id=173" target="_blank" style="color: #006699; ">、173</a>、<a href="http://code.google.com/p/xmemcached/issues/detail?id=176" target="_blank" style="color: #006699; ">176</a>、<a href="http://code.google.com/p/xmemcached/issues/detail?id=179" target="_blank" style="color: #006699; ">179</a>和<a href="http://code.google.com/p/xmemcached/issues/detail?id=180" target="_blank" style="color: #006699; ">180</a>。</p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">&nbsp;</p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">项目主页：<a href="http://code.google.com/p/xmemcached/" target="_blank" style="color: #006699; ">http://code.google.com/p/xmemcached/</a></p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">项目文档：<a href="http://code.google.com/p/xmemcached/w/list" target="_blank" style="color: #006699; ">http://code.google.com/p/xmemcached/w/list</a></p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">下载：<a href="http://code.google.com/p/xmemcached/downloads/list" target="_blank" style="color: #006699; ">http://code.google.com/p/xmemcached/downloads/list</a></p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">源码：<a href="https://github.com/killme2008/xmemcached" target="_blank" style="color: #006699; ">https://github.com/killme2008/xmemcached</a></p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">&nbsp;</p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; "><strong style="font-weight: bold; ">Maven依赖：</strong></p>
<p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-family: Helvetica, Tahoma, Arial, sans-serif; line-height: 25px; text-align: left; background-color: #f7f7f7; ">&nbsp;<span style="background-color: #eeeeee; font-size: 13px; ">&lt;</span><span style="background-color: #eeeeee; font-size: 13px; ">dependency</span><span style="background-color: #eeeeee; font-size: 13px; ">&gt;</span><span style="background-color: #eeeeee; font-size: 13px; ">&nbsp;&nbsp;</span></p>
<div style="font-size: 13px; border-top-width: 1px; border-right-width: 1px; border-bottom-width: 1px; border-left-width: 1px; border-top-style: solid; border-right-style: solid; border-bottom-style: solid; border-left-style: solid; border-top-color: #cccccc; border-right-color: #cccccc; border-bottom-color: #cccccc; border-left-color: #cccccc; border-image: initial; padding-right: 5px; padding-bottom: 4px; padding-left: 4px; padding-top: 4px; width: 98%; word-break: break-all; background-color: #eeeeee; ">&nbsp;&nbsp;&nbsp;&nbsp;&lt;groupId&gt;com.googlecode.xmemcached&lt;/groupId&gt;&nbsp;&nbsp;<br />
&nbsp;&nbsp;&nbsp;&nbsp;&lt;artifactId&gt;xmemcached&lt;/artifactId&gt;&nbsp;&nbsp;<br />
&nbsp;&nbsp;&nbsp;&nbsp;&lt;version&gt;1.3.6&lt;/version&gt;&nbsp;&nbsp;<br />
&lt;/dependency&gt;&nbsp;</div>
<br />&nbsp; &nbsp; 最后感谢所有提出issue和改进意见的朋友们。<img src ="http://www.blogjava.net/killme2008/aggbug/372179.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2012-03-19 10:51 <a href="http://www.blogjava.net/killme2008/archive/2012/03/19/372179.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>淘宝开源MQ——metamorphosis的github分支</title><link>http://www.blogjava.net/killme2008/archive/2012/03/16/372019.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Fri, 16 Mar 2012 02:39:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2012/03/16/372019.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/372019.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2012/03/16/372019.html#Feedback</comments><slash:comments>3</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/372019.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/372019.html</trackback:ping><description><![CDATA[<br />
&nbsp; &nbsp; 上周我在淘宝的同事开源了一个消息中间件<a href="http://metaq.taobao.org/">metamorphosis</a>，放在了<a href="http://metaq.taobao.org/">淘蝌蚪</a>上。我从淘蝌蚪的svn上fork了一个github的分支，放在了这里：<br />
<div>&nbsp;1.主体工程：<a href="https://github.com/killme2008/Metamorphosis">https://github.com/killme2008/Metamorphosis<br />
</a>
<div style="display: inline !important; ">&nbsp;2.示例项目：<a href="https://github.com/killme2008/metamorphosis-example">https://github.com/killme2008/metamorphosis-example<br />
</a></div>
<div>&nbsp;3.Twitter storm的spout项目：<a href="https://github.com/killme2008/storm-metamorphosis-spout">https://github.com/killme2008/storm-metamorphosis-spout<br />
</a></div>
&nbsp;&nbsp; &nbsp; <br />
&nbsp; &nbsp; 主要做了一些pom文件的简化，发布1.4.0.2版本到maven central仓库，并且写了几个简单的入门文档，提供了一个完整打包可运行的下载，有兴趣的自己看github页面吧。 Wiki文档放在：<br />
&nbsp; &nbsp;&nbsp;<a href="https://github.com/killme2008/Metamorphosis/wiki">https://github.com/killme2008/Metamorphosis/wiki</a><br />
&nbsp; &nbsp;<br />
&nbsp; &nbsp; &nbsp;客户端Maven依赖包括，可自行选择添加：<br />
<div style="font-size: 13px; border-top-width: 1px; border-right-width: 1px; border-bottom-width: 1px; border-left-width: 1px; border-top-style: solid; border-right-style: solid; border-bottom-style: solid; border-left-style: solid; border-top-color: #cccccc; border-right-color: #cccccc; border-bottom-color: #cccccc; border-left-color: #cccccc; border-image: initial; padding-right: 5px; padding-bottom: 4px; padding-left: 4px; padding-top: 4px; width: 98%; word-break: break-all; background-color: #eeeeee; "><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span><br />
&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span>com.taobao.metamorphosis<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span><br />
&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span>metamorphosis-client<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span><br />
&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span>1.4.0.2<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span><br />
<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span><br />
<br />
<span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span><br />
&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span>com.taobao.metamorphosis<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span><br />
&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span>metamorphosis-client-extension<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span><br />
&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span>1.4.0.2<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span><br />
<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span><br />
<br />
<span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span><br />
&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span>com.taobao.metamorphosis<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span><br />
&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span>storm-metamorphosis-spout<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span><br />
&nbsp;&nbsp;&nbsp;&nbsp;<span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span>1.0.0<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span><br />
<span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">dependency</span><span style="color: #0000ff; ">&gt;</span></div>
</div>
<br />&nbsp; &nbsp; &nbsp;ps.我开通了新浪微博，有兴趣相互关注下：<a href="http://weibo.com/fnil">http://weibo.com/fnil</a>，你看，偏见是可以改变的。<img src ="http://www.blogjava.net/killme2008/aggbug/372019.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2012-03-16 10:39 <a href="http://www.blogjava.net/killme2008/archive/2012/03/16/372019.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>storm常见问题解答</title><link>http://www.blogjava.net/killme2008/archive/2011/12/19/366763.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Mon, 19 Dec 2011 07:25:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/12/19/366763.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/366763.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/12/19/366763.html#Feedback</comments><slash:comments>5</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/366763.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/366763.html</trackback:ping><description><![CDATA[&nbsp;&nbsp;&nbsp; 最近有朋友给我邮件问一些storm的问题，集中解答在这里。<br /><strong>一、我有一个数据文件，或者我有一个系统里面有数据，怎么导入storm做计算？</strong><br /><br />你需要实现一个Spout，Spout负责将数据emit到storm系统里，交给bolts计算。怎么实现spout可以参考官方的kestrel spout实现：<br /><a href="https://github.com/nathanmarz/storm-kestrel">https://github.com/nathanmarz/storm-kestrel</a><br /><br />如果你的数据源不支持事务性消费，那么就无法得到storm提供的可靠处理的保证，也没必要实现ISpout接口中的ack和fail方法。<br /><br /><strong>二、Storm为了保证tuple的可靠处理，需要保存tuple信息，这会不会导致内存OOM？</strong><br /><br />Storm为了保证tuple的可靠处理，acker会保存该节点创建的tuple id的xor值，这称为ack value，那么每ack一次，就将tuple id和ack value做异或(xor)。当所有产生的tuple都被ack的时候， ack value一定为0。这是个很简单的策略，对于每一个tuple也只要占用约20个字节的内存。对于100万tuple，也才20M左右。关于可靠处理看这个：<br /><a href="https://github.com/nathanmarz/storm/wiki/Guaranteeing-message-processing">https://github.com/nathanmarz/storm/wiki/Guaranteeing-message-processing</a><br /><br /><strong>三、Storm计算后的结果保存在哪里？可以保存在外部存储吗？</strong><br /><br />Storm不处理计算结果的保存，这是应用代码需要负责的事情，如果数据不大，你可以简单地保存在内存里，也可以每次都更新数据库，也可以采用NoSQL存储。storm并没有像s4那样提供一个Persist API，根据时间或者容量来做存储输出。这部分事情完全交给用户。<br /><br />数据存储之后的展现，也是你需要自己处理的，storm UI只提供对topology的监控和统计。<br /><br /><strong>四、Storm怎么处理重复的tuple？</strong><br /><br />因为Storm要保证tuple的可靠处理，当tuple处理失败或者超时的时候，spout会fail并重新发送该tuple，那么就会有tuple重复计算的问题。这个问题是很难解决的，storm也没有提供机制帮助你解决。一些可行的策略：<br />（1）不处理，这也算是种策略。因为实时计算通常并不要求很高的精确度，后续的批处理计算会更正实时计算的误差。<br />（2）使用第三方集中存储来过滤，比如利用mysql,memcached或者redis根据逻辑主键来去重。<br />（3）使用bloom filter做过滤，简单高效。<br /><br /><strong>五、Storm的动态增删节点</strong><br /><br />我在storm和s4里比较里谈到的动态增删节点，是指storm可以动态地添加和减少supervisor节点。对于减少节点来说，被移除的supervisor上的worker会被nimbus重新负载均衡到其他supervisor节点上。在storm 0.6.1以前的版本，增加supervisor节点不会影响现有的topology，也就是现有的topology不会重新负载均衡到新的节点上，在扩展集群的时候很不方便，需要重新提交topology。因此我在storm的邮件列表里提了这个问题，storm的开发者nathanmarz创建了一个issue 54并在0.6.1提供了rebalance命令来让正在运行的topology重新负载均衡，具体见：<br /><a href="https://github.com/nathanmarz/storm/issues/54">https://github.com/nathanmarz/storm/issues/54</a><br />和0.6.1的变更：<br /><a href="http://groups.google.com/group/storm-user/browse_thread/thread/24a8fce0b2e53246">http://groups.google.com/group/storm-user/browse_thread/thread/24a8fce0b2e53246</a><br /><br />storm并不提供机制来动态调整worker和task数目。<br /><br /><strong>六、Storm UI里spout统计的complete latency的具体含义是什么？为什么emit的数目会是acked的两倍？</strong><br />这个事实上是storm邮件列表里的一个问题。Storm作者marz的解答：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">The&nbsp;complete&nbsp;latency&nbsp;is&nbsp;the&nbsp;time&nbsp;<strong>from&nbsp;the&nbsp;spout&nbsp;emitting&nbsp;a&nbsp;tuple&nbsp;to&nbsp;that<br />tuple&nbsp;being&nbsp;acked&nbsp;on&nbsp;the&nbsp;spout</strong>.&nbsp;So&nbsp;it&nbsp;tracks&nbsp;the&nbsp;time&nbsp;</span><span style="color: #0000FF; ">for</span><span style="color: #000000; ">&nbsp;the&nbsp;whole&nbsp;tuple<br />tree&nbsp;to&nbsp;be&nbsp;processed.<br /><br />If&nbsp;you&nbsp;dive&nbsp;into&nbsp;the&nbsp;spout&nbsp;component&nbsp;in&nbsp;the&nbsp;UI,&nbsp;you</span><span style="color: #000000; ">'</span><span style="color: #000000; ">ll&nbsp;see&nbsp;that&nbsp;a&nbsp;lot&nbsp;of</span><span style="color: #000000; "><br /></span><span style="color: #000000; ">the&nbsp;emitted</span><span style="color: #000000; ">/</span><span style="color: #000000; ">transferred&nbsp;is&nbsp;on&nbsp;the&nbsp;__ack</span><span style="color: #000000; ">*</span><span style="color: #000000; ">&nbsp;stream.&nbsp;<strong>This&nbsp;is&nbsp;the&nbsp;spout<br />communicating&nbsp;with&nbsp;the&nbsp;ackers&nbsp;which&nbsp;take&nbsp;care&nbsp;of&nbsp;tracking&nbsp;the&nbsp;tuple&nbsp;trees. </strong><br /></span></div><br />简单地说，complete latency表示了tuple从emit到被acked经过的时间，可以认为是tuple以及该tuple的后续子孙（形成一棵树）整个处理时间。其次spout的emit和transfered还统计了spout和acker之间内部的通信信息，比如对于可靠处理的spout来说，会在emit的时候同时发送一个_ack_init给acker，记录tuple id到task id的映射，以便ack的时候能找到正确的acker task。<br /><br /><img src ="http://www.blogjava.net/killme2008/aggbug/366763.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-12-19 15:25 <a href="http://www.blogjava.net/killme2008/archive/2011/12/19/366763.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Storm源码浅析之topology的提交</title><link>http://www.blogjava.net/killme2008/archive/2011/12/01/364112.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Thu, 01 Dec 2011 13:48:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/12/01/364112.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/364112.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/12/01/364112.html#Feedback</comments><slash:comments>10</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/364112.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/364112.html</trackback:ping><description><![CDATA[&nbsp;&nbsp;&nbsp; 原文：<a href="http://www.blogjava.net/killme2008/archive/2011/11/17/364112.html">http://www.blogjava.net/killme2008/archive/2011/11/17/364112.html</a><br />&nbsp;&nbsp;&nbsp; 作者：dennis (killme2008@gmail.com)<br />&nbsp;&nbsp;&nbsp; 转载请注明出处。<br /><br />&nbsp;&nbsp;&nbsp; 最近一直在读twitter开源的这个分布式流计算框架&#8212;&#8212;storm的源码，还是有必要记录下一些比较有意思的地方。我按照storm的主要概念进行组织，并且只分析我关注的东西，因此称之为浅析。&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; <br /><br />一、介绍<br />&nbsp;&nbsp;&nbsp; Storm的开发语言主要是Java和Clojure，其中Java定义骨架，而Clojure编写核心逻辑。源码统计结果：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">180</span><span style="color: #000000; ">&nbsp;text&nbsp;files.<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">177</span><span style="color: #000000; ">&nbsp;unique&nbsp;files.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">7</span><span style="color: #000000; ">&nbsp;files&nbsp;ignored.<br /><br />http:</span><span style="color: #008000; ">//</span><span style="color: #008000; ">cloc.sourceforge.net&nbsp;v&nbsp;1.55&nbsp;&nbsp;T=1.0&nbsp;s&nbsp;(171.0&nbsp;files/s,&nbsp;46869.0&nbsp;lines/s)</span><span style="color: #008000; "><br /></span><span style="color: #000000; ">-------------------------------------------------------------------------------</span><span style="color: #000000; "><br />Language&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;files&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;blank&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;comment&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;code<br /></span><span style="color: #000000; ">-------------------------------------------------------------------------------</span><span style="color: #000000; "><br />Java&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">125</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">5010</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">2414</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">25661</span><span style="color: #000000; "><br />Lisp&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">33</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">732</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">283</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">4871</span><span style="color: #000000; "><br />Python&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">7</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">742</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">433</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">4675</span><span style="color: #000000; "><br />CSS&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">1</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">12</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">45</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">1837</span><span style="color: #000000; "><br /><a title="" href="http://www.ruby-lang.org">ruby</a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">2</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">22</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">0</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">104</span><span style="color: #000000; "><br />Bourne&nbsp;Shell&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">1</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">0</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">0</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">6</span><span style="color: #000000; "><br />Javascript&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">2</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">1</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">15</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">6</span><span style="color: #000000; "><br /></span><span style="color: #000000; ">-------------------------------------------------------------------------------</span><span style="color: #000000; "><br />SUM:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">171</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">6519</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">3190</span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">37160</span><span style="color: #000000; "><br /></span><span style="color: #000000; ">-------------------------------------------------------------------------------</span></div><br />&nbsp; &nbsp; Java代码25000多行，而Clojure(Lisp)只有4871行，说语言不重要再次证明是扯淡。<br />&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; <br />二、Topology和Nimbus &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp; Topology是storm的核心理念，将spout和bolt组织成一个topology，运行在storm集群里，完成实时分析和计算的任务。这里我主要想介绍下topology部署到storm集群的大概过程。提交一个topology任务到Storm集群是通过StormSubmitter.submitTopology方法提交：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">StormSubmitter.submitTopology(name,&nbsp;conf,&nbsp;builder.createTopology());</span></div>&nbsp;&nbsp;&nbsp; 我们将topology打成jar包后，利用bin/storm这个python脚本，执行如下命令：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">bin</span><span style="color: #000000; ">/</span><span style="color: #000000; ">storm&nbsp;jar&nbsp;xxxx.jar&nbsp;com.taobao.MyTopology&nbsp;args</span></div>&nbsp;&nbsp;&nbsp; 将jar包提交给storm集群。storm脚本会启动JVM执行Topology的main方法，执行submitTopology的过程。而submitTopology会将jar文件上传到nimbus，上传是通过socket传输。在storm这个python脚本的jar方法里可以看到：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #0000FF; ">def</span><span style="color: #000000; ">&nbsp;jar(jarfile,&nbsp;klass,&nbsp;</span><span style="color: #000000; ">*</span><span style="color: #000000; ">args):&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;exec_storm_class(&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;klass,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;jvmtype</span><span style="color: #000000; ">=</span><span style="color: #800000; ">"</span><span style="color: #800000; ">-client</span><span style="color: #800000; ">"</span><span style="color: #000000; ">,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;extrajars</span><span style="color: #000000; ">=</span><span style="color: #000000; ">[jarfile,&nbsp;CONF_DIR,&nbsp;STORM_DIR&nbsp;</span><span style="color: #000000; ">+</span><span style="color: #000000; ">&nbsp;</span><span style="color: #800000; ">"</span><span style="color: #800000; ">/bin</span><span style="color: #800000; ">"</span><span style="color: #000000; ">],&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;args</span><span style="color: #000000; ">=</span><span style="color: #000000; ">args,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<strong>prefix</strong></span><strong><span style="color: #000000; ">=</span><span style="color: #800000; ">"</span><span style="color: #800000; ">export&nbsp;STORM_JAR=</span><span style="color: #800000; ">"</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">+</span><span style="color: #000000; ">&nbsp;jarfile&nbsp;</span><span style="color: #000000; ">+</span><span style="color: #000000; ">&nbsp;</span><span style="color: #800000; ">"</span><span style="color: #800000; ">;</span><span style="color: #800000; ">"</span></strong><span style="color: #000000; "><strong>)</strong> <br /></span></div>&nbsp;&nbsp;&nbsp;&nbsp; 将jar文件的地址设置为环境变量STORM_JAR，这个环境变量在执行submitTopology的时候用到：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #008000; ">//</span><span style="color: #008000; ">StormSubmitter.java&nbsp;</span><span style="color: #008000; "><br /></span><span style="color: #0000FF; ">private</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">static</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">void</span><span style="color: #000000; ">&nbsp;submitJar(Map&nbsp;conf)&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">if</span><span style="color: #000000; ">(submittedJar</span><span style="color: #000000; ">==</span><span style="color: #0000FF; ">null</span><span style="color: #000000; ">)&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;LOG.info(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">Jar&nbsp;not&nbsp;uploaded&nbsp;to&nbsp;master&nbsp;yet.&nbsp;Submitting&nbsp;jar<img src="http://www.blogjava.net/Images/dot.gif" alt="" /></span><span style="color: #000000; ">"</span><span style="color: #000000; ">);<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;String&nbsp;localJar&nbsp;</span><span style="color: #000000; ">=</span><strong><span style="color: #000000; ">&nbsp;System.getenv(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">STORM_JAR</span><span style="color: #000000; ">"</span></strong><span style="color: #000000; "><strong>)</strong>;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;submittedJar&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;submitJar(conf,&nbsp;localJar);<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp;</span><span style="color: #0000FF; ">else</span><span style="color: #000000; ">&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;LOG.info(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">Jar&nbsp;already&nbsp;uploaded&nbsp;to&nbsp;master.&nbsp;Not&nbsp;submitting&nbsp;jar.</span><span style="color: #000000; ">"</span><span style="color: #000000; ">);<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br />&nbsp;&nbsp;&nbsp;&nbsp;}</span></div>&nbsp;&nbsp;&nbsp; 通过环境变量找到jar包的地址，然后上传。利用环境变量传参是个小技巧。<br /><br />&nbsp;&nbsp;&nbsp; 其次，nimbus在接收到jar文件后，存放到数据目录的inbox目录，<strong>nimbus数据目录的结构</strong>：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">-</span><span style="color: #000000; ">nimbus<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">inbox<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">stormjar</span><span style="color: #000000; ">-</span><span style="color: #000000; ">57f1d694</span><span style="color: #000000; ">-</span><span style="color: #000000; ">2865</span><span style="color: #000000; ">-</span><span style="color: #000000; ">4b3b</span><span style="color: #000000; ">-</span><span style="color: #000000; ">8a7c</span><span style="color: #000000; ">-</span><span style="color: #000000; ">99104fc0aea3.jar<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">stormjar</span><span style="color: #000000; ">-</span><span style="color: #000000; ">76b4e316</span><span style="color: #000000; ">-</span><span style="color: #000000; ">b430</span><span style="color: #000000; ">-</span><span style="color: #000000; ">4215</span><span style="color: #000000; ">-</span><span style="color: #000000; ">9e26</span><span style="color: #000000; ">-</span><span style="color: #000000; ">4f33ba4ee520.jar<br /><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">stormdist<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">storm</span><span style="color: #000000; ">-</span><span style="color: #000000; ">id<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">stormjar.jar<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">stormconf.ser<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">stormcode.ser</span></div>&nbsp;&nbsp;&nbsp;&nbsp; 其中inbox用于存放提交的jar文件，每个jar文件都重命名为stormjar加上一个32位的UUID。而stormdist存放的是启动topology后生成的文件，每个topology都分配一个唯一的id，ID的规则是&#8220;name-计数-时间戳&#8221;。启动后的topology的jar文件名命名为storm.jar ,而它的配置经过java序列化后存放在stormconf.ser文件，而stormcode.ser是将topology本身序列化后存放的文件。<strong>这些文件在部署的时候，supervisor会从这个目录下载这些文件，然后在supervisor本地执行这些代码。</strong><br />&nbsp;&nbsp;&nbsp; 进入重点，topology任务的分配过程(zookeeper路径说明忽略root):<br />1.在zookeeper上创建/taskheartbeats/{storm id} 路径，用于任务的心跳检测。storm对zookeeper的一个重要应用就是利用zk的临时节点做存活检测。task将定时刷新节点的时间戳，然后nimbus会检测这个时间戳是否超过timeout设置。<br />2.从topology中获取bolts,spouts设置的并行数目以及全局配置的最大并行数，然后产生task id列表，如[1 2 3 4]<br />3.在zookeeper上创建/tasks/{strom id}/{task id}路径，并存储task信息<br />4.开始分配任务（内部称为assignment)， 具体步骤：<br />&nbsp;(1)从zk上获得已有的assignment(新的toplogy当然没有了）<br />&nbsp;(2)查找所有可用的slot，所谓slot就是可用的worker，在所有supervisor上配置的多个worker的端口。<br />&nbsp;(3)将任务均匀地分配给可用的worker，这里有两种情况：<br />&nbsp;(a)task数目比worker多，例如task是[1 2 3 4],可用的slot只有[host1:port1 host2:port1]，那么最终是这样分配<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">{</span><span style="color: #000000; ">1</span><span style="color: #000000; ">:&nbsp;[host1:port1]&nbsp;</span><span style="color: #000000; ">2</span><span style="color: #000000; ">&nbsp;:&nbsp;[host2:port1]<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">3</span><span style="color: #000000; ">&nbsp;:&nbsp;[host1:port1]&nbsp;</span><span style="color: #000000; ">4</span><span style="color: #000000; ">&nbsp;:&nbsp;[host2:port1]}</span></div>，可以看到任务平均地分配在两个worker上。<br />(b)如果task数目比worker少，例如task是[1 2]，而worker有[host1:port1 host1:port2 host2:port1 host2:port2]，那么首先会将woker排序，<strong>将不同host间隔排列</strong>，保证task不会全部分配到同一个worker上，也就是将worker排列成<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">[host1:port1&nbsp;host2:port1&nbsp;host1:port2&nbsp;host2:port2]</span></div>，然后分配任务为<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">{</span><span style="color: #000000; ">1</span><span style="color: #000000; ">:&nbsp;host1:port1&nbsp;,&nbsp;</span><span style="color: #000000; ">2</span><span style="color: #000000; ">&nbsp;:&nbsp;host2:port2}</span></div><br />(4)记录启动时间<br />(5)判断现有的assignment是否跟重新分配的assignment相同，如果相同，不需要变更，否则更新assignment到zookeeper的/assignments/{storm id}上。<br />5.启动topology，所谓启动，只是将zookeeper上/storms/{storm id}对应的数据里的active设置为true。<br />6.nimbus会检查task的心跳，如果发现task心跳超过超时时间，那么会重新跳到第4步做re-assignment。<br /><img src ="http://www.blogjava.net/killme2008/aggbug/364112.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-12-01 21:48 <a href="http://www.blogjava.net/killme2008/archive/2011/12/01/364112.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Yahoo! s4和Twitter storm的粗略比较</title><link>http://www.blogjava.net/killme2008/archive/2011/11/08/363238.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Tue, 08 Nov 2011 14:25:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/11/08/363238.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/363238.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/11/08/363238.html#Feedback</comments><slash:comments>1</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/363238.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/363238.html</trackback:ping><description><![CDATA[<table border="2" cellpadding="2" cellspacing="2" width="804"><tbody><tr><td>Items\Projects<br /></td>
            <td>Yahoo! s4<br /></td>
            <td>Twitter Storm<br /></td>
        </tr>
        <tr>
            <td>协议<br /></td>
            <td>Apache license 2.0<br /></td>
            <td>Eclipse Public License 1.0<br /></td>
        </tr>
        <tr>
            <td>开发语言<br /></td>
            <td>Java<br /></td>
            <td>Clojure,Java,Clojure编写了核心代码</td>
        </tr>
        <tr>
            <td>结构<br /></td>
            <td>去中心化的对等结构<br /></td>
            <td>有中心节点nimbus，但非关键</td>
        </tr>
        <tr>
            <td>通信<br /></td>
            <td>可插拔的通讯层，目前是基于UDP的实现</td>
            <td>基于facebook开源的thrift框架</td>
        </tr>
        <tr>
            <td>事件/Stream<br /></td>
            <td>&lt;K,A&gt;序列，用户可自定义事件类</td>
            <td>提供Tuple类，用户不可自定义事件类，<br />但是可以命名field和注册序列化器</td>
        </tr>
        <tr>
            <td>处理单元</td>
            <td>Processing Elements，内置PE处理<br />count,join和aggregate等常见任务</td>
            <td>Bolt，没有内置任务，提供IBasicBolt处理<br />自动ack</td>
        </tr>
        <tr>
            <td>第三方交互<br /></td>
            <td>提供API,Client Adapter/Driver，第三方客户端输入或者输出事件</td>
            <td>定义Spout用于产生Stream，没有标准输出API</td>
        </tr>
        <tr>
            <td>持久化</td>
            <td>提供Persist API规范，可根据频率或者次数做<br />持久化<br /></td>
            <td>无特定API，用户可自行选择处理<br /></td></tr><tr><td>可靠处理</td><td>&nbsp;无，可能会丢失事件</td><td>&nbsp;提供对事件处理的可靠保证（可选）</td></tr><tr><td>路由</td><td>EventType + Keyed attribute + value匹配<br />内置count,join和aggregate标准任务</td><td>Stream Groupings:<br />Shuffle,Fields,All,Global,None,Direct<br />非常灵活的路由方式</td></tr><tr><td>多语言支持</td><td>&nbsp;暂时只支持Java</td><td>多语言支持良好，本身支持Java,Clojure，<br />其他非JVM语言通过thrift和进程间通讯</td></tr><tr><td>Failover<br /></td><td>&nbsp;部分支持，数据无法failover</td><td>&nbsp;部分支持，数据同样无法failover</td></tr><tr><td>Load Balance<br /></td><td>不支持</td><td>&nbsp;不支持</td></tr><tr><td>&nbsp;并行处理</td><td>&nbsp;取决于节点数目，不可调节</td><td>&nbsp;可配置worker和task数目，storm会尽量将worker和task均匀分布</td></tr><tr><td>动态增删节点</td><td>不支持 <br /></td><td>&nbsp;支持</td></tr><tr><td>动态部署<br /></td><td>&nbsp;不支持</td><td>&nbsp;支持</td></tr><tr><td>web管理</td><td>&nbsp;不支持</td><td>&nbsp;支持</td></tr><tr><td>代码成熟度</td><td>&nbsp;半成品</td><td>&nbsp;成熟</td></tr><tr><td>活跃度</td><td>&nbsp;低</td><td>&nbsp;活跃</td></tr><tr><td>编程</td><td>&nbsp;编程 + XML配置<br /></td><td>&nbsp; 纯编程<br /></td></tr><tr><td>参考文档</td><td>&nbsp;<a href="http://docs.s4.io/">http://docs.s4.io/</a></td><td><a href="https://github.com/nathanmarz/storm/wiki/">https://github.com/nathanmarz/storm/wiki/</a><br /><a href="http://xumingming.sinaapp.com/category/storm/">http://xumingming.sinaapp.com/category/storm/</a> （非常好的中文翻译)<br /></td></tr></tbody></table><img src ="http://www.blogjava.net/killme2008/aggbug/363238.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-11-08 22:25 <a href="http://www.blogjava.net/killme2008/archive/2011/11/08/363238.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>紧急发布xmemcached 1.3.5</title><link>http://www.blogjava.net/killme2008/archive/2011/10/01/359898.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Sat, 01 Oct 2011 07:11:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/10/01/359898.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/359898.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/10/01/359898.html#Feedback</comments><slash:comments>6</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/359898.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/359898.html</trackback:ping><description><![CDATA[<br />&nbsp;&nbsp;&nbsp; xmemcached紧急发布1.3.5版本，主要是修复两个相对严重的bug:<br /><br /><a href="http://code.google.com/p/xmemcached/issues/detail?id=154">Issue 154</a>: 在重连本地memcached的时候，有可能出现重连无法成功的情况，导致连接丢失，详情见<a href="http://code.google.com/p/xmemcached/issues/detail?id=154">这里</a>。<br /><a href="http://code.google.com/p/xmemcached/issues/detail?id=155">Issue 155:</a> 重连导致文件句柄数超过限制的bug，这是由于重连失败情况下没有合理关闭socket引起的，详情见<a href="http://code.google.com/p/xmemcached/issues/detail?id=155">这里</a>。<br /><br />&nbsp;&nbsp;&nbsp; 如果你使用maven，简单升级版本即可：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">com.googlecode.xmemcached</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">xmemcached</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">1.3.5</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span></div><br /><br />&nbsp;&nbsp;&nbsp; 下载地址：<a href="http://code.google.com/p/xmemcached/downloads/list">http://code.google.com/p/xmemcached/downloads/list</a><br /><br />&nbsp;&nbsp;&nbsp; 此版本推荐升级，最后感谢两位老外开发者的帮助： 
 <a style="white-space: nowrap" href="http://code.google.com/u/ilkinulas/">ilkinulas</a>和<a href="http://code.google.com/u/@VBZVQFxQBxJNXgV0/">MrRubato</a><img src ="http://www.blogjava.net/killme2008/aggbug/359898.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-10-01 15:11 <a href="http://www.blogjava.net/killme2008/archive/2011/10/01/359898.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>UniqTask for android</title><link>http://www.blogjava.net/killme2008/archive/2011/09/20/359033.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Mon, 19 Sep 2011 20:10:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/09/20/359033.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/359033.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/09/20/359033.html#Feedback</comments><slash:comments>5</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/359033.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/359033.html</trackback:ping><description><![CDATA[&nbsp;&nbsp;&nbsp; 好吧，我知道现在是凌晨4点了，写完这个就睡觉。<br /><br />&nbsp;&nbsp;&nbsp; 我一直很不爽android的ES任务管理器，它的广告设置的地方非常恶心，就放在kill键的下面，而且每次都突然跳出来，让你很容易错误点击。我很佩服他们能想出这种提高点击率的办法，但是又无比鄙视这种做法。今天（哦，不是昨天）晚上在twitter上说了，我想自己写个任务管理器，类似ES任务管理器，并且没有广告。那好吧，说干就干，奋斗了一个晚上，终于搞出了成果，这就是隆重登场的UniqTask，先看看运行时截图：<br /><br /><div align="center"><img alt="" src="http://www.blogjava.net/images/blogjava_net/killme2008/SC20110920-040819.png" height="800" width="480" /></div><br /><br />&nbsp;&nbsp;&nbsp; 这是运行在我的GS2上的截图。<br /><br />&nbsp;&nbsp;&nbsp; UniqTask的功能跟ES任务管理器的功能完全一致，可以记录kill的历史，每次启动UniqTask的时候自动标记过去kill过的进程。但是UniqTask完全绿色无毒，绝对没有广告，咔咔。<br /><br />&nbsp;&nbsp;&nbsp; 许久没写android程序，拿起手来不是很顺利，折腾到现在才搞定，我将代码放到了github上，也提供了APK下载，非常欢迎试用啊。<br /><br />&nbsp;&nbsp;&nbsp; 源码地址：<br />&nbsp;&nbsp;&nbsp; <a href="https://github.com/killme2008/UniqTask">https://github.com/killme2008/UniqTask</a><br />&nbsp;&nbsp;&nbsp; APK下载：<br />&nbsp;&nbsp;&nbsp; <a href="https://github.com/killme2008/UniqTask/blob/master/UniqTask.apk">https://github.com/killme2008/UniqTask/blob/master/UniqTask.apk</a><br /><br />&nbsp;&nbsp;&nbsp; 白天还有重要的事情要处理，睡觉去了。<img src ="http://www.blogjava.net/killme2008/aggbug/359033.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-09-20 04:10 <a href="http://www.blogjava.net/killme2008/archive/2011/09/20/359033.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>高性能EL——Fel探秘，兼谈EL</title><link>http://www.blogjava.net/killme2008/archive/2011/09/17/358863.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Sat, 17 Sep 2011 04:52:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/09/17/358863.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/358863.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/09/17/358863.html#Feedback</comments><slash:comments>5</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/358863.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/358863.html</trackback:ping><description><![CDATA[&nbsp;&nbsp;&nbsp; Fel是最近javaeye比较火的关键词，这是由网友<a href="http://lotusyu.iteye.com/">lotusyu</a>开发的一个高性能的EL，从作者给出的数据来看，性能非常优异，跟前段时间温少开源的<a href="http://www.iteye.com/topic/993292">Simple EL</a>有的一拼。首先要说，这是个好现象，国内的开源项目越来越多，可以看出开发者的水平是越来越高了，比如我最近还看到有人开源的类似kestel的轻量级MQ&#8212;&#8212;<a href="http://code.google.com/p/fqueue/">fqueue</a>也非常不错，有兴趣可以看下我的分析《<a href="../archive/2011/09/16/358827.html">fqueue初步分析</a>》。<br />
<br />


<p>&nbsp;&nbsp;&nbsp; 进入正文，本文是尝试分析下Fel的实现原理，以及优缺点和<a href="http://code.google.com/p/aviator/">aviator</a>&#8212;&#8212;我自己开源的EL之间的简单比较。</p>


<p>&nbsp; &nbsp; Fel的实现原理跟Simple 
EL是类似，都是使用template生成中间代码&#8212;&#8212;也就是普通的java代码，然后利用javac编译成class，最后运行，当然，这个过程都是动
态的。JDK6已经引入了编译API，在此之前的版本可以调用sun的类来编译，因为javac其实就是用java实现的。回到Fel里
面，FelCompiler15就是用 <em><strong>com.sun.tools.javac.Main</strong></em>来编译，而FelCompiler16用标准的<a href="http://download.oracle.com/javase/6/docs/api/javax/tools/JavaCompiler.html">javax.tools.JavaCompiler</a>来编译的。</p>


<p> &nbsp; &nbsp; 文法和语法解释这块是使用antlr这个parse generator生成的，这块不多说，有兴趣可以看下antlr，整体一个运行的过程是这样：</p>
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;expression&nbsp;string&nbsp;</span><span style="color: #000000; ">-&gt;</span><span style="color: #000000; ">&nbsp;antlr&nbsp;</span><span style="color: #000000; ">-&gt;</span><span style="color: #000000; ">&nbsp;AST&nbsp;</span><span style="color: #000000; ">-&gt;</span><span style="color: #000000; ">&nbsp;comiple&nbsp;</span><span style="color: #000000; ">-&gt;</span><span style="color: #000000; ">&nbsp;java&nbsp;source&nbsp;template&nbsp;</span><span style="color: #000000; ">-&gt;</span><span style="color: #000000; ">&nbsp;java&nbsp;</span><span style="color: #0000FF; ">class</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">-&gt;</span><span style="color: #000000; ">&nbsp;Expression&nbsp; <br />
</span></div>

<p>&nbsp;&nbsp;&nbsp; 这个思路我在实现<a href="http://code.google.com/p/aviator/">aviator</a>之前就想过，但是后来考虑到API需要用的sun独有的类，而且要求classpath必须有tools.jar这个依赖包，就放弃了这个思路，还是采用ASM生成字节码的方式。题外，velocity的优化可以采用这个思路，我们有这么一个项目是这么做的，也准备开源了。</p>

<p>&nbsp;</p>

<p>&nbsp;&nbsp;&nbsp; 看看Fel生成的中间代码，例如a+b这样的一个简单的表达式，假设我一开始不知道a和b的类型，编译是这样：</p>
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;FelEngine&nbsp;fel&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;FelEngineImpl();&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;Expression&nbsp;exp&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000;">&nbsp; </span><span style="color: #000000; ">fel.compile(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">a+b</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #0000FF; ">null</span><span style="color: #000000; ">);&nbsp; <br /></span></div>
<p> </p>

<p>&nbsp;&nbsp;&nbsp; 我稍微改了下FEL的源码，让它打印中间生成的java代码，a+b生成的中间结果为：</p>
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">package</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.compile;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.common.NumberUtil;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.Expression;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.context.FelContext;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;org.apache.commons.lang.ObjectUtils;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;org.apache.commons.lang.StringUtils;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">class</span><span style="color: #000000; ">&nbsp;Fel_0&nbsp;&nbsp;</span><span style="color: #0000FF; ">implements</span><span style="color: #000000; ">&nbsp;Expression{&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;Object&nbsp;eval(FelContext&nbsp;context)&nbsp;{&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;java.lang.Object&nbsp;var_1&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;(java.lang.Object)context.get(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">b</span><span style="color: #000000; ">"</span><span style="color: #000000; ">);&nbsp;&nbsp;&nbsp;</span><span style="color: #008000; ">//</span><span style="color: #008000; ">b&nbsp;&nbsp;</span><span style="color: #008000; "><br /></span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;java.lang.Object&nbsp;var_0&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;(java.lang.Object)context.get(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">a</span><span style="color: #000000; ">"</span><span style="color: #000000; ">);&nbsp;&nbsp;&nbsp;</span><span style="color: #008000; ">//</span><span style="color: #008000; ">a&nbsp;&nbsp;</span><span style="color: #008000; "><br /></span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">return</span><span style="color: #000000; ">&nbsp;(ObjectUtils.toString(var_0))</span><span style="color: #000000; ">+</span><span style="color: #000000; ">(ObjectUtils.toString(var_1));&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp; <br />
</span></div>

<p>&nbsp;&nbsp;&nbsp;&nbsp; 可见，FEL对表达式解析和解释后，利用template生成这么一个普通的java类，而a和b都从context中获取并转化为<strong>Object</strong>类型，这里没有做任何判断就直接认为a和b是要做字符串相加，然后拼接字符串并返回。</p>

<p>&nbsp;</p>

<p>&nbsp;&nbsp;&nbsp;&nbsp; 问题出来了，因为没有在编译的时候传入context（我们这里是null），FEL会将a和b的类型默认都为java.lang.Object，a+b解释为字符串拼接。但是运行的时候，我完全可以传入a和b都为数字，那么结果就非常诡异了：</p>
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;FelEngine&nbsp;fel&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;FelEngineImpl();&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;Expression&nbsp;exp&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;fel.compile(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">a+b</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #0000FF; ">null</span><span style="color: #000000; ">);&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;Map</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">String,&nbsp;Object</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;env</span><span style="color: #000000; ">=</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;HashMap</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">String,&nbsp;Object</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">();&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;env.put(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">a</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #000000; ">1</span><span style="color: #000000; ">);&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;env.put(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">b</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #000000; ">3.14</span><span style="color: #000000; ">);&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(exp.eval(</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;MapContext(env)));&nbsp; <br />
</span></div>

<p>输出：</p>
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">13.14</span><span style="color: #000000; ">&nbsp; <br />
</span></div>


<p>&nbsp;&nbsp;&nbsp; 1+3.14的结果，作为字符串拼接就是13.14，而不是我们想要的4.14。如果将表达式换成a*b，就完全运行不了</p>
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;com.greenpineyu.fel.exception.CompileException:&nbsp;</span><span style="color: #0000FF; ">package</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.compile;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.common.NumberUtil;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.Expression;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.context.FelContext;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;org.apache.commons.lang.ObjectUtils;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;org.apache.commons.lang.StringUtils;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">class</span><span style="color: #000000; ">&nbsp;Fel_0&nbsp;&nbsp;</span><span style="color: #0000FF; ">implements</span><span style="color: #000000; ">&nbsp;Expression{&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;Object&nbsp;eval(FelContext&nbsp;context)&nbsp;{&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;java.lang.Object&nbsp;var_1&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;(java.lang.Object)context.get(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">b</span><span style="color: #000000; ">"</span><span style="color: #000000; ">);&nbsp;&nbsp;&nbsp;</span><span style="color: #008000; ">//</span><span style="color: #008000; ">b&nbsp;&nbsp;</span><span style="color: #008000; "><br /></span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;java.lang.Object&nbsp;var_0&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;(java.lang.Object)context.get(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">a</span><span style="color: #000000; ">"</span><span style="color: #000000; ">);&nbsp;&nbsp;&nbsp;</span><span style="color: #008000; ">//</span><span style="color: #008000; ">a&nbsp;&nbsp;</span><span style="color: #008000; "><br /></span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">return</span><span style="color: #000000; ">&nbsp;(var_0)</span><span style="color: #000000; ">*</span><span style="color: #000000; ">(var_1);&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;[Fel_0.java:</span><span style="color: #000000; ">14</span><span style="color: #000000; ">:&nbsp;运算符&nbsp;</span><span style="color: #000000; ">*</span><span style="color: #000000; ">&nbsp;不能应用于&nbsp;java.lang.Object,java.lang.Object]&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;com.greenpineyu.fel.compile.FelCompiler16.compileToClass(FelCompiler16.java:</span><span style="color: #000000; ">113</span><span style="color: #000000; ">)&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;com.greenpineyu.fel.compile.FelCompiler16.compile(FelCompiler16.java:</span><span style="color: #000000; ">87</span><span style="color: #000000; ">)&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;com.greenpineyu.fel.compile.CompileService.compile(CompileService.java:</span><span style="color: #000000; ">66</span><span style="color: #000000; ">)&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;com.greenpineyu.fel.FelEngineImpl.compile(FelEngineImpl.java:</span><span style="color: #000000; ">62</span><span style="color: #000000; ">)&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;TEst.main(TEst.java:</span><span style="color: #000000; ">14</span><span style="color: #000000; ">)&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;Exception&nbsp;in&nbsp;thread&nbsp;</span><span style="color: #000000; ">"</span><span style="color: #000000; ">main</span><span style="color: #000000; ">"</span><span style="color: #000000; ">&nbsp;java.lang.NullPointerException&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;TEst.main(TEst.java:</span><span style="color: #000000; ">18</span><span style="color: #000000; ">)&nbsp; <br />
</span></div>
<p>&nbsp;</p>

<p>&nbsp;&nbsp;&nbsp; 这个问题对于Simple EL同样存在，如果没有在编译的时候能确定变量类型，这无法生成正确的中间代码，导致运行时出错，并且有可能造成非常诡异的bug。</p>

<p>&nbsp;</p>

<p>&nbsp;&nbsp;&nbsp; 这个问题的本质是<strong>因为Fel和Simple EL没有自己的类型系统，他们都是直接使用java的类型的系统，并且必须在编译的时候确定变量类型，才能生成高效和正确的代码，我们可以将它们称为&#8220;强类型的EL&#8220;。</strong></p>

<p>&nbsp;</p>


<p>&nbsp;&nbsp;&nbsp; 现在让我们在编译的时候给a和b加上类型，看看生成的中间代码：</p>
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;FelEngine&nbsp;fel&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;FelEngineImpl();&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;fel.getContext().set(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">a</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #000000; ">1</span><span style="color: #000000; ">);&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;fel.getContext().set(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">b</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #000000; ">3.14</span><span style="color: #000000; ">);&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;Expression&nbsp;exp&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;fel.compile(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">a+b</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #0000FF; ">null</span><span style="color: #000000; ">);&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;Map</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">String,&nbsp;Object</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;env&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;HashMap</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">String,&nbsp;Object</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">();&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;env.put(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">a</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #000000; ">1</span><span style="color: #000000; ">);&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;env.put(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">b</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #000000; ">3.14</span><span style="color: #000000; ">);&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(exp.eval(</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;MapContext(env)));&nbsp; <br />
</span></div>
<p>&nbsp;&nbsp;&nbsp; 查看中间代码：</p>
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">package</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.compile;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.common.NumberUtil;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.Expression;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;com.greenpineyu.fel.context.FelContext;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;org.apache.commons.lang.ObjectUtils;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">import</span><span style="color: #000000; ">&nbsp;org.apache.commons.lang.StringUtils;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">class</span><span style="color: #000000; ">&nbsp;Fel_0&nbsp;&nbsp;</span><span style="color: #0000FF; ">implements</span><span style="color: #000000; ">&nbsp;Expression{&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;Object&nbsp;eval(FelContext&nbsp;context)&nbsp;{&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">double</span><span style="color: #000000; ">&nbsp;var_1&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;((java.lang.Number)context.get(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">b</span><span style="color: #000000; ">"</span><span style="color: #000000; ">)).doubleValue();&nbsp;&nbsp;&nbsp;</span><span style="color: #008000; ">//</span><span style="color: #008000; ">b&nbsp;&nbsp;</span><span style="color: #008000; "><br /></span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">double</span><span style="color: #000000; ">&nbsp;var_0&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;((java.lang.Number)context.get(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">a</span><span style="color: #000000; ">"</span><span style="color: #000000; ">)).doubleValue();&nbsp;&nbsp;&nbsp;</span><span style="color: #008000; ">//</span><span style="color: #008000; ">a&nbsp;&nbsp;</span><span style="color: #008000; "><br /></span><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">return</span><span style="color: #000000; ">&nbsp;(var_0)</span><span style="color: #000000; ">+</span><span style="color: #000000; ">(var_1);&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp; <br />
</span></div>

<p>可以看到这次将a和b都强制转为double类型了，做数值相加，结果也正确了：</p>
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000; ">4.140000000000001</span><span style="color: #000000; ">&nbsp; <br />
</span></div>
<p> </p>

<p>&nbsp;&nbsp;&nbsp; Simple EL我没看过代码，这里猜测它的实现也应该是类似的，也应该有同样的问题。</p>


<p>&nbsp;&nbsp;&nbsp; 相比来说，<a href="http://code.google.com/p/aviator/">aviator</a>这是<strong>一个弱类型的EL</strong>，<strong>在编译的时候不对变量类型做任何假设，而是在运行时做类型判断和自动转化</strong>。过去提过，我给aviator的定位是一个介于EL和script之间的东西，<strong>它有自己的类型系统</strong>。
例如，3这个数字，在java里可能是long,int,short,byte，而aviator统一为AviatorLong这个类型。为了在这两个类
型之间做适配，就需要做很多的判断和box,unbox操作。这些判断和转化都是运行时进行的，因此aviator没有办法做到Fel这样的高效，但是已
经做到至少跟groovy这样的弱类型脚本语言一个级别，也超过了JXEL这样的纯解释EL，具体可以看这个<a href="http://code.google.com/p/aviator/wiki/Performance">性能测试</a>。</p>

<p>&nbsp;</p>

<p>&nbsp;&nbsp; 强类型还是弱类型，这是一个选择问题，如果你能在运行前就确定变量的类型，那么使用Fel应该可以达到或者接近于原生java执行的效率，但是失去了灵活性；如果你无法确定变量类型，则只能采用弱类型的EL。</p>

<p>&nbsp;</p>

<p>&nbsp;&nbsp; 
EL涌现的越来越多，这个现象有点类似消息中间件领域，越来越多面向特定领域的轻量级MQ的出现，而不是原来那种大而笨重的通用MQ大行其道，一方面是互
联网应用的发展，需求不是通用系统能够满足的，另一方面我认为也是开发者素质的提高，大家都能造适合自己的轮子。从EL这方面来说，我也认为会有越来越多
特定于领域的，优点和缺点一样鲜明的EL出现，它们包含设计者自己的目标和口味，选择很多，就看取舍。</p><img src ="http://www.blogjava.net/killme2008/aggbug/358863.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-09-17 12:52 <a href="http://www.blogjava.net/killme2008/archive/2011/09/17/358863.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>fqueue初步分析</title><link>http://www.blogjava.net/killme2008/archive/2011/09/16/358827.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Fri, 16 Sep 2011 12:10:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/09/16/358827.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/358827.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/09/16/358827.html#Feedback</comments><slash:comments>7</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/358827.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/358827.html</trackback:ping><description><![CDATA[<br />&nbsp;&nbsp;&nbsp; fqueue是国产的一个类似memcacheq,kestrel这样的支持memcached协议的轻量级开源MQ。它的项目主页：<br /><a href="http://code.google.com/p/fqueue/downloads/list" target="_blank">http://code.google.com/p/<wbr>fqueue/downloads/list</a>，介绍和特点都可以看主页，我就不废话了。<br /><br />&nbsp;&nbsp;&nbsp; 今天老大提到， co了源码看了下，写个初步分析报告。<br /><br />&nbsp;&nbsp;&nbsp; 首先是它的存储层，主要是一个FQueue这么一个抽象队列，内部实现是FSQueue，也就是基于文件的FIFO队列。这个队列是多个文件组成的。每个文件默认大小在150M，超过即切换一个新文件来写。读的时候如果读到尾部，则查找下一个文件进行读取。数据文件名以idb为后缀，并且从编号1开始递增，除了数据文件外，每个队列还有个db为后缀的索引文件，记录当前写和读的数据文件编号和偏移量。目录结构大概是这样：<br />&nbsp;&nbsp;&nbsp; --fqueue<br />&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; --fqueuedata_1.idb<br />&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; --fqueuedata_2.idb<br />&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; --&#8230;&#8230;<br />&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; --icqueue.db<br /><br />&nbsp;&nbsp;&nbsp; 文件的存储比较有特色，采用<a href="http://download.oracle.com/javase/1.4.2/docs/api/java/nio/MappedByteBuffer.html">MappedByteBuffer</a>做文件读写，<a href="http://download.oracle.com/javase/1.4.2/docs/api/java/nio/MappedByteBuffer.html">MappedByteBuffer</a>是java nio引入的文件内存映射方案，读写性能极高，但是也有一定的问题，比如说内存占用，以及数据刷入设备的不确定性和关闭问题。在fqueue中，每隔10毫秒会强制force一次buffer，将修改过的数据刷入设备。对于关闭问题，则采用那个技巧，示例代码：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #008000; ">/**</span><span style="color: #008000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;*&nbsp;关闭索引文件<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #008000; ">*/</span><span style="color: #000000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">void</span><span style="color: #000000; ">&nbsp;close()&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">try</span><span style="color: #000000; ">&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mappedByteBuffer.force();<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AccessController.doPrivileged(</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;PrivilegedAction</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">Object</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">()&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;Object&nbsp;run()&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">try</span><span style="color: #000000; ">&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Method&nbsp;getCleanerMethod&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;mappedByteBuffer.getClass().getMethod(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">cleaner</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;Class[</span><span style="color: #000000; ">0</span><span style="color: #000000; ">]);<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;getCleanerMethod.setAccessible(</span><span style="color: #0000FF; ">true</span><span style="color: #000000; ">);<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sun.misc.Cleaner&nbsp;cleaner&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;(sun.misc.Cleaner)&nbsp;getCleanerMethod.invoke(mappedByteBuffer,<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;Object[</span><span style="color: #000000; ">0</span><span style="color: #000000; ">]);<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cleaner.clean();<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp;</span><span style="color: #0000FF; ">catch</span><span style="color: #000000; ">&nbsp;(Exception&nbsp;e)&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;log.error(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">close&nbsp;logindexy&nbsp;file&nbsp;error:</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;e);<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">return</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">null</span><span style="color: #000000; ">;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;});<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;fc.close();<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dbRandFile.close();<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mappedByteBuffer&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">null</span><span style="color: #000000; ">;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;fc&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">null</span><span style="color: #000000; ">;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dbRandFile&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">null</span><span style="color: #000000; ">;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp;</span><span style="color: #0000FF; ">catch</span><span style="color: #000000; ">&nbsp;(IOException&nbsp;e)&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;log.error(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">close&nbsp;logindex&nbsp;file&nbsp;error:</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;e);<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br />&nbsp;&nbsp;&nbsp;&nbsp;}</span></div><br />&nbsp;&nbsp;&nbsp; 利用反射，并且使用了sun特有的类，不具有可移植性。MappedByteBuffer还有一个问题是map的代价比较高，可能在切换文件的时候fqueue会有一定程度的阻塞现象。<br /><br />&nbsp;&nbsp;&nbsp; 存储的性能，我在我的机器测试了下，似乎没有作者宣称的那么高，我的机器是5400转的普通SATA盘，写入1K数据的平均QPS在8000左右。我估计fqueue的性能跟磁盘有很大关系，如果使用15000转的SAS盘应该能有很大改观。<br /><br />&nbsp;&nbsp;&nbsp; 网络层直接使用了jmemcached的实现，<a href="http://code.google.com/p/jmemcache-daemon/">jmemcached</a>是一个java实现的memcached，通常用于单元测试之类。看情况fqueue也支持memcached的二进制协议了。网络框架使用了netty3，这些就不多说了。自己看都明白。额外提一下，作者做的单元测试使用了<a href="http://code.google.com/p/xmemcached">xmemcached</a>，咔咔，广而告之。<br /><br />&nbsp;&nbsp;&nbsp; 总体来说fqueue是一个整体上很清爽和轻量级的MQ实现，适合一些特定的场景，至于性能，我们下周准备做个压测，到时候再谈吧。<img src ="http://www.blogjava.net/killme2008/aggbug/358827.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-09-16 20:10 <a href="http://www.blogjava.net/killme2008/archive/2011/09/16/358827.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>xmemcached发布1.3.4</title><link>http://www.blogjava.net/killme2008/archive/2011/09/08/358317.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Thu, 08 Sep 2011 10:55:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/09/08/358317.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/358317.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/09/08/358317.html#Feedback</comments><slash:comments>3</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/358317.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/358317.html</trackback:ping><description><![CDATA[<div id="news_content">
      <p>&nbsp;&nbsp;&nbsp; 开源的java memcached client&#8212;&#8212; <a href="http://code.google.com/p/xmemcached/">xmemcached</a>发布1.3.4版本，主要改进如下：</p>
<p>&nbsp;</p>
<p>1、修复一个相对严重的bug，在解析二进制协议时如果遇到从服务端返回的错误信息，会导致连接异常断开；如果你没有使用binary协议，不会遇到此问题。<strong><strong>建议使用xmemcached并且使用二进制协议的朋友升级到此版本</strong>。</strong></p>


<p>2、允许XMemcachedClientFactoryBean配置opTimeout选项。 <br /></p>
<p>3、添加RoundRobinMemcachedSessionLocator，轮询的连接选择器，仅用于kestrel或者memcacheq集群，这些应用都不要求同一个key要保存在固定的服务器上，而仅是作为集群分担负载。</p>

<p>4、<a href="http://xmemcached.googlecode.com/svn/trunk/apidocs/net/rubyeye/xmemcached/impl/KetamaMemcachedSessionLocator.html">KetamaMemcachedSessionLocator</a>添加额外选项，允许配置是否兼容 <a href="https://github.com/dctrwatson/nginx-upstream-consistent">nginx-upstream-consistent</a>，这个是网友<strong> </strong> 
 <span>
 <a href="http://code.google.com/u/wolfg1969/">wolfg1969</a>贡献的patch。如果要使得xmc的一致性哈希算法兼容</span>nginx-upstream-consistent，只要设置cwNginxUpstreamConsistent为true即可，示范代码：</p>

<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;MemcachedClientBuilder&nbsp;builder&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;XMemcachedClientBuilder(&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddrUtil.getAddresses(servers));&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;builder.setSessionLocator(</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;KetamaMemcachedSessionLocator(&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">true</span><span style="color: #000000; ">));&nbsp; <br /></span></div><p>5、修复bug，包括<a class="closed_ref" title="Typo in INFO message: log.info(&quot;Creating &quot; + selectorPoolSize + &quot; rectors...&quot;);" href="http://code.google.com/p/xmemcached/issues/detail?id=132">issue 132&nbsp;</a>,<a class="closed_ref" title="Counter的问题，请确认是否为BUG" href="http://code.google.com/p/xmemcached/issues/detail?id=142">&nbsp;issue 142&nbsp;</a>,<a class="closed_ref" title="XMemcachedClientFactoryBean doesn't allow overriding of certain client properties" href="http://code.google.com/p/xmemcached/issues/detail?id=133">&nbsp;issue 133&nbsp;</a>,<a class="closed_ref" title="Network layout exception" href="http://code.google.com/p/xmemcached/issues/detail?id=139">&nbsp;issue 139&nbsp;</a>,<a class="closed_ref" title="Counter的问题，请确认是否为BUG" href="http://code.google.com/p/xmemcached/issues/detail?id=142">&nbsp;issue 142&nbsp;</a>,<a class="closed_ref" title="为配合nginx-upstream-consistent模块做了一个patch" href="http://code.google.com/p/xmemcached/issues/detail?id=145">&nbsp;issue 145&nbsp;</a>,<a title="Throw &quot;Not a proper response&quot; MemcachedException when using binary protocol" href="http://code.google.com/p/xmemcached/issues/detail?id=150">issue 150</a>等。</p>
<p>&nbsp;</p>
<p>如果你使用maven，只要简单升级版本即可：&nbsp; <br /></p><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">com.googlecode.xmemcached</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">xmemcached</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">1.3.4</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">&nbsp;&nbsp;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">&nbsp; <br /></span></div><p>下载地址：</p>
<p><a href="http://code.google.com/p/xmemcached/downloads/list">http://code.google.com/p/xmemcached/downloads/list</a></p>
    </div><img src ="http://www.blogjava.net/killme2008/aggbug/358317.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-09-08 18:55 <a href="http://www.blogjava.net/killme2008/archive/2011/09/08/358317.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>找bug记(2)</title><link>http://www.blogjava.net/killme2008/archive/2011/09/02/357774.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Thu, 01 Sep 2011 16:02:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/09/02/357774.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/357774.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/09/02/357774.html#Feedback</comments><slash:comments>5</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/357774.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/357774.html</trackback:ping><description><![CDATA[<br />&nbsp;&nbsp;&nbsp; 这篇blog迟到了很久，本来是想写另一个跟网络相关bug的查找过程，偷偷懒，写下最近印象比较深刻的bug。这个bug是我的同事水寒最终定位到的。<br />&nbsp;&nbsp;&nbsp; 前几个月同事报告称有一个线上MQ集群会同一时间抛出ArrayIndexOutOfBoundsException这个异常，也就是数组越界。查看源码，除去一些无关紧要的细节大概是这样子：<br /><div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">class</span><span style="color: #000000; ">&nbsp;ConnectionSelector{<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">private</span><span style="color: #000000; ">&nbsp;AtomicInteger&nbsp;sets</span><span style="color: #000000; ">=</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;AtomicInteger(</span><span style="color: #000000; ">0</span><span style="color: #000000; ">);<br /><br />&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">void</span><span style="color: #000000; ">&nbsp;selectConnection(List</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">Connection</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;connList){<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">if</span><span style="color: #000000; ">(connList</span><span style="color: #000000; ">==</span><span style="color: #0000FF; ">null</span><span style="color: #000000; ">)｛<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">return</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">null</span><span style="color: #000000;">;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ｝<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">final</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">int</span><span style="color: #000000; ">&nbsp;size&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;connList.size();<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">if</span><span style="color: #000000; ">&nbsp;(size&nbsp;</span><span style="color: #000000; ">==</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">0</span><span style="color: #000000; ">)&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">return</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">null</span><span style="color: #000000;">;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: red;">return</span><span style="color: red;">&nbsp;connList.get(sets.incrementAndGet()&nbsp;</span><span style="color: red;">%</span><span style="color: red;">&nbsp;size);<br /></span><span>}</span><span style="color: #000000;"><br /><br />&nbsp;&nbsp;&nbsp;}</span></div><br />&nbsp;&nbsp;&nbsp; 很显然，这里的本意是实现一个轮询的连接选择器，返回一个选中的连接。使用AtomicInteger递增并对链表大小取模，返回结果索引位置的连接。异常抛出的位置就是我代码中标红的位置。<br /><br />&nbsp;&nbsp;&nbsp; 显然，这里有两种可能，一种情况下是说在执行那一行代码的时候，connList的大小缩小了（也就是说连接可能被其他线程移出），那么导致取模的结果越界。另一种可能是取模的结果本身确实超过了列表范围。<br /><br />&nbsp;&nbsp;&nbsp; 第一种情况是完全可能的，因为服务器的连接可能随时断开或者重连，但是这种情况相对非常少见，因此我们这里并没有对这个选择过程做同步，主要是从性能的角度出发，偶尔的失败可以接受。很遗憾的是，我被我的思维惯性误导了，从来没有怀疑过第二种情况，总是认为是不是真的连接恰巧断开导致这个异常，但是却无法解释这个异常发生后就一直错误下去，无法自行恢复。<br />&nbsp;&nbsp;&nbsp; 为什么说思维惯性误导呢？这里的问题其实是负数取模的问题，对一个负数进行取模，结果会是正数还是负数？答案是结果因语言而异。<br />&nbsp;&nbsp;&nbsp; 我很早以前在使用Ruby的时候做过测试，负数取模结果为正数，例如在irb里尝试下：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">&gt;&gt;</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">1000</span><span style="color: #000000; ">%</span><span style="color: #000000; ">3</span><span style="color: #000000; "><br /></span><span style="color: #000000; ">=&gt;</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">2</span><span style="color: #000000; "><br /></span><span style="color: #000000; ">&gt;&gt;</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">2001</span><span style="color: #000000; ">%</span><span style="color: #000000; ">4</span><span style="color: #000000; "><br /></span><span style="color: #000000; ">=&gt;</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">3</span></div><br />&nbsp;&nbsp;&nbsp; 这个印象持续至今，在clojure里结果也是这样子：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">Clojure&nbsp;</span><span style="color: #000000; ">1.2</span><span style="color: #000000; ">.</span><span style="color: #000000; ">1</span><span style="color: #000000; "><br />user</span><span style="color: #000000; ">=&gt;</span><span style="color: #000000; ">&nbsp;(mod&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">1000</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">3</span><span style="color: #000000; ">)<br /></span><span style="color: #000000; ">2</span><span style="color: #000000; "><br />user</span><span style="color: #000000; ">=&gt;</span><span style="color: #000000; ">&nbsp;(mod&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">2001</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">4</span><span style="color: #000000; ">)<br /></span><span style="color: #000000; ">3</span></div><br />&nbsp;&nbsp;&nbsp; 可以再试试python:<br /><div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">Python&nbsp;</span><span style="color: #000000; ">2.7</span><span style="color: #000000; ">.</span><span style="color: #000000; ">1</span><span style="color: #000000; ">&nbsp;(r271:</span><span style="color: #000000; ">86832</span><span style="color: #000000; ">,&nbsp;Jun&nbsp;</span><span style="color: #000000; ">16</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">2011</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #000000; ">16</span><span style="color: #000000; ">:</span><span style="color: #000000; ">59</span><span style="color: #000000; ">:</span><span style="color: #000000; ">05</span><span style="color: #000000; ">)&nbsp;<br />[GCC&nbsp;</span><span style="color: #000000; ">4.2</span><span style="color: #000000; ">.</span><span style="color: #000000; ">1</span><span style="color: #000000; ">&nbsp;(Based&nbsp;on&nbsp;Apple&nbsp;Inc.&nbsp;build&nbsp;</span><span style="color: #000000; ">5658</span><span style="color: #000000; ">)&nbsp;(LLVM&nbsp;build&nbsp;</span><span style="color: #000000; ">2335.15</span><span style="color: #000000; ">.</span><span style="color: #000000; ">00</span><span style="color: #000000; ">)]&nbsp;on&nbsp;darwin<br />Type&nbsp;</span><span style="color: #000000; ">"</span><span style="color: #000000; ">help</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #000000; ">"</span><span style="color: #000000; ">copyright</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;</span><span style="color: #000000; ">"</span><span style="color: #000000; ">credits</span><span style="color: #000000; ">"</span><span style="color: #000000; ">&nbsp;or&nbsp;</span><span style="color: #000000; ">"</span><span style="color: #000000; ">license</span><span style="color: #000000; ">"</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">for</span><span style="color: #000000; ">&nbsp;more&nbsp;information.<br /></span><span style="color: #000000; ">&gt;&gt;&gt;</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">10000</span><span style="color: #000000; ">%</span><span style="color: #000000; ">3</span><span style="color: #000000; "><br /></span><span style="color: #000000; ">2</span><span style="color: #000000; "><br /></span><span style="color: #000000; ">&gt;&gt;&gt;</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">2001</span><span style="color: #000000; ">%</span><span style="color: #000000; ">4</span><span style="color: #000000; "><br /></span><span style="color: #000000; ">3</span></div><br />&nbsp;&nbsp;&nbsp; 这三种语言的结果完全一致，结果都为正数。这个惯性思维延续到java却不成立了，可惜我根本没做测试，让我们试下：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">static</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">void</span><span style="color: #000000; ">&nbsp;main(</span><span style="color: #0000FF; ">final</span><span style="color: #000000; ">&nbsp;String[]&nbsp;args)&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(</span><span style="color: #000000; ">-</span><span style="color: #000000; ">1000</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">%</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">3</span><span style="color: #000000; ">);<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(</span><span style="color: #000000; ">-</span><span style="color: #000000; ">2001</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">%</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">4</span><span style="color: #000000; ">);<br />&nbsp;&nbsp;&nbsp;&nbsp;}</span></div><br />打印结果为：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">-</span><span style="color: #000000; ">1</span><span style="color: #000000; "><br /></span><span style="color: #000000; ">-</span><span style="color: #000000; ">1</span></div><br />&nbsp;&nbsp;&nbsp; 果然，在java里负数取模的结果为负数，而不是我习惯性地认为是正数。因此最终的定位到的原因就是sets这个变量递增超过Integer.MAX_VALUE后越界变成负数了，取模的结果为负数，导致抛出数组越界的异常，这也解释了为什么同一个集群都在同一时间出问题，因为这个集群内的机器启动时间相邻并且调用这个方法次数相对平均。<strike>修正问题很简单，加个Math.abs就好。</strike><br /><br />&nbsp;&nbsp;&nbsp; Update:加个abs是不够的，因为Math.abs的javadoc提醒了：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">Note&nbsp;that&nbsp;</span><span style="color: #0000FF; ">if</span><span style="color: #000000; ">&nbsp;the&nbsp;argument&nbsp;is&nbsp;equal&nbsp;to&nbsp;the&nbsp;value&nbsp;of&nbsp;Integer.MIN_VALUE,&nbsp;the&nbsp;most&nbsp;negative&nbsp;representable&nbsp;</span><span style="color: #0000FF; ">int</span><span style="color: #000000; ">&nbsp;value,&nbsp;the&nbsp;result&nbsp;is&nbsp;that&nbsp;same&nbsp;value,&nbsp;which&nbsp;is&nbsp;negative. <br /></span></div><br />&nbsp;&nbsp;&nbsp; 也就是说对Integer.MIN_VALUE做abs结果仍然是负数。尽管在这个场景中失败一次可以接受，但是最好的办法还是回复中steven提到的抵消符号位的做法：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">(sets.incrementAndGet()&nbsp;</span><span style="color: #000000; ">&amp;</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">0x7FFFFFFF</span><span style="color: #000000; ">)&nbsp;</span><span style="color: #000000; ">%</span><span style="color: #000000; ">&nbsp;size</span></div>&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp; 这个问题更详细的讨论后来我找到<a href="http://ceeji.net/blog/mod-in-real/">这篇博客</a>，作者讨论几种语言和计算器的这个问题的结果，给出了一些结论。不过我觉的这个结论可能也不是那么可靠，特别是对c/c++来说，很大程度上应该还是依赖于实现，最可靠的办法还是强制结果为正。<br /><br />&nbsp;&nbsp;&nbsp; 这个bug的几个教训：<br />1、首先是第一次出现的时候没有引起足够重视，重启解决问题后没有深究。有句玩笑话：99％的程序问题都可以通过重启解决。但是事实上问题仍然存在，该发生的终究还会发生。不管你信不信，它就是发生了，这是一个奇迹。<br />2、注意大脑的思维惯性，经验主义和教条主义都不可取。最近在读一本好书《<a href="http://book.douban.com/subject/6709809/">暗时间</a>》，大脑误导我们的手段可是多种多样。<br />3、最后就是这个负数取模的结果因语言而异，不要依赖于特定实现。<br />&nbsp;&nbsp;&nbsp;<img src ="http://www.blogjava.net/killme2008/aggbug/357774.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-09-02 00:02 <a href="http://www.blogjava.net/killme2008/archive/2011/09/02/357774.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>轻量级高性能的表达式求值器——aviator发布2.0</title><link>http://www.blogjava.net/killme2008/archive/2011/07/13/354296.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Wed, 13 Jul 2011 14:34:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/07/13/354296.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/354296.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/07/13/354296.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/354296.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/354296.html</trackback:ping><description><![CDATA[<p>&nbsp;</p>
<p>&nbsp;&nbsp;&nbsp; <a href="http://code.google.com/p/aviator/">aviator</a>是一个轻量级的、高性能的Java表达式求值器，主要应用在如工作流引擎节点条件判断、MQ中的消息过滤以及某些特定的业务场景。</p>
<p>&nbsp;&nbsp;&nbsp; 自从上次发布1.0后，还发过1.01版本，不过都没怎么宣传。这次发布一个2.0的里程碑版本，主要改进如下：</p>
<br />
<p>1、完整支持位运算符，与java完全一致。位预算符对实现bit set之类的需求还是非常必须的。 <br />
</p>
<p>2、性能优化，平均性能提升100%，函数调用性能提升200%，最新的与groovy和JEXL的性能测试看这里</p>
<p><a href="http://code.google.com/p/aviator/wiki/Performance">http://code.google.com/p/aviator/wiki/Performance</a> <br />
</p>
<p>3、添加了新函数，包括long、double、str用于类型转换，添加了string.indexOf函数。</p>
<p>4、完善了用户手册，更新性能测试。</p>
<p>&nbsp;</p>
<p>下载地址：&nbsp;<a href="http://code.google.com/p/aviator/downloads/list"> http://code.google.com/p/aviator/downloads/list</a></p>
<p>项目主页：&nbsp; <a href="http://code.google.com/p/aviator/">http://code.google.com/p/aviator/</a></p>
<p>用户指南：&nbsp; <a href="http://code.google.com/p/aviator/w/list">http://code.google.com/p/aviator/w/list</a></p>
<p>性能报告：&nbsp; <a href="http://code.google.com/p/aviator/wiki/Performance">http://code.google.com/p/aviator/wiki/Performance</a></p>
<p>源码：&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <a href="https://github.com/killme2008/aviator">https://github.com/killme2008/aviator</a></p>
<p>&nbsp;</p><p>Maven引用（感谢许老大的帮助）：</p><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">com.googlecode.aviator</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">aviator</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">2.0</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span></div>
<p>&nbsp;&nbsp;&nbsp;&nbsp; 这个项目目前用在我们的MQ产品中做消息过滤，也有几个公司外的用户告诉我他们也在用，不过估计不会很多。有这种需求的场景还是比较少的。这个项目实际上是为我们的MQ定制的，我主要想做到这么几点：</p>
<p>（1）控制用户能够使用的函数，不允许调用任何不受控制的函数。</p>
<p>（2）轻量级，不需要嵌入groovy这么大的脚本引擎，我们只需要一个剪裁过的表达式语法即可。</p>
<p>（3）高性能，最终的性能在某些场景比groovy略差，但是已经非常接近。</p>
<p>（4）易于扩展，可以容易地添加函数扩展功能。语法相对固定。</p>
<p>（5）函数的调用避免使用反射。因此没使用dot运算符的函数调用方式，而是更类似c语言和lua语言的函数调用风格。函数是一等公民，seq库的风格很符合我的喜好。</p>
<p>&nbsp; seq这概念来自clojure，我将实现了java.util.Collection接口的类和数组都称为seq集合，可以统一使用seq库操作。例如假设我有个list:</p>
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">String,&nbsp;Object</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;env&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;HashMap</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">String,&nbsp;Object</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ArrayList</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">Integer</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;list&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;ArrayList</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">Integer</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;list.add(</span><span style="color: #000000; ">3</span><span style="color: #000000; ">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;list.add(</span><span style="color: #000000; ">100</span><span style="color: #000000; ">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;list.add(</span><span style="color: #000000; ">-</span><span style="color: #000000; ">100</span><span style="color: #000000; ">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;env.put(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">list</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,&nbsp;list);</span></div>
<br />&nbsp;&nbsp; 可以做这么几个事情，度量大小：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">count(list)</span></div>&nbsp;&nbsp; 判断元素是否存在：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">include(list,</span><span style="color: #000000; ">3</span><span style="color: #000000; ">)</span></div>&nbsp;&nbsp; 过滤元素，返回大于0的元素组成的seq：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">filter(list,seq.gt(</span><span style="color: #000000; ">0</span><span style="color: #000000; ">))<br /></span></div>&nbsp;&nbsp; 对集合里的元素求和，应用reduce:<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">reduce(list,</span><span style="color: #000000; ">+</span><span style="color: #000000; ">,</span><span style="color: #000000; ">0</span><span style="color: #000000; ">)</span></div>&nbsp;&nbsp; 遍历集合元素并打印：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">map(list,println)</span></div>&nbsp;&nbsp; 最后，你还可以排序:<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">sort(list)</span></div><br />&nbsp;&nbsp;&nbsp; 这些函数类似FP里的高阶函数，使用起来还是非常爽的。<br /><br />&nbsp;&nbsp;&nbsp; 对函数调用的优化，其实只干了一个事情，原来函数调用我是将所有参数收集到一个list里面，然后再转成数组元素交给AviatorFunction调用。这里创建了两个临时对象：list和数组。这其实是没有必要的，我只要在AviatorFunction里定义一系列重载方法，如：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;AviatorObject&nbsp;call(Map</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">String,&nbsp;Object</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;env);<br /><br /><br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;AviatorObject&nbsp;call(Map</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">String,&nbsp;Object</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;env,&nbsp;AviatorObject&nbsp;arg1);<br /><br /><br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;AviatorObject&nbsp;call(Map</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">String,&nbsp;Object</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;env,&nbsp;AviatorObject&nbsp;arg1,&nbsp;AviatorObject&nbsp;arg2);<br /><br /><br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;AviatorObject&nbsp;call(Map</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">String,&nbsp;Object</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;env,&nbsp;AviatorObject&nbsp;arg1,&nbsp;AviatorObject&nbsp;arg2,&nbsp;AviatorObject&nbsp;arg3);<br /><br />&nbsp;&nbsp;&nbsp;&nbsp;<img src="http://www.blogjava.net/Images/dot.gif" alt="" /><img src="http://www.blogjava.net/Images/dot.gif" alt="" /></span></div><br />&nbsp;&nbsp; 就不需要收集参数，而是直接invokeinterface调用AviatorFunction相应的重载方法即可。我看到在JRuby和Clojure里的方法调用都这样干的。过去的思路走岔了。最终也不需要区分内部的method和外部的function，统一为一个对象即可，进一步减少了对象创建的开销。<img src ="http://www.blogjava.net/killme2008/aggbug/354296.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-07-13 22:34 <a href="http://www.blogjava.net/killme2008/archive/2011/07/13/354296.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>找bug记（1）</title><link>http://www.blogjava.net/killme2008/archive/2011/07/10/354062.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Sun, 10 Jul 2011 15:29:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/07/10/354062.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/354062.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/07/10/354062.html#Feedback</comments><slash:comments>16</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/354062.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/354062.html</trackback:ping><description><![CDATA[<br />
&nbsp;&nbsp;&nbsp; 转载请注明出处 <a href="http://www.blogjava.net/killme2008/archive/2011/07/10/354062.html">http://www.blogjava.net/killme2008/archive/2011/07/10/354062.html</a><br />
<br />
&nbsp;&nbsp;&nbsp; 上周在线上系统发现了两个bug，值得记录下查找的过程和原因。以后如果还有查找bug比较有价值的经历，我也会继续分享。<br />
&nbsp;&nbsp;&nbsp; 第一个bug的起始，是在线上日志发现一个频繁打印的异常&#8212;&#8212;java.lang.ArrayIndexOutOfBoundsException。但是却没有堆栈，只有一行一行的ArrayIndexOutOfBoundsException。没有堆栈，不知道异常是从什么地方抛出来的，也就不能找到问题的根源，更谈不上解决。题外，工程师在用log4j记录错误异常的时候，我看到很多人这样用（假设e是异常对象）：<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000; ">log.error(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">发生错误:</span><span style="color: #000000; ">"</span><span style="color: #000000; ">+</span><span style="color: #000000; ">e);<br />
</span></div>
或者：<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000; ">log.error(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">发生错误:</span><span style="color: #000000; ">"</span><span style="color: #000000; ">+</span><span style="color: #000000; ">e.getMessage());</span></div>
&nbsp;&nbsp;&nbsp; 这样的写法是不对，只记录了异常的信息，而没有将堆栈输出到日志，正确的写法是利用error的重载方法：<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000; ">log.error(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">xxx发生错误</span><span style="color: #000000; ">"</span><span style="color: #000000; ">,e);</span></div>
<br />
&nbsp;&nbsp;&nbsp; 这样才能在日志中完整地输出异常堆栈来。如何写好日志是另一个话题，这里不展开。继续我们的找bug经历。刚才提到，我们线上日志一直出现一行错误信息ArrayIndexOutOfBoundsException却没有堆栈，是我们没有正确地写日志吗？检查代码不是的，这个问题其实是跟JDK5引入的一个新特性有关，对于一些频繁抛出的异常，JDK为了性能会做一个优化，在JIT重新编译后会抛出没有堆栈的异常。在使用server模式的时候，这个优化是开启的，我们的服务器跑在server模式下并且jdk版本是6，因此在频繁抛出ArrayIndexOutOfBoundsException异常一段时间后优化开始起作用，只抛出没有堆栈的异常信息了。<br />
<br />
&nbsp;&nbsp;&nbsp; 那么怎么解决这个问题呢？因为这个优化是在JIT重新编译后才起作用，因此一开始抛出异常的时候还是有堆栈的，所以可以查看较旧的日志，寻找有完整堆栈的异常信息。但是由于我们的日志太大，会定期删除，我们的服务器也启动了很长时间，因此查找日志不是很靠谱的方法。<br />
&nbsp;&nbsp;&nbsp; 另一个解决办法是暂时禁用这个优化，强制要求每次都要抛出有堆栈的异常。幸好JDK提供了选项来关闭这个优化，配置JVM参数<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000; ">-</span><span style="color: #000000; ">XX:</span><span style="color: #000000; ">-</span><span style="color: #000000; ">OmitStackTraceInFastThrow</span></div>
&nbsp;&nbsp;&nbsp; 就可以禁止这个优化（注意选项中的减号，加号是启用）。<br />
<br />
&nbsp;&nbsp;&nbsp; 我们找了台机器，配置了这个参数并重启一下。过了一会就找到问题所在，堆栈类似这样<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000; ">Caused&nbsp;by:&nbsp;java.lang.ArrayIndexOutOfBoundsException:&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">1831238</span><span style="color: #000000; "><br />
&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;sun.util.calendar.BaseCalendar.getCalendarDateFromFixedDate(BaseCalendar.java:</span><span style="color: #000000; ">436</span><span style="color: #000000;">)<br />
&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;java.util.GregorianCalendar.computeFields(GregorianCalendar.java:</span><span style="color: #000000; ">2081</span><span style="color: #000000; ">)<br />
&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;java.util.GregorianCalendar.computeFields(GregorianCalendar.java:</span><span style="color: #000000; ">1996</span><span style="color: #000000; ">)<br />
&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;java.util.Calendar.setTimeInMillis(Calendar.java:</span><span style="color: #000000; ">1109</span><span style="color: #000000; ">)<br />
&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;java.util.Calendar.setTime(Calendar.java:</span><span style="color: #000000; ">1075</span><span style="color: #000000; ">)<br />
&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;java.text.SimpleDateFormat.format(SimpleDateFormat.java:</span><span style="color: #000000; ">876</span><span style="color: #000000; ">)<br />
&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;java.text.SimpleDateFormat.format(SimpleDateFormat.java:</span><span style="color: #000000; ">869</span><span style="color: #000000; ">)<br />
&nbsp;&nbsp;&nbsp;&nbsp;at&nbsp;java.text.DateFormat.format(DateFormat.java:</span><span style="color: #000000; ">316</span><span style="color: #000000; ">)</span></div>
<br />
&nbsp;&nbsp;&nbsp; 读者肯定猜到了，这个问题是由于SimpleDateFormat的误用引起的。<a href="http://download.oracle.com/javase/6/docs/api/java/text/SimpleDateFormat.html">SimpleDateFormat</a>的<a href="http://download.oracle.com/javase/6/docs/api/java/text/SimpleDateFormat.html">javadoc</a>中有这么句话：<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000; ">Date&nbsp;formats&nbsp;are&nbsp;not&nbsp;</span><span style="color: #0000FF; ">synchronized</span><span style="color: #000000; ">.&nbsp;It&nbsp;is&nbsp;recommended&nbsp;to&nbsp;create&nbsp;separate&nbsp;format&nbsp;instances&nbsp;</span><span style="color: #0000FF; ">for</span><span style="color: #000000; ">&nbsp;each&nbsp;thread. <br />
If&nbsp;multiple&nbsp;threads&nbsp;access&nbsp;a&nbsp;format&nbsp;concurrently,&nbsp;it&nbsp;must&nbsp;be&nbsp;</span><span style="color: #0000FF; ">synchronized</span><span style="color: #000000; ">&nbsp;externally. <br />
</span></div>
&nbsp;&nbsp;&nbsp; 但是很悲剧的是这句话是放在整个doc的最后面，在我看来，这句话应该放在最前面才对。简单来说就是SimpleDateFormat不是线程安全的，你要么每次都new一个来用，要么做加锁来同步使用。<br />
<br />
&nbsp;&nbsp;&nbsp; 出问题的代码就是由于工程师认为SimpleDateFormat的创建代价很高，然后搞了个map做缓存，所有线程共用这个instance做format，同时没有做同步。悲剧就诞生了。<br />
&nbsp;&nbsp;&nbsp; 这里就引出我想提到的第二点问题，在使用一个类或者方法的时候<strong>，最好能详细地看下该类的javadoc，JDK的javadoc是做的非常好的</strong>，javadoc除了做说明之外，通常还会给示例，并且会点出一些关键问题，如线程安全性和平台移植性。<br />
<br />
&nbsp;&nbsp;&nbsp; 最后，我将做个测试，到底在使用SimpleDateFormat怎么做才是最好的方式？假设我们要实现一个formatDate方法将日期格式化成"yyyy-MM-dd"的格式。<br />
&nbsp;&nbsp;&nbsp; 第一个方法是每次使用都创建一个instance，并调用format方法：<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">static</span><span style="color: #000000; ">&nbsp;String&nbsp;formatDate1(Date&nbsp;date)&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SimpleDateFormat&nbsp;format&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;SimpleDateFormat(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">yyyy-MM-dd</span><span style="color: #000000; ">"</span><span style="color: #000000; ">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">return</span><span style="color: #000000; ">&nbsp;format.format(date);<br />
&nbsp;&nbsp;&nbsp;&nbsp;}<br />
</span></div>
<br />
&nbsp;&nbsp;&nbsp; 第二个方法是只创建一个instance，但是在调用方法的时候做同步：<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000; "> &nbsp;&nbsp; </span><span style="color: #0000FF; ">private</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">static</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">final</span><span style="color: #000000; ">&nbsp;SimpleDateFormat&nbsp;formatter&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;SimpleDateFormat(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">yyyy-MM-dd</span><span style="color: #000000; ">"</span><span style="color: #000000; ">);<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">static</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">synchronized</span><span style="color: #000000; ">&nbsp;String&nbsp;formatDate2(Date&nbsp;date)&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">return</span><span style="color: #000000; ">&nbsp;formatter.format(date);<br />
&nbsp;&nbsp;&nbsp;&nbsp;}</span></div>
<br />
&nbsp;&nbsp;&nbsp; 第三个方法比较特殊，我们为每个线程都缓存一个instance，存放在ThreadLocal里，使用的时候从ThreadLocal里取就可以了:<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">private</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">static</span><span style="color: #000000; ">&nbsp;ThreadLocal</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">SimpleDateFormat</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;formatCache&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;ThreadLocal</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">SimpleDateFormat</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">()&nbsp;{<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;@Override<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">protected</span><span style="color: #000000; ">&nbsp;SimpleDateFormat&nbsp;initialValue()&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">return</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">new</span><span style="color: #000000; ">&nbsp;SimpleDateFormat(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">yyyy-MM-dd</span><span style="color: #000000; ">"</span><span style="color: #000000; ">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;};<br />
<br />
&nbsp;&nbsp;&nbsp; </span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">static</span><span style="color: #000000; ">&nbsp;String&nbsp;formatDate3(Date&nbsp;date)&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SimpleDateFormat&nbsp;format&nbsp;</span><span style="color: #000000; ">=</span><span style="color: #000000; ">&nbsp;formatCache.get();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">return</span><span style="color: #000000; ">&nbsp;format.format(date);<br />
&nbsp;&nbsp;&nbsp;&nbsp;}</span></div>
<br />
&nbsp;&nbsp;&nbsp; 然后我们测试下三个方法并发调用下的性能并做一个比较，并发100个线程循环调用1000万次，记录耗时。我们设置了JVM参数：<br />
<div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000; ">-</span><span style="color: #000000; ">Xmx512m&nbsp;</span><span style="color: #000000; ">-</span><span style="color: #000000; ">XX:CompileThreshold</span><span style="color: #000000; ">=</span><span style="color: #000000; ">10000</span></div>
&nbsp;&nbsp;&nbsp; 设置堆最大为512M，设置当一个方法被调用1万次的时候就被JIT编译。测试的结果如下：<br />
<table style="border: 1px none; margin: 1px;" bgcolor="#C0C0C0" border="1" bordercolor="#000000" cellpadding="2" cellspacing="2" width="500">
     <tbody>
         <tr>
             <td>&nbsp; <br />
             </td>
             <td> 第1次测试<br />
             </td>
             <td> 第2次测试<br />
             </td>
             <td> 第3次测试<br />
             </td>
         </tr>
         <tr>
             <td> formatDate1<br />
             </td>
             <td> 50545<br />
             </td>
             <td> 49365<br />
             </td>
             <td> 53532<br />
             </td>
         </tr>
         <tr>
             <td> formatDate2<br />
             </td>
             <td> 10895<br />
             </td>
             <td> 10761<br />
             </td>
             <td> 10673<br />
             </td>
         </tr>
         <tr>
             <td>formatDate3</td>
             <td>10386</td>
             <td>9919</td>
             <td>9527</td>
         </tr>
     </tbody>
</table>(单位：毫秒）<br />&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp; 从结果来看，方法1最慢，方法3最快，但是就算是最慢的方法1也可以达到每秒钟<strike>200</strike> 20万次的调用量，很少有系统能达到这个量级。这个点很难成为你系统的瓶颈所在。从我的角度出发，我会建议你用方法1或者方法2，如果你追求那么一点性能提升的话，可以考虑用方法3，也就是用ThreadLocal做缓存。<br /><br />&nbsp;&nbsp;&nbsp; 总结下本文找bug经历想表达的几点想法：<br />（1）正确地打印错误日志<br />（2）在server模式下，最好都设置<span style="color: #000000; ">-</span><span style="color: #000000; ">XX:</span><span style="color: #000000; ">-</span><span style="color: #000000;">OmitStackTraceInFastThrow<br />（3）使用类或者方法的时候，最好能详细阅读下javadoc，很多问题都能找到答案<br />（4）使用SimpleDateFormat的时候要注意线程安全性，要么每次new，要么做同步，两者的性能有差距，但是这个差距很难成为你的性能瓶颈。<br /><br /></span>&nbsp;&nbsp;&nbsp; 下篇文章我再分享另一个bug的查找经历，也是比较有趣，可以看到一些工具的使用。<br /><span style="color: #000000; "><br /></span><img src ="http://www.blogjava.net/killme2008/aggbug/354062.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-07-10 23:29 <a href="http://www.blogjava.net/killme2008/archive/2011/07/10/354062.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Java NIO编程的技巧和陷阱</title><link>http://www.blogjava.net/killme2008/archive/2011/06/30/353422.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Thu, 30 Jun 2011 03:07:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/06/30/353422.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/353422.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/06/30/353422.html#Feedback</comments><slash:comments>4</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/353422.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/353422.html</trackback:ping><description><![CDATA[<br />
去年做的分享，一直上传slideshare失败，今天又试了下，成功了。这个主题主要介绍Java NIO编程的技巧和陷阱，解读了一些NIO框架的源码，以及编写高性能NIO网络框架所需要注意的技巧和缺陷。关注这方面的朋友可以看一下。去年写了篇blog提供了pdf版本的下载，看<a href="http://www.blogjava.net/killme2008/archive/2010/11/22/338420.html">这里</a>。<br />
<br />
<div style="width:500px" id="__ss_8464155"> <strong style="display:block;margin:12px 0 4px"><a href="http://www.slideshare.net/killme2008/nio-trick-and-trap-8464155" title="Nio trick and trap" target="_blank">Nio trick and trap</a></strong> <iframe src="http://www.slideshare.net/slideshow/embed_code/8464155" marginwidth="0" marginheight="0" frameborder="0" height="450" scrolling="no" width="500"></iframe>
<div style="padding:5px 0 12px"> View more <a href="http://www.slideshare.net/" target="_blank">presentations</a> from <a href="http://www.slideshare.net/killme2008" target="_blank">dennis zhuang</a> </div>
</div><img src ="http://www.blogjava.net/killme2008/aggbug/353422.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-06-30 11:07 <a href="http://www.blogjava.net/killme2008/archive/2011/06/30/353422.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>xmemcached发布1.3.3版本——支持touch和GAT</title><link>http://www.blogjava.net/killme2008/archive/2011/06/12/352120.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Sun, 12 Jun 2011 05:32:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/06/12/352120.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/352120.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/06/12/352120.html#Feedback</comments><slash:comments>2</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/352120.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/352120.html</trackback:ping><description><![CDATA[&nbsp;&nbsp;&nbsp; 开源memcached的java客户端<a href="http://code.google.com/p/xmemcached/">xmemcached</a>发布1.3.3，主要改进如下：<br /><br />1、memcached 1.6添加了不少新特性，具体可以参考《what's new in memcached》(<a href="http://groups.google.com/group/memcached/browse_thread/thread/10253a6996735f2">1</a>) (<a href="http://groups.google.com/group/memcached/browse_thread/thread/4041bead80c8f14c">2</a>)这两个帖子。xmemcached将及时跟进这些新特性。1.3.3这个版本<strong>实现了二进制协议中新的两个命令touch和GAT</strong>（get and touch)。这两个功能可以说是千呼万唤始出来，终于可以不用get-set来重新设置数据的超时时间，利用touch或者GAT可以简单地更新数据的超时时间。1.3.3新增加四个方法：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">boolean</span><span style="color: #000000; ">&nbsp;touch(</span><span style="color: #0000FF; ">final</span><span style="color: #000000; ">&nbsp;String&nbsp;key,&nbsp;</span><span style="color: #0000FF; ">int</span><span style="color: #000000; ">&nbsp;exp,&nbsp;</span><span style="color: #0000FF; ">long</span><span style="color: #000000; ">&nbsp;opTimeout)<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">throws</span><span style="color: #000000;">&nbsp;TimeoutException,&nbsp;InterruptedException,&nbsp;MemcachedException;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">boolean</span><span style="color: #000000; ">&nbsp;touch(</span><span style="color: #0000FF; ">final</span><span style="color: #000000; ">&nbsp;String&nbsp;key,&nbsp;</span><span style="color: #0000FF; ">int</span><span style="color: #000000; ">&nbsp;exp)&nbsp;</span><span style="color: #0000FF; ">throws</span><span style="color: #000000; ">&nbsp;TimeoutException,<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;InterruptedException,&nbsp;MemcachedException;<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">T</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;T&nbsp;getAndTouch(</span><span style="color: #0000FF; ">final</span><span style="color: #000000; ">&nbsp;String&nbsp;key,&nbsp;</span><span style="color: #0000FF; ">int</span><span style="color: #000000; ">&nbsp;newExp,&nbsp;</span><span style="color: #0000FF; ">long</span><span style="color: #000000; ">&nbsp;opTimeout)<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">throws</span><span style="color: #000000; ">&nbsp;TimeoutException,&nbsp;InterruptedException,&nbsp;MemcachedException;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">public</span><span style="color: #000000; ">&nbsp;</span><span style="color: #000000; ">&lt;</span><span style="color: #000000; ">T</span><span style="color: #000000; ">&gt;</span><span style="color: #000000; ">&nbsp;T&nbsp;getAndTouch(</span><span style="color: #0000FF; ">final</span><span style="color: #000000; ">&nbsp;String&nbsp;key,&nbsp;</span><span style="color: #0000FF; ">int</span><span style="color: #000000; ">&nbsp;newExp)<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">throws</span><span style="color: #000000;">&nbsp;TimeoutException,&nbsp;InterruptedException,&nbsp;MemcachedException;</span></div><br />其中touch用于设置数据新的超时时间，getAndTouch则是在获取数据的同时更新超时时间。例如用memcached存储session，可以在每次get的时候更新下数据的超时时间来保活。<strong>请注意，这四个方法仅在使用memcached 1.6并且使用二进制协议的时候有效</strong>。<br /><strong><br />2、setLoggingLevelVerbosity方法可以作用于二进制协议。</strong><br /><strong><br />3、重构错误处理模块，使得异常信息更友好。</strong><br /><br /><strong>4、将KeyIterator和getKeyIterator声明为deprecated，</strong>因为memached 1.6将移除stats cachedump协议，并且stats cachedump返回数据有大小限制，遍历功能不具实用性。<br /><br /><strong>5、修复Bug</strong>，包括<a title="Error getting memcached Detail Statistics" class="closed_ref" href="http://code.google.com/p/xmemcached/issues/detail?id=126">issue 126&nbsp;</a>,<a title="Unknow Response status:130" href="http://code.google.com/p/xmemcached/issues/detail?id=127">issue 127</a>,<a title="&quot;SERVER_ERROR out of memory storing object&quot; cause session disconnected" href="http://code.google.com/p/xmemcached/issues/detail?id=128">issue 128</a>,<a title="Memory leak after shutdown" href="http://code.google.com/p/xmemcached/issues/detail?id=129">issue 129</a>。<br /><br />下载地址：<a href="http://code.google.com/p/xmemcached/downloads/list">http://code.google.com/p/xmemcached/downloads/list</a><br />源码：&nbsp; <a href="https://github.com/killme2008/xmemcached">https://github.com/killme2008/xmemcached</a><br />maven引用：<br /><div style="background-color:#eeeeee;font-size:13px;border:1px solid #CCCCCC;padding-right: 5px;padding-bottom: 4px;padding-left: 4px;padding-top: 4px;width: 98%;word-break:break-all"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />http://www.CodeHighlighter.com/<br /><br />--><span style="color: #000000; ">&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">com.googlecode.xmemcached</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">groupId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">xmemcached</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">artifactId</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000FF; ">&lt;</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; ">1.3.3</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">version</span><span style="color: #0000FF; ">&gt;</span><span style="color: #000000; "><br />&nbsp;</span><span style="color: #0000FF; ">&lt;/</span><span style="color: #800000; ">dependency</span><span style="color: #0000FF; ">&gt;</span></div><img src ="http://www.blogjava.net/killme2008/aggbug/352120.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-06-12 13:32 <a href="http://www.blogjava.net/killme2008/archive/2011/06/12/352120.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>HandlerSocket client for java——hs4j正式发布0.1版本</title><link>http://www.blogjava.net/killme2008/archive/2011/03/29/347160.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Mon, 28 Mar 2011 22:55:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/03/29/347160.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/347160.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/03/29/347160.html#Feedback</comments><slash:comments>3</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/347160.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/347160.html</trackback:ping><description><![CDATA[<br />
<br />
&nbsp;&nbsp;&nbsp; <a href="https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL">HandlerSocket</a>是一个mysql插件，可以将mysql作为NoSQL来使用，具体可以看我过去写的<a href="http://www.blogjava.net/killme2008/default.html?page=2">这篇Blog</a>。<a href="http://code.google.com/p/hs4j/">hs4j</a>是HandlerSocket的一个java客户端，自认为它比日本人写的那个客户端更实用和易用一些。写完好久，经过不少朋友使用和测试，现在正式发一个0.1版本，并已同步到maven中心仓库。<br />
<br />
&nbsp;&nbsp;&nbsp; 项目主页：<a title="http://code.google.com/p/hs4j/" href="http://code.google.com/p/hs4j/">http://code.google.com/p/hs4j/</a><br />
&nbsp;&nbsp;&nbsp; 项目描述：hs4j is a practical java client for <a href="https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL" rel="nofollow">HandlerSocket</a>,it is nio based and turned to get better performance. <br />
&nbsp;&nbsp;&nbsp; 使用文档：<a href="http://code.google.com/p/hs4j/w/list">http://code.google.com/p/hs4j/w/list</a><br />
&nbsp;&nbsp;&nbsp; 下载地址：<a title="http://code.google.com/p/hs4j/downloads/list" href="http://code.google.com/p/hs4j/downloads/list">http://code.google.com/p/hs4j/downloads/list</a><br />
&nbsp;&nbsp;&nbsp; 源码仓库：<a href="https://github.com/killme2008/hs4j" rel="nofollow">https://github.com/killme2008/hs4j</a><br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp; 如果你使用maven2，可以直接引用：<br />
<div style="background-color: rgb(238, 238, 238); font-size: 13px; border: 1px solid rgb(204, 204, 204); padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: rgb(0, 0, 255);">&lt;</span><span style="color: rgb(128, 0, 0);">dependency</span><span style="color: rgb(0, 0, 255);">&gt;</span><span style="color: rgb(0, 0, 0);"><br />
&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 255);">&lt;</span><span style="color: rgb(128, 0, 0);">groupId</span><span style="color: rgb(0, 0, 255);">&gt;</span><span style="color: rgb(0, 0, 0);">com.googlecode.hs4j</span><span style="color: rgb(0, 0, 255);">&lt;/</span><span style="color: rgb(128, 0, 0);">groupId</span><span style="color: rgb(0, 0, 255);">&gt;</span><span style="color: rgb(0, 0, 0);"><br />
&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 255);">&lt;</span><span style="color: rgb(128, 0, 0);">artifactId</span><span style="color: rgb(0, 0, 255);">&gt;</span><span style="color: rgb(0, 0, 0);">hs4j</span><span style="color: rgb(0, 0, 255);">&lt;/</span><span style="color: rgb(128, 0, 0);">artifactId</span><span style="color: rgb(0, 0, 255);">&gt;</span><span style="color: rgb(0, 0, 0);"><br />
&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 255);">&lt;</span><span style="color: rgb(128, 0, 0);">version</span><span style="color: rgb(0, 0, 255);">&gt;</span><span style="color: rgb(0, 0, 0);">0.1</span><span style="color: rgb(0, 0, 255);">&lt;/</span><span style="color: rgb(128, 0, 0);">version</span><span style="color: rgb(0, 0, 255);">&gt;</span><span style="color: rgb(0, 0, 0);"><br />
</span><span style="color: rgb(0, 0, 255);">&lt;/</span><span style="color: rgb(128, 0, 0);">dependency</span><span style="color: rgb(0, 0, 255);">&gt;</span></div>
<br />
&nbsp;&nbsp;&nbsp;&nbsp; 有疑问和bug请联系我。<br /><img src ="http://www.blogjava.net/killme2008/aggbug/347160.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-03-29 06:55 <a href="http://www.blogjava.net/killme2008/archive/2011/03/29/347160.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>xmemcached发布1.3.2版本</title><link>http://www.blogjava.net/killme2008/archive/2011/03/27/347102.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Sun, 27 Mar 2011 06:06:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/03/27/347102.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/347102.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/03/27/347102.html#Feedback</comments><slash:comments>1</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/347102.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/347102.html</trackback:ping><description><![CDATA[<p><a href="http://code.google.com/p/xmemcached">&nbsp;&nbsp;&nbsp;&nbsp; Xmemcached</a>是一个开源的java memcached client，具有高性能、更易用、功能完善等优点，距离上次发布1.3.1已经超过两个月，现在正式发布1.3.2这个新版本，主要的改进如下：</p>
<br />
<strong>1、Bug修复</strong>，从1.3.1版本以来发现的bug并修复，包括：
<p><a href="http://code.google.com/p/xmemcached/issues/detail?id=112">issue 112</a>:: 新引入的failure模式在启动的时候，如果memcached故障，运行不符合预期的bug.</p>
<p><a href="http://code.google.com/p/xmemcached/issues/detail?id=113">issue 113</a>: 新增加一个delete方法，可以设置操作超时</p>
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">boolean</span><span style="color: #000000;">&nbsp;delete(</span><span style="color: #0000ff;">final</span><span style="color: #000000;">&nbsp;String&nbsp;key,&nbsp;</span><span style="color: #0000ff;">long</span><span style="color: #000000;">&nbsp;opTimeout)<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">throws</span><span style="color: #000000;">&nbsp;TimeoutException,&nbsp;InterruptedException,&nbsp;MemcachedException;<br />
<br />
</span></div>
<p><strong>2、性能调优</strong>，存储操作(set/add/replace/prepend/append/cas)的性能提升5%。</p>
<p><strong>3、修复pom.xml</strong>，使得xmemcached可以在其他机器上编译。</p>
<p><strong>4、使用github作为源码仓库</strong>，版本管理使用git替换svn，源码转移到</p>
<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <a href="https://github.com/killme2008/xmemcached">https://github.com/killme2008/xmemcached</a></p>
<p>新版本下载地址：</p>
<a href="http://code.google.com/p/xmemcached/downloads/list">&nbsp;&nbsp;&nbsp; http://code.google.com/p/xmemcached/downloads/list</a>
<p>使用maven可以直接引用：&nbsp;
</p>
<span style="color: #0000ff;">&lt;</span><span style="color: #800000;">dependency</span><span style="color: #0000ff;">&gt;</span><br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">&lt;</span><span style="color: #800000;">groupId</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;">com.googlecode.xmemcached</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">groupId</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;"><br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">&lt;</span><span style="color: #800000;">artifactId</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;">xmemcached</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">artifactId</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;"><br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">&lt;</span><span style="color: #800000;">version</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;">1.3.2</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">version</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;"><br />
&nbsp;</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">dependency</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;"><br />
<br />
</span></div>
<p>项目文档：</p>
<p><a href="http://code.google.com/p/xmemcached/w/list">http://code.google.com/p/xmemcached/w/list</a></p><img src ="http://www.blogjava.net/killme2008/aggbug/347102.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-03-27 14:06 <a href="http://www.blogjava.net/killme2008/archive/2011/03/27/347102.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Xmemcached 1.3.0正式发布</title><link>http://www.blogjava.net/killme2008/archive/2011/01/04/342287.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Tue, 04 Jan 2011 12:10:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2011/01/04/342287.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/342287.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2011/01/04/342287.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/342287.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/342287.html</trackback:ping><description><![CDATA[&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://code.google.com/p/xmemcached/">Xmemcached</a>是一个开源的memcached的Java客户端，最近引入了一些关键特性，因此版本号直接从1.2.6.2升级到1.3.0。主要的更改如下：<br />
<p>
<strong>1、引入了failure模式</strong>，所谓failure模式是指在当一个memcached由于各种原因不可用的情况下，发往这个节点的请求将直接抛出异常，而非使用下一个可用的节点。具体可以看memached的<a href="http://code.google.com/p/memcached/wiki/NewConfiguringClient#Failure,_or_Failover">这个文档</a>。默认不启用，启用failure模式很简单：</p>
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">MemcachedClientBuilder&nbsp;builder</span><span style="color: #000000;">=</span><span style="color: #000000;">&#8230;&#8230;<br />
</span><span style="color: #008000;">//</span><span style="color: #008000;">启用failure模式。</span><span style="color: #008000;"><br />
</span><span style="color: #000000;">builder.setFailureMode(</span><span style="color: #0000ff;">true</span><span style="color: #000000;">);<br />
</span></div>
<p>也可以采用spring配置。<br />
</p>
<strong>2、在启用failure模式的情况下，允许为每个memcached设置一个备份节点，</strong>当主节点挂掉的情况下，会将请求转交给备份节点，主节点恢复后又自动切换到主节点。请注意，要设置备份节点的前提是启用failure模式。假设我们已经有两个memcached节点：host1:port和host2:port，为host1:port设置一个备份节点host3:port可以实现为：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">MemcachedClientBuilder&nbsp;builder</span><span style="color: #000000;">=</span><span style="color: #0000ff;">new</span><span style="color: #000000;">&nbsp;XmemcachedClientBuilder(AddrUtil.getAddressMap(</span><span style="color: #000000;">"</span><span style="color: #000000;">host1:port,host3:port&nbsp;host2:port</span><span style="color: #000000;">"</span><span style="color: #000000;">))<br />
&#8230;&#8230;</span></div>
<br />
主备节点之间用逗号隔开，不同分组之间用空格隔开，完全兼容1.2。并且当备份节点连接意外断开的情况下，xmemcached也会自动修复备份节点的连接并加入映射。<br />
<p>
关于failure模式和standby节点更多内容可以参考<a href="http://www.blogjava.net/killme2008/archive/2010/12/28/341731.html">这篇blog</a>.<br />
</p>
<p>
<strong>3、修正BUG和新功能</strong>，包括<a href="http://code.google.com/p/xmemcached/issues/detail?id=104">issue 104</a>,<a href="http://code.google.com/p/xmemcached/issues/detail?id=105">issue 105</a>,<a href="http://code.google.com/p/xmemcached/issues/detail?id=107">issue 107</a>等。<br />
</p>
<p>
项目主页 <a href="http://code.google.com/p/xmemcached/" title="http://code.google.com/p/xmemcached/">http://code.google.com/p/xmemcached/</a><br />
</p>
<p>
下载地址 <a href="http://code.google.com/p/xmemcached/downloads/list" title="http://code.google.com/p/xmemcached/downloads/list">http://code.google.com/p/xmemcached/downloads/list</a><br />
</p>
<p>
用户指南 <a href="http://code.google.com/p/xmemcached/wiki/TableOfContents" title="http://code.google.com/p/xmemcached/wiki/TableOfContents">http://code.google.com/p/xmemcached/wiki/TableOfContents</a><br />
</p>
<p>&nbsp;&nbsp;&nbsp;&nbsp;
如果你使用maven构建，可以直接引用：<br />
</p>
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #0000ff;">&lt;</span><span style="color: #800000;">dependency</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;"><br />
</span><span style="color: #0000ff;">&lt;</span><span style="color: #800000;">groupId</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;">com.googlecode.xmemcached</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">groupId</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;"><br />
</span><span style="color: #0000ff;">&lt;</span><span style="color: #800000;">artifactId</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;">xmemcached</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">artifactId</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;"><br />
</span><span style="color: #0000ff;">&lt;</span><span style="color: #800000;">version</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;">1.3.1</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">version</span><span style="color: #0000ff;">&gt;</span><span style="color: #000000;"><br />
</span><span style="color: #0000ff;">&lt;/</span><span style="color: #800000;">dependency</span><span style="color: #0000ff;">&gt;</span></div>
<p><br />
</p>
<p>&nbsp;&nbsp;&nbsp; 更新：发布1.3.1了，如果你还在使用1.3.0，建议升级。1.3.0因为改变了memcached地址服务器顺序，可能导致原有的缓存失效。<br />
</p>
<br /><img src ="http://www.blogjava.net/killme2008/aggbug/342287.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2011-01-04 20:10 <a href="http://www.blogjava.net/killme2008/archive/2011/01/04/342287.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Xmemcached 1.3预览：failure模式和standby节点</title><link>http://www.blogjava.net/killme2008/archive/2010/12/28/341731.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Tue, 28 Dec 2010 02:47:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2010/12/28/341731.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/341731.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2010/12/28/341731.html#Feedback</comments><slash:comments>5</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/341731.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/341731.html</trackback:ping><description><![CDATA[<br />
&nbsp;&nbsp;&nbsp; <a href="http://code.google.com/p/xmemcached/">Xmemcached</a>在元旦左右准备发1.3这个版本，这个版本新增加的一个关键特性就是所谓的failure模式。关于这个，可以看下memcached官方文档的解释<br />
<a title=" 《Failure,or Failover》" href="http://code.google.com/p/memcached/wiki/NewConfiguringClient#Failure,_or_Failover"> 《Failure,or Failover》</a>。<br />
<br />
&nbsp;&nbsp;&nbsp; 展开来说，在某个memcached节点挂掉或者由于其他故障连接断开的时候，大部分客户端的默认策略都是failover的，也就是会查找下一个可用的memcached节点继续使用，挂掉或者连接不上的节点的数据会转移到其他节点上，路由的策略可以是Round Robin，也可以是一致性哈希。这样的模式在节点意外故障挂掉的情况下运行的很好，但是memached节点也完全可能因为一个意外的事故而<strong>短暂</strong>挂掉，比如你不小心弄掉了网线又马上接上去，比如机房交换机突然停电又立即恢复了，假设在故障前，用户A正要更新数据到节点A，节点A意外断开，那么这些数据就更新到下一个有效节点B，但是节点A又马上恢复，这时候用户又从节点A去读数据，读到却是更新前的老数据了（新数据更新到B节点去了），这种情况对用户来说就非常困惑，你告诉我更新成功，但是看到却还是更新前的数据。<br />
<br />
&nbsp;&nbsp; 怎么解决呢？一个简单的方案就是所谓failure模式，当某个节点挂掉的时候，不会从节点列表移除，请求也不会转移到下一个有效节点，而是直接将请求置为失败，就刚才的场景来说，在用户更新数据到节点A的时候，节点A意外断开，那么用户的这次更新请求不会转移到节点B，而是直接告诉用户更新失败，用户再次查询数据则绕过节点A直接查询后端存储。这种模式很适合这种节点短暂不可用的状况，请求会穿透缓存到后端，但是避免了新旧数据的问题。<br />
&nbsp;&nbsp;&nbsp; Xmemcached 1.3将支持failure模式，只要你设置下failureMode为true即可，简单示例：<br />
<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; XMemcachedClientBuilder&nbsp;builder&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&#8230;&#8230;<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #008000;">//</span><span style="color: #008000;">设置使用failure模式</span><span style="color: #008000;"><br />
</span><span style="color: #000000;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;builder.setFailureMode(</span><span style="color: #0000ff;">true</span><span style="color: #000000;">);</span></div>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 在此模式下，某个节点挂掉的情况下，往这个节点的请求都将直接抛出MemcachedException的异常。<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 不仅如此，xmemcached 1.3还将引入standby node的概念，你可以设置某个memached节点的备份节点，当这个节点挂掉的时候会将请求转发给这个备份节点，不会简单地抛出异常，也不会转发给其他节点。要使用standby node，必须首先设置使用failure mode，一个例子：<br />
<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">XMemcachedClientBuilder&nbsp;builder&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">new</span><span style="color: #000000;">&nbsp;XMemcachedClientBuilder(AddrUtil<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.getAddressMap(</span><span style="color: #000000;">"</span><span style="color: #000000;">192.168.1.99:11211,192.168.1.100:11211&nbsp;192.168.1.101:11211,192.168.1.102:11211</span><span style="color: #000000;">"</span><span style="color: #000000;">));<br />
builder.setFailureMode(</span><span style="color: #0000ff;">true</span><span style="color: #000000;">);</span></div>
<br />
&nbsp;&nbsp;&nbsp;&nbsp; 可以看到，新的服务器字符串格式变化为host:port,host:port host:port,host:port的格式，以空格隔开的是两个节点组成的一个分组，以逗号隔开的是主节点和备份节点，以上面的例子来说，我们设置客户端使用的节点是192.168.1.99和192.168.1.101，其中99对应的备份节点是100，而101的备份节点是102。并且我们需要设置使用failure mode为true。<br />
&nbsp;&nbsp;&nbsp; <br />
&nbsp;&nbsp;&nbsp;&nbsp; Failure mode加上standby节点可以比较好的解决新旧数据的问题，并且也可以防止请求穿透缓存到DB，但是主备两个节点之间的数据同步，xmemcached不准备帮你做，我的建议是可以使用repcached这个patch做复制。<br />
&nbsp;&nbsp;&nbsp; 有的朋友可能希望，在使用备份节点之前先flush掉备份节点的数据，防止使用到老的数据，请求还是可以穿透缓存去DB查找，并存储到备份节点，我仔细考虑了这个方案，衡量之下还是不准备做自动flush，主要是并发上很难处理，并且flush数据这个事情可以手工来搞，根据我的经验，做的太透明太自动不一定是好事。你可以在主节点恢复之后，手工flush下备份节点的数据。<br />
<br />
<br />
&nbsp;&nbsp;&nbsp; 目前，xmemcached 1.3已经整装待发，对这些特性有兴趣的朋友可以先从<a href="http://code.google.com/p/xmemcached/source/checkout">svn下载源</a>码尝鲜，有任何改进的建议请发邮件给我。我的邮件地址在博客的右上角。<br />
<br />
<br />
<br />
<br />
<br />
&nbsp;&nbsp; <br /><img src ="http://www.blogjava.net/killme2008/aggbug/341731.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2010-12-28 10:47 <a href="http://www.blogjava.net/killme2008/archive/2010/12/28/341731.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>HandlerSocket client for java——MySql as NoSQL</title><link>http://www.blogjava.net/killme2008/archive/2010/11/30/339386.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Tue, 30 Nov 2010 05:51:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2010/11/30/339386.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/339386.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2010/11/30/339386.html#Feedback</comments><slash:comments>4</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/339386.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/339386.html</trackback:ping><description><![CDATA[<br />
&nbsp;&nbsp;&nbsp; <a href="https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL">HandlerSocket</a>是日本人 akira higuchi 写的一个MySql的插件，通过这个插件，你可以直接跟MySql后端的存储引擎做key-value式的交互，省去了MySql上层的SQL解释、打开关闭表、创建查询计划等CPU消耗型的开销，按照作者给出的数据可以在数据全部在内存的情况下可以达到75W的QPS查询。具体信息可以看这篇<a href="http://yoshinorimatsunobu.blogspot.com/2010/10/using-mysql-as-nosql-story-for.html?showComment=1287715880531#c2105763073739952882">Blog</a>，中文介绍可以看这篇文章《<a href="http://whitesock.javaeye.com/blog/811339">HandlerSocket in action</a>》。<br />
<br />
&nbsp;&nbsp;&nbsp; 这个东西为什么让我很激动呢？首先性能是程序员的G点，一听高性能你不由地激动，其次，这也解决了缓存跟数据库的一致性问题，因为缓存就在数据库里面，第三，这个东西不仅仅是NoSQL，简单的CRUD你可以通过HandlerSocket，但是复杂的查询你仍然可以走MySql，完全符合我们应用的场景，并且从实际测试来看，性能确实非常优秀。但是呢，这个东西的代价也少不了，例如没有权限检查（未来可能添加）；不能启用MySql的查询缓存，否则会导致数据的不一致；<strike>协议设计也不合理，使用\t做分隔符，使用\n做换行符，那么你插入或者更新的字段数据就不能含有这些字符，否则行为将不如预期。</strike><br />
<br />
&nbsp;&nbsp; <a href="https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL">HandlerSocket</a>有一个日本人的java客户端实现，我去尝试了下，结果发现这玩意完全不具实用性，封装的层次非常原始。因此我自己写了个新的客户端，这就是本文要介绍的HandlerSocket Client for Java，简称<a href="http://code.google.com/p/hs4j/">hs4j</a>，项目放在了<a href="http://code.google.com/p/hs4j/">googlecode</a>，代码的网络层复用xmemcached，重新实现了协议和上层接口，目前的状态完全可用，也希望有需要的朋友参与测试。<br />
<br />
&nbsp;&nbsp; 项目地址：<a href="http://code.google.com/p/hs4j/">http://code.google.com/p/hs4j/</a><br />
<br />
&nbsp;&nbsp;&nbsp; HS4J的使用很简单，所有的操作都通过HSClient这个接口进行，如我们创建一个客户端对象：<br />
<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #0000ff;">import</span><span style="color: #000000;">&nbsp;com.google.code.hs4j.HSClient;<br />
</span><span style="color: #0000ff;">import</span><span style="color: #000000;">&nbsp;com.google.code.hs4j.impl.HSClientImpl;<br />
<br />
&nbsp;&nbsp;&nbsp;HSClient&nbsp;hsClient&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">new</span><span style="color: #000000;">&nbsp;HSClientImpl(</span><span style="color: #0000ff;">new</span><span style="color: #000000;">&nbsp;InetSocketAddress(</span><span style="color: #000000;">9999</span><span style="color: #000000;">));</span></div>
<br />
&nbsp;&nbsp; 假设HandlerSocket运行在本地的9999端口，默认的9998是只读的，9999才是允许读和写。HSClient是线程安全的。<br />
<br />
&nbsp;&nbsp; 在执行操作前需要先open index：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #0000ff;">import</span><span style="color: #000000;">&nbsp;com.google.code.hs4j.IndexSession;<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IndexSession&nbsp;session&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;hsClient.openIndexSession(db,&nbsp;table,<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">PRIMARY</span><span style="color: #000000;">"</span><span style="color: #000000;">,&nbsp;columns);</span></div>
<br />
&nbsp;&nbsp; 其中db是数据库名,table是表名，"PRIMARY"表示使用主键索引，columns是一个字符串数组代表你要查询的字段名称。这里没有指定indexid，默认会产生一个indexid，你也可以指定indexid，返回表示一次open-index会话对象，IndexSession同样是线程安全的。<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">IndexSession&nbsp;session&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;hsClient.openIndexSession(indexid,db,&nbsp;table,<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">PRIMARY</span><span style="color: #000000;">"</span><span style="color: #000000;">,&nbsp;columns);</span></div>
<br />
&nbsp;&nbsp; 查询操作通过find方法:<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #0000ff;">import</span><span style="color: #000000;">&nbsp;java.sql.ResultSet;<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">final</span><span style="color: #000000;">&nbsp;String[]&nbsp;keys&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;{&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">dennis</span><span style="color: #000000;">"</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">killme2008@gmail.com</span><span style="color: #000000;">"</span><span style="color: #000000;">&nbsp;};<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ResultSet&nbsp;rs&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;session.find(keys);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">while</span><span style="color: #000000;">(rs.next()){<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;String&nbsp;name</span><span style="color: #000000;">=</span><span style="color: #000000;">rs.getString(</span><span style="color: #000000;">1</span><span style="color: #000000;">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;String&nbsp;mail</span><span style="color: #000000;">=</span><span style="color: #000000;">rs.getString(</span><span style="color: #000000;">2</span><span style="color: #000000;">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}</span></div>
<br />
&nbsp;&nbsp; find返回的是java.sql.ResultSet，你完全可以像使用jdbc那样去操作结果集。当然我的简单实现并不符合JDBC规范，只实现了最常见的一些方法，如getStrng、getLong等。find(keys)方法默认使用的op是"="。其他重载方法可以设置其他类型的op，统一封装为枚举类型FindOperator。<br />
<br />
&nbsp;&nbsp; 更新操作：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #0000ff;">import</span><span style="color: #000000;">&nbsp;com.google.code.hs4j.FindOperator;<br />
<br />
&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">int</span><span style="color: #000000;">&nbsp;result</span><span style="color: #000000;">=</span><span style="color: #000000;">session.update(keys,&nbsp;</span><span style="color: #0000ff;">new</span><span style="color: #000000;">&nbsp;String[]&nbsp;{&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">1</span><span style="color: #000000;">"</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">dennis</span><span style="color: #000000;">"</span><span style="color: #000000;">,<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">test@163.com</span><span style="color: #000000;">"</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">109</span><span style="color: #000000;">"</span><span style="color: #000000;">&nbsp;},&nbsp;FindOperator.EQ);</span></div>
<br />
&nbsp;&nbsp; keys表示索引的字段列表对应的值数组，通过FindOperator.EQ比较这些值和索引，第二个参数values表示要更新的字段值，这些值跟你在open-index的时候传入的columns一一对应，最后返回作用的记录数。<br />
<br />
&nbsp;&nbsp;&nbsp; 删除操作：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">int</span><span style="color: #000000;">&nbsp;result</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;session.delete(</span><span style="color: #0000ff;">new</span><span style="color: #000000;">&nbsp;String[]&nbsp;{&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">dennis</span><span style="color: #000000;">"</span><span style="color: #000000;">&nbsp;},<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;FindOperator.EQ)</span></div>
<br />
&nbsp;&nbsp;&nbsp; HS4J同样支持连接池，可以在构建客户端的时候传入连接池大小：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">&nbsp;&nbsp;</span><span style="color: #008000;">//</span><span style="color: #008000;">100-connections&nbsp;pool</span><span style="color: #008000;"><br />
</span><span style="color: #000000;">&nbsp;&nbsp;&nbsp;HSClient&nbsp;hsClient&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">new</span><span style="color: #000000;">&nbsp;HSClientImpl(</span><span style="color: #0000ff;">new</span><span style="color: #000000;">&nbsp;InetSocketAddress(</span><span style="color: #000000;">9999</span><span style="color: #000000;">),</span><span style="color: #000000;">100</span><span style="color: #000000;">);</span></div>
<br />
&nbsp;&nbsp; 在open index的时候，会在连接池里所有的连接上都open。并且在连接因为意外情况（如网络错误）断开的时候，HS4J会自动重连，并在重连成功的情况下自动发送已经open的index，保证应用的操作不受重连影响。<br />
<br />
&nbsp;&nbsp;&nbsp; 因为HS4J是我在两天内写就的一个东西，可能还有不少隐藏的bug，并且HandlerSocket本身也是个新东西，如果有什么问题或者改进建议，随时欢迎告诉我，多谢。<br />
<br />
<br />
&nbsp;&nbsp; <br />
&nbsp;&nbsp; <br />
<br /><img src ="http://www.blogjava.net/killme2008/aggbug/339386.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2010-11-30 13:51 <a href="http://www.blogjava.net/killme2008/archive/2010/11/30/339386.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Java NIO trick and trap</title><link>http://www.blogjava.net/killme2008/archive/2010/11/22/338420.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Mon, 22 Nov 2010 10:22:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2010/11/22/338420.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/338420.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2010/11/22/338420.html#Feedback</comments><slash:comments>16</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/338420.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/338420.html</trackback:ping><description><![CDATA[<br />
&nbsp;&nbsp;&nbsp;&nbsp; 上周在内部做的一个Java NIO框架的实现技巧和陷阱的分享，对编写NIO网络框架有兴趣的朋友可能有点帮助，上传slideshare.net一直出错，直接提供下载吧。<br />
&nbsp;&nbsp;&nbsp;&nbsp; <br />
&nbsp;&nbsp;&nbsp;&nbsp; 下载地址：<a href="http://www.blogjava.net/Files/killme2008/NIO%20trick%20and%20trap.pdf.zip">Nio Trick and Trap.pdf.zip</a><br />
<br />
<img alt="" src="http://www.blogjava.net/images/blogjava_net/killme2008/nio.png" height="576" width="808" /><br />
<br />
<br />
<br /><img src ="http://www.blogjava.net/killme2008/aggbug/338420.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2010-11-22 18:22 <a href="http://www.blogjava.net/killme2008/archive/2010/11/22/338420.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>对kilim nio模块的改进</title><link>http://www.blogjava.net/killme2008/archive/2010/11/19/338504.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Fri, 19 Nov 2010 10:40:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2010/11/19/338504.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/338504.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2010/11/19/338504.html#Feedback</comments><slash:comments>3</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/338504.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/338504.html</trackback:ping><description><![CDATA[<br />
&nbsp;&nbsp;&nbsp; 前段时间对kilim的当前版本做了一些改进，集中在nio调度器这一块。Kilim新版本引入了nio调度器，可以跟非阻塞IO结合在一起，从这个版本开始，kilim才真正具有实用性。协程只有跟非阻塞IO结合起来才能发挥威力啊。但是Kilim的默认的nio调度器还只是使用一个nio worker做调度，这跟现有的NIO框架采用多个nio worker来提升效率比较起来相对落伍。我改进了<a href="https://github.com/killme2008/kilim/blob/master/src/kilim/nio/NioSelectorScheduler.java">NioSelectorScheduler</a>，引入了类似Netty3的boss和woker的概念，boss负责连接接入，而worker负责连接的IO读写，并且默认设置worker数目为CPU个数的两倍。经过我的测试，改进后的NIO调度器的效率远远超过了现有的调度器，有兴趣可以用netty的benchmark跑一下example里的<a title="EchoServer" href="https://github.com/killme2008/kilim/blob/master/examples/kilim/examples/EchoServer.java">EchoServer</a>。<br />
<br />
&nbsp;&nbsp;&nbsp; Kilim默认还提供了一个简易Http Server框架，但是没有提供Http Client的实现，我的另一个改进是提供了一个简易的Http Client实现，也是利用Ragel做协议解析，一个简单的使用例子如下：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #0000ff;">package</span><span style="color: #000000;">&nbsp;kilim.examples;<br />
<br />
</span><span style="color: #0000ff;">import</span><span style="color: #000000;">&nbsp;kilim.Pausable;<br />
</span><span style="color: #0000ff;">import</span><span style="color: #000000;">&nbsp;kilim.Task;<br />
</span><span style="color: #0000ff;">import</span><span style="color: #000000;">&nbsp;kilim.http.HttpClient;<br />
</span><span style="color: #0000ff;">import</span><span style="color: #000000;">&nbsp;kilim.http.HttpResponse;<br />
<br />
<br />
</span><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">class</span><span style="color: #000000;">&nbsp;SimpleHttpClient&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">static</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">class</span><span style="color: #000000;">&nbsp;SimpleTask&nbsp;</span><span style="color: #0000ff;">extends</span><span style="color: #000000;">&nbsp;Task&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;@Override<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">void</span><span style="color: #000000;">&nbsp;execute()&nbsp;</span><span style="color: #0000ff;">throws</span><span style="color: #000000;">&nbsp;Pausable,&nbsp;Exception&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;HttpClient&nbsp;client&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">new</span><span style="color: #000000;">&nbsp;HttpClient();<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;HttpResponse&nbsp;resp&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;client.get(</span><span style="color: #000000;">"</span><span style="color: #000000;">http://www.google.com.hk/</span><span style="color: #000000;">"</span><span style="color: #000000;">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(resp.status());<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(resp.content());<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br />
&nbsp;&nbsp;&nbsp;&nbsp;}<br />
<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">static</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">void</span><span style="color: #000000;">&nbsp;main(String[]&nbsp;args)&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SimpleTask&nbsp;task&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">new</span><span style="color: #000000;">&nbsp;SimpleTask();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;task.start();<br />
&nbsp;&nbsp;&nbsp;&nbsp;}<br />
<br />
}<br />
</span></div>
<br />
<br />
&nbsp;&nbsp;&nbsp; 这个简陋的HttpClient目前只支持GET/POST，同时支持Http chunk编码（得益于kilim原有代码），做一些简单的HTTP调用已经足够。我尝试在一个项目里使用这个HttpClient去替代java默认的HttpURLConnection，效率有部分提升，但是同时由于大量协程存在占用了很大部分的内存，给GC也带来了不小的压力。<br />
<br />
&nbsp;&nbsp;&nbsp; 我的代码直接从kilim的主干fork出来，有兴趣可以直接git clone下来玩玩,地址&nbsp;<a title="https://github.com/killme2008/kilim" href="https://github.com/killme2008/kilim"> https://github.com/killme2008/kilim</a><br /><img src ="http://www.blogjava.net/killme2008/aggbug/338504.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2010-11-19 18:40 <a href="http://www.blogjava.net/killme2008/archive/2010/11/19/338504.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Kilim的小BUG</title><link>http://www.blogjava.net/killme2008/archive/2010/11/03/337179.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Wed, 03 Nov 2010 15:22:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2010/11/03/337179.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/337179.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2010/11/03/337179.html#Feedback</comments><slash:comments>2</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/337179.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/337179.html</trackback:ping><description><![CDATA[<br />
&nbsp;&nbsp;&nbsp; 我最近在实现一个基于Kilim的HttpClient，在处理响应body特别大的情形下遇到了kilim的一个BUG，有必要记录下。<br />
&nbsp;&nbsp;&nbsp; 问题是这样，Kilim将连接封装为EndPoint对象，EndPoint有个方法fill用于从管道读数据到缓冲区，并且可以指定希望至少读到多少个字节（atLeastN)才返回。那么在进入此方法的时候会判断缓冲区是否有足够空间容纳atLeastN个字节，如果没有，则创建一个更大的缓冲区，并将&#8220;老&#8221;的缓冲区的数据拷贝到新缓冲区，这部分代码是这样实现：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;ByteBuffer&nbsp;fill(ByteBuffer&nbsp;buf,&nbsp;</span><span style="color: #0000ff;">int</span><span style="color: #000000;">&nbsp;atleastN)&nbsp;</span><span style="color: #0000ff;">throws</span><span style="color: #000000;">&nbsp;IOException,&nbsp;Pausable&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">if</span><span style="color: #000000;">&nbsp;(buf.remaining()&nbsp;</span><span style="color: #000000;">&lt;</span><span style="color: #000000;">&nbsp;atleastN)&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ByteBuffer&nbsp;newbb&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;ByteBuffer.allocate(Math.max(buf.capacity()&nbsp;</span><span style="color: #000000;">*</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">3</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">/</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">2</span><span style="color: #000000;">,&nbsp;buf.position()&nbsp;</span><span style="color: #000000;">+</span><span style="color: #000000;">&nbsp;atleastN));<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;buf.rewind();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;newbb.put(buf);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;buf&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;newbb;<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&#8230;&#8230;<br />
}</span></div>
<br />
&nbsp;&nbsp;&nbsp; 后面的代码我省略了，这个BUG就出现在这段代码里。这段代码的逻辑很简单，先是创建一个新的更大的缓冲区，然后将老的缓冲区的数据put到新的缓冲区，在put之前调用rewind方法将老的缓冲区的position设置为0。查看rewind干了什么：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">final</span><span style="color: #000000;">&nbsp;Buffer&nbsp;rewind()&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;position&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">0</span><span style="color: #000000;">;<br />
&nbsp;&nbsp;&nbsp;&nbsp;mark&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">-</span><span style="color: #000000;">1</span><span style="color: #000000;">;<br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">return</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">this</span><span style="color: #000000;">;<br />
&nbsp;&nbsp;&nbsp;&nbsp;}</span></div>
<br />
&nbsp;&nbsp;&nbsp;
仅仅是将position设置为0，并让mark失效。position指向下一个读或者写的位置，这里在写入到新缓冲区之前确实需要将position设置为0，以便写入从老的缓冲区第一个位置开始。问题是什么？问题是position仅仅指定了下一个读取数据的位置，却没有指定有效数据的大小，换句话说，没有指定老的缓冲区的limit。因此这里造成的后果是老的缓冲区整个被写入到新的老缓冲区，包括有效数据和无效数据，默认情况下缓冲区的limit等于capacity。<br />
<br />
&nbsp;&nbsp; 这个bug可以通过下面程序看出来：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ByteBuffer&nbsp;old&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;ByteBuffer.allocate(</span><span style="color: #000000;">8</span><span style="color: #000000;">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;old.putInt(</span><span style="color: #000000;">99</span><span style="color: #000000;">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ByteBuffer&nbsp;newBuf&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;ByteBuffer.allocate(</span><span style="color: #000000;">16</span><span style="color: #000000;">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;old.rewind();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;newBuf.put(old);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;newBuf.putInt(</span><span style="color: #000000;">100</span><span style="color: #000000;">);<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;newBuf.flip();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(newBuf.remaining());<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(newBuf.getInt());<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(newBuf.getInt());<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(newBuf.getInt());</span></div>
<br />
&nbsp;&nbsp;&nbsp; 先往old写入一个整数99，然后创建newBuf并写入old数据，并再写入一个整数100，最后从newBuf读数据。本来我们预期只应该读到两个整数99和100，但是中间却插入一个0,输出如下：<br />
<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">12</span><span style="color: #000000;"><br />
</span><span style="color: #000000;">99</span><span style="color: #000000;"><br />
</span><span style="color: #000000;">0</span><span style="color: #000000;"><br />
</span><span style="color: #000000;">100</span></div>
<br />
&nbsp;&nbsp;&nbsp; 12表示缓冲区可读的数据，本来应该是8个字节，却多了4个字节的无效数据。<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp; 这个BUG解决很简单，将rewind修改为flip方法即可，flip不仅将position设置为0，也将limit设置为当前位置：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">&nbsp;&nbsp;</span><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">final</span><span style="color: #000000;">&nbsp;Buffer&nbsp;flip()&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;limit&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;position;<br />
&nbsp;&nbsp;&nbsp;&nbsp;position&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">0</span><span style="color: #000000;">;<br />
&nbsp;&nbsp;&nbsp;&nbsp;mark&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">-</span><span style="color: #000000;">1</span><span style="color: #000000;">;<br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">return</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">this</span><span style="color: #000000;">;<br />
&nbsp;&nbsp;&nbsp;&nbsp;}</span></div>
<br />
<br />
&nbsp;&nbsp;&nbsp; 修改上面的测试程序，符合我们的预期了：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">&nbsp; &nbsp; &nbsp; &nbsp; ByteBuffer&nbsp;old&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;ByteBuffer.allocate(</span><span style="color: #000000;">8</span><span style="color: #000000;">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;old.putInt(</span><span style="color: #000000;">99</span><span style="color: #000000;">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ByteBuffer&nbsp;newBuf&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;ByteBuffer.allocate(</span><span style="color: #000000;">16</span><span style="color: #000000;">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;old.flip();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;newBuf.put(old);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;newBuf.putInt(</span><span style="color: #000000;">100</span><span style="color: #000000;">);<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;newBuf.flip();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(newBuf.remaining());<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(newBuf.getInt());<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(newBuf.getInt());;</span></div>
<br />
&nbsp;&nbsp;&nbsp; 输出：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #000000;">8</span><span style="color: #000000;"><br />
</span><span style="color: #000000;">99</span><span style="color: #000000;"><br />
</span><span style="color: #000000;">100</span></div>
<br />
&nbsp;&nbsp;&nbsp; 总结，使用rewind的前提是limit已经正确设置，例如你将buffer写入成功并想记录这个buffer，可以使用rewind:<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><!--<br />
<br />
Code highlighting produced by Actipro CodeHighlighter (freeware)<br />
http://www.CodeHighlighter.com/<br />
<br />
--><span style="color: #0000ff;">while</span><span style="color: #000000;">&nbsp;(buffer.hasRemaining())&nbsp;</span><span style="color: #008000;">//</span><span style="color: #008000;">发送数据</span><span style="color: #008000;"><br />
</span><span style="color: #000000;">&nbsp; &nbsp; networkChannel.write(buffer);<br />
buffer.rewind();&nbsp;</span><span style="color: #008000;">//</span><span style="color: #008000;">&nbsp;重置buffer，准备写入日志管道</span><span style="color: #008000;"><br />
</span><span style="color: #0000ff;">while</span><span style="color: #000000;">&nbsp;(buffer.hasRemaining())&nbsp;</span><span style="color: #008000;">//</span><span style="color: #008000;">&nbsp;写入日志</span><span style="color: #008000;"><br />
</span><span style="color: #000000;">&nbsp;&nbsp;&nbsp; loggerChannel.write(buffer);<br />
</span></div>
<br />
&nbsp;&nbsp; 而flip用于缓冲区发送或者读取之前，也就是将缓冲区设置为等待传出状态。<br /><img src ="http://www.blogjava.net/killme2008/aggbug/337179.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2010-11-03 23:22 <a href="http://www.blogjava.net/killme2008/archive/2010/11/03/337179.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>xmemcached 1.2.6.2紧急发布（升级到1.2.6.1的朋友注意）</title><link>http://www.blogjava.net/killme2008/archive/2010/10/22/335897.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Fri, 22 Oct 2010 07:52:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2010/10/22/335897.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/335897.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2010/10/22/335897.html#Feedback</comments><slash:comments>4</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/335897.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/335897.html</trackback:ping><description><![CDATA[<br />
&nbsp;&nbsp;&nbsp; 今年在阅读某个项目源码的时候看到DelayQueue的使用，<a href="http://code.google.com/p/xmemcached/">xmemcached</a> 1.2.6.1的重连任务也是采用DelayQueue管理，ReconnectRequest实现Delayed接口，我突然想起去review下xmc的源码，发现一个严重的BUG，原始代码如下：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">final</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">class</span><span style="color: #000000;">&nbsp;ReconnectRequest&nbsp;</span><span style="color: #0000ff;">implements</span><span style="color: #000000;">&nbsp;Delayed&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">long</span><span style="color: #000000;">&nbsp;getDelay(TimeUnit&nbsp;unit)&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">return</span><span style="color: #000000;">&nbsp;&nbsp;nextReconnectTimestamp&nbsp;</span><span style="color: #000000;">-</span><span style="color: #000000;">&nbsp;System.currentTimeMillis();<br />
&nbsp;&nbsp;&nbsp;&nbsp;}<br />
}</span></div>
<br />
&nbsp;&nbsp;&nbsp;
getDelay返回该任务还剩下多少时间可以被执行，将下次执行的时间戳减去当前时间即可，问题在于这里返回的是毫秒，而没有调用getDelay传入的TimeUnit做转换，在DelayQueue内部其实是用纳秒做单位交给Condition对象去等待<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;">&nbsp;&nbsp;</span><span style="color: #0000ff;">for</span><span style="color: #000000;">&nbsp;(;;)&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;E&nbsp;first&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;q.peek();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">if</span><span style="color: #000000;">&nbsp;(first&nbsp;</span><span style="color: #000000;">==</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">null</span><span style="color: #000000;">)&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;available.await();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp;</span><span style="color: #0000ff;">else</span><span style="color: #000000;">&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">long</span><span style="color: #000000;">&nbsp;delay&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;&nbsp;first.getDelay(TimeUnit.NANOSECONDS);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">if</span><span style="color: #000000;">&nbsp;(delay&nbsp;</span><span style="color: #000000;">&gt;</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">0</span><span style="color: #000000;">)&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">long</span><span style="color: #000000;">&nbsp;tl&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;available.awaitNanos(delay);<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp;</span><span style="color: #0000ff;">else</span><span style="color: #000000;">&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;E&nbsp;x&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;q.poll();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">assert</span><span style="color: #000000;">&nbsp;x&nbsp;</span><span style="color: #000000;">!=</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">null</span><span style="color: #000000;">;<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">if</span><span style="color: #000000;">&nbsp;(q.size()&nbsp;</span><span style="color: #000000;">!=</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">0</span><span style="color: #000000;">)<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;available.signalAll();&nbsp;</span><span style="color: #008000;">//</span><span style="color: #008000;">&nbsp;wake&nbsp;up&nbsp;other&nbsp;takers</span><span style="color: #008000;"><br />
</span><span style="color: #000000;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">return</span><span style="color: #000000;">&nbsp;x;<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}</span></div>
&nbsp;&nbsp; <br />
&nbsp;&nbsp;&nbsp;&nbsp; 最终导致的问题是，<span style="color: #000000;">awaitNanos很快返回（</span><span style="color: #000000;">awaitNanos接受的是纳秒，这里却传入毫秒）</span><span style="color: #000000;">，循环执行发现重新计算的delay仍然大于0，循环等到getDelay返回的越来越小直到0才执行相应的Task,，造成的现象是在重连的时候cpu占用率很高。<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp; 单元测试的时候没有发现这个问题，主要是因为功能正常，没有关注资源消耗情况，因此惭愧地忽略了。<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 解决的办法很简单，修改getDelay方法即可：<br />
</span>
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;">&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">long</span><span style="color: #000000;">&nbsp;getDelay(TimeUnit&nbsp;unit)&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">return</span><span style="color: #000000;">&nbsp;unit.convert(<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nextReconnectTimestamp&nbsp;</span><span style="color: #000000;">-</span><span style="color: #000000;">&nbsp;System.currentTimeMillis(),<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeUnit.MILLISECONDS);<br />
&nbsp;&nbsp;&nbsp;&nbsp;}<br />
</span></div>
<br />
<span style="color: #000000;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 这个BUG比较严重，已经升级1.2.6.1的朋友建议马上升级到1.2.6.2，使用maven的朋友只要修改版本即可，没有使用maven的请到<a href="http://code.google.com/p/xmemcached/downloads/list">这里</a>下载。<br />
<br />
</span><img src ="http://www.blogjava.net/killme2008/aggbug/335897.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2010-10-22 15:52 <a href="http://www.blogjava.net/killme2008/archive/2010/10/22/335897.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Selector.wakeup实现注记</title><link>http://www.blogjava.net/killme2008/archive/2010/10/22/335861.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Fri, 22 Oct 2010 02:35:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2010/10/22/335861.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/335861.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2010/10/22/335861.html#Feedback</comments><slash:comments>3</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/335861.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/335861.html</trackback:ping><description><![CDATA[<br />
&nbsp;&nbsp;&nbsp; NIO中的Selector封装了底层的系统调用，其中wakeup用于唤醒阻塞在select方法上的线程，它的实现很简单，在linux上就是创建一个管道并加入poll的fd集合，wakeup就是往管道里写一个字节，那么阻塞的poll方法有数据可读就立即返回。证明这一点很简单，strace即可知道：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">class</span><span style="color: #000000;">&nbsp;SelectorTest&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">static</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">void</span><span style="color: #000000;">&nbsp;main(String[]&nbsp;args)&nbsp;</span><span style="color: #0000ff;">throws</span><span style="color: #000000;">&nbsp;Exception&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Selector&nbsp;selector&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;Selector.open();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selector.wakeup();<br />
&nbsp;&nbsp;&nbsp;&nbsp;}<br />
}</span></div>
<br />
&nbsp;&nbsp;&nbsp;&nbsp;
使用strace调用,只关心write的系统调用<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;">sudo&nbsp;strace&nbsp;</span><span style="color: #000000;">-</span><span style="color: #000000;">f&nbsp;</span><span style="color: #000000;">-</span><span style="color: #000000;">e&nbsp;write&nbsp;java SelectorTest<br />
</span></div>
<br />
&nbsp;&nbsp;&nbsp;&nbsp; 输出：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;">Process&nbsp;</span><span style="color: #000000;">29181</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29182</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29183</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29184</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29185</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29186</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29187</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29188</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29189</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29190</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29191</span><span style="color: #000000;">&nbsp;attached<br />
[pid&nbsp;</span><span style="color: #000000;">29181</span><span style="color: #000000;">]&nbsp;write(</span><span style="color: #000000;">36</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">\1</span><span style="color: #000000;">"</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;">)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;"><br />
Process&nbsp;</span><span style="color: #000000;">29191</span><span style="color: #000000;">&nbsp;detached<br />
Process&nbsp;</span><span style="color: #000000;">29184</span><span style="color: #000000;">&nbsp;detached<br />
Process&nbsp;</span><span style="color: #000000;">29181</span><span style="color: #000000;">&nbsp;detached<br />
</span></div>
<br />
&nbsp;&nbsp;&nbsp; 有的同学说了，怎么证明这个write是wakeup方法调用的，而不是其他方法呢，这个很好证明，我们多调用几次：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">class</span><span style="color: #000000;">&nbsp;SelectorTest&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">static</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">void</span><span style="color: #000000;">&nbsp;main(String[]&nbsp;args)&nbsp;</span><span style="color: #0000ff;">throws</span><span style="color: #000000;">&nbsp;Exception&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Selector&nbsp;selector&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;Selector.open();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selector.wakeup();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selector.selectNow();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selector.wakeup();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selector.selectNow();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selector.wakeup();<br />
&nbsp;&nbsp;&nbsp;&nbsp;}<br />
}</span></div>
<br />
&nbsp;&nbsp;&nbsp; 修改程序调用三次wakeup，心细的朋友肯定注意到我们还调用了两次selectNow，这是因为在两次成功的select方法之间调用wakeup多次都只算做一次，为了显示3次write，这里就每次调用前select一下将前一次写入的字节读到，同样执行上面的strace调用，输出：<br />
<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;">Process&nbsp;</span><span style="color: #000000;">29303</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29304</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29305</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29306</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29307</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29308</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29309</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29310</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29311</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29312</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29313</span><span style="color: #000000;">&nbsp;attached<br />
[pid&nbsp;</span><span style="color: #000000;">29303</span><span style="color: #000000;">]&nbsp;write(</span><span style="color: #000000;">36</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">\1</span><span style="color: #000000;">"</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;">)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;"><br />
[pid&nbsp;</span><span style="color: #000000;">29303</span><span style="color: #000000;">]&nbsp;write(</span><span style="color: #000000;">36</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">\1</span><span style="color: #000000;">"</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;">)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;"><br />
[pid&nbsp;</span><span style="color: #000000;">29303</span><span style="color: #000000;">]&nbsp;write(</span><span style="color: #000000;">36</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">\1</span><span style="color: #000000;">"</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;">)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;"><br />
Process&nbsp;</span><span style="color: #000000;">29313</span><span style="color: #000000;">&nbsp;detached<br />
Process&nbsp;</span><span style="color: #000000;">29309</span><span style="color: #000000;">&nbsp;detached<br />
Process&nbsp;</span><span style="color: #000000;">29306</span><span style="color: #000000;">&nbsp;detached<br />
Process&nbsp;</span><span style="color: #000000;">29303</span><span style="color: #000000;">&nbsp;detached<br />
</span></div>
<br />
&nbsp;&nbsp;&nbsp;&nbsp; 果然是3次write的系统调用，都是写入一个字节，如果我们去掉selectNow，那么三次wakeup还是等于一次：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">class</span><span style="color: #000000;">&nbsp;SelectorTest&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">static</span><span style="color: #000000;">&nbsp;</span><span style="color: #0000ff;">void</span><span style="color: #000000;">&nbsp;main(String[]&nbsp;args)&nbsp;</span><span style="color: #0000ff;">throws</span><span style="color: #000000;">&nbsp;Exception&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Selector&nbsp;selector&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;Selector.open();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selector.wakeup();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selector.wakeup();<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selector.wakeup();<br />
&nbsp;&nbsp;&nbsp;&nbsp;}<br />
}<br />
</span></div>
&nbsp; <br />
&nbsp;&nbsp; 输出：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;">Process&nbsp;</span><span style="color: #000000;">29331</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29332</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29333</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29334</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29335</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29336</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29337</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29338</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29339</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29340</span><span style="color: #000000;">&nbsp;attached<br />
Process&nbsp;</span><span style="color: #000000;">29341</span><span style="color: #000000;">&nbsp;attached<br />
[pid&nbsp;</span><span style="color: #000000;">29331</span><span style="color: #000000;">]&nbsp;write(</span><span style="color: #000000;">36</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">"</span><span style="color: #000000;">\1</span><span style="color: #000000;">"</span><span style="color: #000000;">,&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;">)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;"><br />
Process&nbsp;</span><span style="color: #000000;">29341</span><span style="color: #000000;">&nbsp;detached<br />
Process&nbsp;</span><span style="color: #000000;">29337</span><span style="color: #000000;">&nbsp;detached<br />
Process&nbsp;</span><span style="color: #000000;">29334</span><span style="color: #000000;">&nbsp;detached<br />
Process&nbsp;</span><span style="color: #000000;">29331</span><span style="color: #000000;">&nbsp;detached<br />
</span></div>
<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; wakeup方法的API说明没有欺骗我们。wakeup方法的API还告诉我们，如果当前Selector没有阻塞在select方法上，那么本次wakeup调用会在下一次select阻塞的时候生效，这个道理很简单，wakeup方法写入一个字节，下次poll等待的时候立即发现可读并返回，因此不会阻塞。<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp; 具体到源码级别，在linux平台上的wakeup方法其实调用了pipe创建了管道，wakeup调用了<span id="cls"><a title="EPollArrayWrapper" href="http://www.docjar.com/docs/api/sun/nio/ch/EPollArrayWrapper.html">EPollArrayWrapper</a>的</span>interrupt方法：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #0000ff;">public</span><span style="color: #000000;">&nbsp;&nbsp;</span><span style="color: #0000ff;">void</span><span style="color: #000000;">&nbsp;interrupt()&nbsp;<br />
<br />
{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;interrupt(outgoingInterruptFD);<br />
}</span></div>
<br />
&nbsp;&nbsp;&nbsp; 实际调用的是interrupt(fd)的native方法，查看EPollArrayWrapper.c可见清晰的write系统调用：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;"><br />
JNIEXPORT&nbsp;</span><span style="color: #0000ff;">void</span><span style="color: #000000;">&nbsp;JNICALL<br />
Java_sun_nio_ch_EPollArrayWrapper_interrupt(JNIEnv&nbsp;</span><span style="color: #000000;">*</span><span style="color: #000000;">env,&nbsp;jobject&nbsp;</span><span style="color: #0000ff;">this</span><span style="color: #000000;">,&nbsp;jint&nbsp;fd)<br />
{<br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">int</span><span style="color: #000000;">&nbsp;fakebuf[</span><span style="color: #000000;">1</span><span style="color: #000000;">];<br />
&nbsp;&nbsp;&nbsp;&nbsp;fakebuf[</span><span style="color: #000000;">0</span><span style="color: #000000;">]&nbsp;</span><span style="color: #000000;">=</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;">;<br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: #0000ff;">if</span><span style="color: #000000;">&nbsp;(write(fd,&nbsp;fakebuf,&nbsp;</span><span style="color: #000000;">1</span><span style="color: #000000;">)&nbsp;</span><span style="color: #000000;">&lt;</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">0</span><span style="color: #000000;">)&nbsp;{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;JNU_ThrowIOExceptionWithLastError(env,</span><span style="color: #000000;">"</span><span style="color: #000000;">write&nbsp;to&nbsp;interrupt&nbsp;fd&nbsp;failed</span><span style="color: #000000;">"</span><span style="color: #000000;">);<br />
&nbsp;&nbsp;&nbsp;&nbsp;}<br />
}</span></div>
&nbsp;&nbsp;&nbsp; 写入一个字节的fakebuf。有朋友问起这个问题，写个注记在此。strace充分利用对了解这些细节很有帮助。<br />
&nbsp; <br />
<br /><img src ="http://www.blogjava.net/killme2008/aggbug/335861.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2010-10-22 10:35 <a href="http://www.blogjava.net/killme2008/archive/2010/10/22/335861.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Xmemcached发布1.2.6.1（推荐升级)</title><link>http://www.blogjava.net/killme2008/archive/2010/10/17/335356.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Sun, 17 Oct 2010 06:06:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2010/10/17/335356.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/335356.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2010/10/17/335356.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/335356.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/335356.html</trackback:ping><description><![CDATA[<br />
&nbsp;&nbsp;&nbsp;&nbsp; <a href="http://code.google.com/p/xmemcached/">Xmemcached</a> 1.2.6.1正式发布，这个版本的主要是做bug fix以及一些细节改进，主要变动如下：<br />
<br />
1、修复BUG，包括：<br />
Issue <a href="http://code.google.com/p/xmemcached/issues/detail?id=85">85</a>:
<span class="h3">当存在多个MemcachedClient的时候，JMX的统计只显示其中一个<br />
</span>Issue <a href="http://code.google.com/p/xmemcached/issues/detail?id=87">87</a>: 当使用一致性哈希的时候，连接池不起作用<br />
Issue <a href="http://code.google.com/p/xmemcached/issues/detail?id=90">90</a>:
用户线程<span class="h3">中断引起的连接关闭</span><br />
Issue <a href="http://code.google.com/p/xmemcached/issues/detail?id=94">94</a>:<span class="h3">BinaryMemcachedClientUnitTest测试失败<br />
</span>Issue <a href="http://code.google.com/p/xmemcached/issues/detail?id=95">95</a>:
<span class="h3">JMX addServer,removeServer存在缺陷<br />
</span>Issue <a href="http://code.google.com/p/xmemcached/issues/detail?id=96">96</a>:
<span class="h3">OOM Error while decompressing 60 KB of actuall data<br />
</span>
Issue <a href="http://code.google.com/p/xmemcached/issues/detail?id=97">97</a>:
<span class="h3">使得关闭连接更友好<br />
<br />
2、改进重连机制，重连不再是以固定间隔（默认2秒）做重试连接，而是以一个等差数列递增间隔时间，第一次2秒，第二次4秒，第三次6秒&#8230;&#8230;直到最大间隔时间1分钟做重连尝试。<br />
<br />
3、改善关闭机制，关闭连接前发送quit命令，尽量做到友好关闭，等待服务器主动断开连接。<br />
<br />
4、添加新的方法用于设置XmemcachedClient实例名称，用于区分不同的缓存集群，方便统计显示：<br />
</span>
<div style="background-color: rgb(238, 238, 238); font-size: 13px; border: 1px solid rgb(204, 204, 204); padding: 4px 5px 4px 4px; width: 98%;"><span style="color: rgb(0, 0, 255);">public</span><span style="color: rgb(0, 0, 0);">&nbsp;</span><span style="color: rgb(0, 0, 255);">interface</span><span style="color: rgb(0, 0, 0);">&nbsp;MemcachedClient{<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 128, 0);">/**</span><span style="color: rgb(0, 128, 0);"><br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;*&nbsp;Return&nbsp;the&nbsp;cache&nbsp;instance&nbsp;name<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;*&nbsp;<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;*&nbsp;</span><span style="color: rgb(128, 128, 128);">@return</span><span style="color: rgb(0, 128, 0);"><br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 128, 0);">*/</span><span style="color: rgb(0, 0, 0);"><br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 255);">public</span><span style="color: rgb(0, 0, 0);">&nbsp;String&nbsp;getName();<br />
<br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 128, 0);">/**</span><span style="color: rgb(0, 128, 0);"><br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;*&nbsp;Set&nbsp;cache&nbsp;instance&nbsp;name<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;*&nbsp;<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;*&nbsp;</span><span style="color: rgb(128, 128, 128);">@param</span><span style="color: rgb(0, 128, 0);">&nbsp;name<br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 128, 0);">*/</span><span style="color: rgb(0, 0, 0);"><br />
&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 255);">public</span><span style="color: rgb(0, 0, 0);">&nbsp;</span><span style="color: rgb(0, 0, 255);">void</span><span style="color: rgb(0, 0, 0);">&nbsp;setName(String&nbsp;name);<br />
}</span></div>
<br />
名称也可通过spring配置。<br />
<br />
5、提供<a href="http://code.google.com/p/xmemcached/wiki/User_Guide">英文的用户指南</a>，非常感谢<a href="http://code.google.com/u/cnscud/">cnscud</a>的帮助，这份文档是他一人搞定的。<br />
<br />
6、更新了<a href="http://xmemcached.googlecode.com/svn/trunk/benchmark/benchmark.html">Java Memcached Client Benchmark</a>，跟最新版本的<a href="http://code.google.com/p/spymemcached">spymemcached</a>,<a href="http://github.com/gwhalin/Memcached-Java-Client">Java-Memcached-Client</a>做对比测试，提供给需要的朋友参考。<br />
<br />
<br />
项目首页&nbsp; <a title="http://code.google.com/p/xmemcached/" href="http://code.google.com/p/xmemcached/">http://code.google.com/p/xmemcached/</a><br />
下载地址&nbsp; <a title="http://code.google.com/p/xmemcached/downloads/list" href="http://code.google.com/p/xmemcached/downloads/list">http://code.google.com/p/xmemcached/downloads/list</a><br />
wiki地址&nbsp;&nbsp; <a href="http://code.google.com/p/xmemcached/w/list">http://code.google.com/p/xmemcached/w/list</a><br />
<br />
<br />
<br />
<br /><img src ="http://www.blogjava.net/killme2008/aggbug/335356.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2010-10-17 14:06 <a href="http://www.blogjava.net/killme2008/archive/2010/10/17/335356.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Java Memcached Client Benchmark</title><link>http://www.blogjava.net/killme2008/archive/2010/10/17/335351.html</link><dc:creator>dennis</dc:creator><author>dennis</author><pubDate>Sun, 17 Oct 2010 05:44:00 GMT</pubDate><guid>http://www.blogjava.net/killme2008/archive/2010/10/17/335351.html</guid><wfw:comment>http://www.blogjava.net/killme2008/comments/335351.html</wfw:comment><comments>http://www.blogjava.net/killme2008/archive/2010/10/17/335351.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.blogjava.net/killme2008/comments/commentRss/335351.html</wfw:commentRss><trackback:ping>http://www.blogjava.net/killme2008/services/trackbacks/335351.html</trackback:ping><description><![CDATA[<br />
&nbsp;&nbsp;&nbsp;&nbsp; Xmemcached 1.2.6.1 released，所以更新了一下<a href="http://xmemcached.googlecode.com/svn/trunk/benchmark/benchmark.html">Java Memcached Client Benchmark</a>。对比下<a href="http://code.google.com/p/xmemcached/">Xmemached</a>,<a href="http://code.google.com/p/spymemcached">Spymemcached</a>和<a href="http://github.com/gwhalin/Memcached-Java-Client">Java-Memcached-Client</a>这三个开源客户端的性能，具体的测试信息可以看这个<a href="http://xmemcached.googlecode.com/svn/trunk/benchmark/benchmark.html">链接</a>。<br />
<br />
&nbsp;&nbsp;&nbsp; 测试源码<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;">svn&nbsp;co&nbsp;http:</span><span style="color: #008000;">//</span><span style="color: #008000;">xmemcached.googlecode.com/svn/trunk/benchmark/</span></div>
&nbsp;&nbsp;&nbsp; 测试结果：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;">svn&nbsp;co&nbsp;http:</span><span style="color: #008000;">//</span><span style="color: #008000;">xmemcached.googlecode.com/svn/trunk/benchmark/result</span></div>
<br />
&nbsp;&nbsp;&nbsp; 总结下测试结果，为还在选择和考察<a target="">java memcached client</a>的朋友提供参考：<br />
<br />
1、<a href="http://github.com/gwhalin/Memcached-Java-Client">Java-Memcached-Client</a> 2.5.1这个版本果然有很大改进，性能上有非常大的提升，从测试结果来看在小于100并发下有非常明显的优势，同时耗费资源也相对较多。但是在300并发访问下，Java-Memcached-Client会不断地报错：<br />
<div style="background-color: #eeeeee; font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%;"><span style="color: #000000;">com.schooner.MemCached.SchoonerSockIOPool&nbsp;Sun&nbsp;Oct&nbsp;</span><span style="color: #000000;">17</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">11</span><span style="color: #000000;">:</span><span style="color: #000000;">09</span><span style="color: #000000;">:</span><span style="color: #000000;">05</span><span style="color: #000000;">&nbsp;GMT</span><span style="color: #000000;">+</span><span style="color: #000000;">08</span><span style="color: #000000;">:</span><span style="color: #000000;">00</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">2010</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">-</span><span style="color: #000000;">&nbsp;</span><span style="color: #000000;">++++</span><span style="color: #000000;">&nbsp;failed&nbsp;to&nbsp;get&nbsp;SockIO&nbsp;obj&nbsp;</span><span style="color: #0000ff;">for</span><span style="color: #000000;">:&nbsp;</span><span style="color: #000000;">10.232</span><span style="color: #000000;">.</span><span style="color: #000000;">36.82</span><span style="color: #000000;">:</span><span style="color: #000000;">12000<br />
com.schooner.MemCached.SchoonerSockIOPool Sun Oct 17 11:09:05 GMT+08:00 2010 - ++++ failed to get SockIO obj for: 10.232.36.82:12000<br />
com.schooner.MemCached.SchoonerSockIOPool Sun Oct 17 11:09:05 GMT+08:00 2010 - ++++ failed to get SockIO obj for: 10.232.36.90:12000<br />
com.schooner.MemCached.SchoonerSockIOPool Sun Oct 17 11:09:05 GMT+08:00 2010 - ++++ failed to get SockIO obj for: 10.232.36.82:12000<br />
com.schooner.MemCached.SchoonerSockIOPool Sun Oct 17 11:09:05 GMT+08:00 2010 - ++++ failed to get SockIO obj for: 10.232.36.90:12000<br />
com.schooner.MemCached.SchoonerSockIOPool Sun Oct 17 11:09:05 GMT+08:00 2010 - ++++ failed to get SockIO obj for: 10.232.36.82:12000</span><br />
</div>
<br />
&nbsp;并且无法正常地存取数据， 而xmc和spy却可以正常应对这一场景。因此可以看到在300并发下，Java-Memcached-Client测试的结果直接为0，因为测试无法完成。尽管我尝试将最大连接数调到2000，仍然是无法完成测试。<br />
<br />
2、<a href="http://code.google.com/p/xmemcached/">Xmemcached</a>无论在低并发还是高并发访问的情况下，都可以保持一个比较优秀的性能表现，从xmc和spy的对比来看，xmc的优势相当大。<br />
<br />
3、<strong>从用户选择角度来说，如果你的应用对memached的访问负载并不高，Java-Memcached-Client是一个不错的选择，但是在高峰访问的时候可能命中率会有个急剧的波动；如果你的应用访问memached的负载较高，此时我推荐你选择<a href="http://code.google.com/p/xmemcached/">xmemcached</a>；如果你需要异步的批量处理(future模式），可以选择<a href="http://code.google.com/p/spymemcached">spymemcached</a>；如果你不知道你的应用是什么状况，我推荐你用<a href="http://code.google.com/p/xmemcached/">xmemcached</a>，可以在任何情况下获得一个比较好的性能表现</strong>。<br />
<br />
<br /><img src ="http://www.blogjava.net/killme2008/aggbug/335351.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.blogjava.net/killme2008/" target="_blank">dennis</a> 2010-10-17 13:44 <a href="http://www.blogjava.net/killme2008/archive/2010/10/17/335351.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item></channel></rss>