随笔 - 53, 文章 - 0, 评论 - 3, 引用 - 0


java.sql.BatchUpdateException: IO Error: Connection reset

during analysis of "IO Error: Connection reset", many articles mentioned that it could be caused by java security code (accessing /dev/random) used in JDBC connection. However it is not the root cause in my case. In my environment, Java already use /dev/urandom. 1. $JAVA_HOME/jre/lib/security/java.security securerandom.source=file:/dev/./urandom 2. check with strace. only -Djava.security.egd=file:/dev/../dev/urandom will trigger system call (read on /dev/urandom) all other other path format like below are OK. -Djava.security.egd=file:/dev/./urandom -Djava.security.egd=file:///dev/urandom 3. Keep checking the retropy size, I have never seen it is exhaused. while [ 1 ]; do cat /proc/sys/kernel/random/entropy_avail sleep 1 done usually the avail is in the range from 1000 to 3000. so far, there is no clue about the root cause of "IO Error: Connection reset".

posted @ 2017-02-20 09:28 InPractice 阅读(1047) | 评论 (0)编辑 收藏

Lessons learned - Oracle GI and Database Installation on SUSE 12

I encountered many issue during installation of Oracle Grid Infrastructure(GI) and Database; with the help of ariticle and documents found through Google search engine, I finally made it. for records, here is the details issues encountered and solutions applied. Major issues were encountered during GI installation. Pre-installation tasks. Issue 1: swapspace is not big enough; (1.3.1 Verify System Requirements) grep MemTotal /proc/meminfo 264G grep SwapTotal /proc/meminfo 2G during OS installation, I take default option and swap space is only 2G. Oracle recommend to have more than 16G swap space in case of more that 32G RAM. dd if=/dev/zero of=/home/swapfile bs=1024 count=33554432 33554432+0 records in 33554432+0 records out 34359738368 bytes (34 GB) copied mkswap /home/swapfile mkswap /home/swapfile chmod 0600 /home/swapfile lessons learned: setup swap space properly according to DB requirement when installing OS. Issue 2: cannot find oracleasm-kmp-default from Oracle site. (1.3.6 Prepare Storage for Oracle Automatic Storage Management) install oracleasmlib and oracleasm-support is easy, just download them from Oracle and install them; Originally oracleasm kernel is provided by Oracle, but now I cannot find it from Oracle; finally I realized that oracleasm kernel is now provided by OS vendor; In my case, it should be installed from SUSE disk; a. to get its name oracleasm-kmp-default zypper se oracle b. map dvd and install zypper in oracleasm-kmp-default rpm -qa|grep oracleasm oracleasm-kmp-default-2.0.8_k3.12.49_11-3.20.x86_64 oracleasm-support-2.1.8-1.SLE12.x86_64 oracleasmlib-2.0.12-1.SLE12.x86_64 asm configure -i asm createdisk DATA /dev/<...> asm listdisks --DATA ls /dev/oracleasm/disks Installation tasks: Issue 3: always failed due to user equivalence check after starting installer OUI with user oracle. however if I manully check with runcluvfy, no issue found at all. ./runcluvfy.sh stage -pre crsinst -n , -verbose I worked around it by using another user to replace user oracle. but it triggered next issue. Issue 4: cannot see ASM disks in OUI. no matter how I change the disk dicovery path. the disk list is empty. but I can find disk manully. /usr/sbin/oracleasm-discover 'ORCL:*' Discovered disk: ORCL:DATA Root cause is that the ASM is configured and created with user oracle. and I aming installing GI with different user other than oracle; so I cannot see the Disk created. change owner of disk device file solved the issue. ls /dev/oracleasm/disks chown /dev/oracleasm/disks -R Issue 5: root.sh execution failed. Failed to create keys in the OLR, rc = 127, Message: clscfg.bin: error while loading shared libraries: libcap.so.1: cannot open shared object file: No such file or directory fixed the issue with command below: zypper in libcap1 ohasd failed to start Failed to start the Clusterware. Last 20 lines of the alert log follow: 2016-07-24 23:10:28.502: [client(1119)]CRS-2101:The OLR was formatted using version 3. I found a good document from SUSE, Oracle RAC on SUSE Linux Enterprise Server 12 - x86_64, it make it clear that SUSE 12 is supported by Oracle GI, it also mentioned Patch 18370031. "During the Oracle Grid Infrastructure installation, you must apply patch 18370031 before configuring the software that is installed. " The patch 18370031 is actually mentioned in "Oracle quick installation guide on Linux", but not mentioned in "Oracle quick installation guide on Linux". I majored followed up with later one and missed Patch 18370031. issue disappeared after I installed the patch 18370031. ./OPatch/opatch napply -oh -local /18370031 Errors in file : ORA-27091: unable to queue I/O ORA-15081: failed to submit an I/O operation to a disk ORA-06512: at line 4 solved by change owner of disk DATA related file ls -l /dev/oracleasm/iid chown on folder /dev/oracleasm/iid and some .* hidden file. Issue during DB installation Issue 6: report error: in invoking target 'agent nmhs' vi $ORACLE_HOME/sysman/lib/ins_emagent.mk Search for the line $(MK_EMAGENT_NMECTL) Change it to: $(MK_EMAGENT_NMECTL) -lnnz11 refer to https://community.oracle.com/thread/1093616?tstart=0

posted @ 2016-07-28 16:55 InPractice 阅读(199) | 评论 (0)编辑 收藏






posted @ 2011-04-11 18:19 InPractice 阅读(604) | 评论 (1)编辑 收藏





posted @ 2011-02-11 11:02 InPractice 阅读(205) | 评论 (0)编辑 收藏






posted @ 2010-08-16 13:49 InPractice 阅读(283) | 评论 (0)编辑 收藏

Meta Information

Just use this blog to share some meta information.


posted @ 2010-06-05 06:44 InPractice 阅读(164) | 评论 (0)编辑 收藏

Notes on Gentoo Installation

After two weeks' struggle, I have successfully installed Gentoo, a popular GNU/Linux Distribution. For Records, the obstacles I encountered are listed below. (but I can not remember the solution exactly)

0. failed to emerge gpm when I install the links package. If I recall correctly, it is resolved by install gpm manually

1. I encounter issue when I install glib 2.22.5. no update-desktop-database. which is in dev-util/desktop-file-utils. When I try to emerge it, there is a circular dependency on glib. no solution and I forget How I resolve the problem.

2. later after I install glib, with ~amd64 keyword I can install gpm-1.20.6, but it conflicts with the manually inatalled gpm. I remove the conflicted file and emerge successfully.

3. Failed to emerge tiff. edit packages.keywords to add the following. / ~amd64 I am able to use latest tiff in beta-version, which is unstable and masked out.

4. later atk-1.28.0 failed to emerge. edit /etc/make.conf with the following. FEATURES="-stricter". then emerge successfully with only some complain. with out this seting. the warining from GCC will cause that emerge fail.

5. when I run emerge --update system actually gcc will be upgraded from 4.3.4 to 4.4.3. but it failed because of compilation warning, again. add "-stricter" into Features variable in /etc/make.conf work around it.

6. The installation takes a long time, the KDE itself take more than 10 hours. There is still a lot of improvement space! Anyway, it is nice to be able to use it daily.

posted @ 2010-06-03 16:33 InPractice 阅读(276) | 评论 (0)编辑 收藏


在C:\Documents and Settings\<user_name>\Application Data\Subversion\servers文件中加入

http-proxy-host = ***.**.com
http-proxy-port = 8080



posted @ 2010-04-21 17:00 InPractice 阅读(709) | 评论 (0)编辑 收藏


call ttGridCreate('$TT_GRID');
call ttGridCreate(' $TT_GRID');

posted @ 2010-04-13 16:13 InPractice 阅读(194) | 评论 (0)编辑 收藏


1. 直接访问对象的属性。
2. 用方法访问对象的属性。
3. 用Map来存储和访问。
4. 反射-Field 访问。
5. 反射-Method访问。

 * 100 field access, 14,806<br/>
 * 100 method access, 20,393<br/>
 * 100 map access, 66,489<br/>
 * 100 reflection field access, 620,190<br/>
 * 100 reflection method access, 1,832,356<br/>
 *100000 field access, 2,938,362
 *100000 method access, 3,039,772
 *100000 map access, 10,784,052
 *100000 reflection field access, 144,489,034
 *100000 reflection method access, 37,525,719 <br/>
1。getter/setter 的性能已经接近直接属性访问(大约慢50%),没有必要担心getter/setter的性能而采用直接属性访问。


posted @ 2010-04-09 15:57 InPractice 阅读(305) | 评论 (0)编辑 收藏

Tomcat Source Code Reading

0. I am reading the source code of Tomcat 6.0.26. To pay off the effort,
I documents some notes for record. Thanks for the articles about Tomcat
source code, especially the book <<How Tomcat works>>.

1. They are two concepts about server, one is called Server, which
is for managing the Tomcat (start and stop); another is called Connector,
which is the server to serve the application request. they are on the different
ports. The server.xml clearly show the difference.

<Server port="8005" shutdown="SHUTDOWN">
  <Service name="Catalina">
    <Connector port="8080" protocol="HTTP/1.1"
               redirectPort="8443" />
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

although the server is the top level element, logically it should not be.
Actually in code, Bootstrap starts the service first, which
in turn start the Server and server's services.

2. My focus in on Connector part. I care how the request is services by the
Tomcat. Here are some key classes.

Connector --> ProtocolHandler (HttpProtocol
                        and AjpProtocol)                       --> JIoEndPoint
                                                                           --> Handler(Http11ConnectionHandler
                                                                           and AjpConnectionHandler)
3. Connector is most obervious class, but the entry point is not here.
The sequence is like this.

--> JioEndPoint.processSocke(Socket socket)
        -->Http11ConnectorHandler.process(Socket socket)
            -->Http11Processor.process(Socket socket)
                -->CoyoteAdapter.service(Request req, Response res)       

The core logic is in method Http11Processor.process(Socket socket)                                                  

CoyoteAdapter.service(Request req, Response res) bridges between Connector module and Container module.

Any comments are welcome. I may continue the source code reading and dig deeper into it if time permit.

posted @ 2010-03-30 17:11 InPractice 阅读(577) | 评论 (0)编辑 收藏

Navigate forth and back with ctrl+] and ctrl+t in Cscope

It is handy to be able to navigate the source code with Ctrl + ] in Cscope, but I always forget how to navigate back and waste effort many times. So for record, Ctrl+t can navigate back in Cscope.

One more time, Ctrl+] and Ctrl+t can navigate forth and back in Cscope.

posted @ 2010-03-29 14:20 InPractice 阅读(273) | 评论 (0)编辑 收藏

Learning Notes of TCP/IP Illustated Volume 2

How to read the source code in <<TCP/IP Illustrated Volume 2>>

1. Get the source code, original link provided in the book is not available now.
You may need to google it.

2. install cscope and vi.

3. refer to http://cscope.sourceforge.net/large_projects.html for the following steps.

It will include all the source code of the whole OS, not only the kernel.
find src -name '*.[ch]' > cscope.files

we actually only care kernel source.
find src/sys -name '*.[ch]' > cscope.files

4.  wc cscope.files
 1613  1613 45585 cscope.files

5. vim
:help cscope
then you can read the help details.

6. if you run vim in the folder where cscope.out resides. then it will be loaded

7. Try a few commands.
:cs find g mbuf
:cs find f vm.h

They works. A good start.

P.S. this book is quite old, if you know it well and can recommend some better alternative for learning TCP/IP, please post a comments, Thanks in advance.

posted @ 2010-03-29 13:30 InPractice 阅读(114) | 评论 (0)编辑 收藏



posted @ 2010-03-26 14:41 InPractice 阅读(94) | 评论 (0)编辑 收藏


以为用两个素数相乘,其附近存在素数的几率很高。比如, 7×11 = 77, 其附近有79,正好是素数。




写了一个程序验证了一下。16位的整数中,大概只有 10% 能使假设成立。

posted @ 2010-03-26 14:37 InPractice 阅读(113) | 评论 (0)编辑 收藏


由于是在Proxy的网络环境,MSYSGIT 的 git clone 总是失败。需要配置如下环境变量。
export http_proxy="http://<proxy domain name>:<port>"
之后http协议git clone没有任何问题。但是用git 协议仍旧有问题。

之后发现git push 和 git pull 经常不能work。多次尝试后发现用更全的命令行参数可以解决问题。

git pull --fail
git pull origin --fail
git pull  git@github.com:ueddieu/mmix.git --it works.

It seems the Command line short cuts are lack of some user information, such as user name "git".
(which is kind of strange at the first glance.)

git push --fail
git push origin --fail
git push git@github.com:ueddieu/mmix.git master --it works.

Anyway, now I can check in code smoothly. :)

posted @ 2009-12-31 17:13 InPractice 阅读(792) | 评论 (1)编辑 收藏

Trouble caused by un-visible blank character

There are a few cases in which the un-visible blank character will cause
problem, but it is hard to detect since they are not visible.

One famous case is the '\t' character used by Make file, it is used to mark
the start of a command. If it is replace by blank space character, it does
not work, but you can not see the difference if you only look at the make file.
This kind of problem may get the newbies crazy.

Last week, I have encounter a similar issue, which is also caused by unnecessary
 blank space.
As you may know, '\' is used as line-continuation when you have a very long line, e.g.
when you configure the class path for Java in a property file, you may have something like this.


But if you add extra blank space after the '\', then you can not get the complete
content of classpath. Because only when '\' is followed by a '\n' on Unix or '\r''\n'
on Windows, it will work as line-continuation ; otherwise, e.g '\' is followed by
' ''\n', the line is complete after the '\n', the content after that will be the start of
a new line.

Fortunately, it is easy to check this kind of extra blank space by using vi in Unix.
use command '$' to go to the end of line, if there is no extra blank space after '\',
the current position should be '\', if there are any blank space after '\', the current position
is after the '\'.

posted @ 2009-07-01 11:48 InPractice 阅读(165) | 评论 (0)编辑 收藏



“……,I like mangoes”妈妈
“妈妈,我昨天刚教会你,又忘了?是I like watermelon。”


posted @ 2009-06-10 14:26 InPractice 阅读(161) | 评论 (0)编辑 收藏



can not find libdb-4.7.so.
我的解决办法是,建立符号链接/usr/lib/libdb-4.7.so, 后者指向/usr/local/BerkeleyDB/lib/libdb-4.7.so

posted @ 2009-05-29 10:33 InPractice 阅读(106) | 评论 (0)编辑 收藏












posted @ 2009-02-03 15:46 InPractice 阅读(85) | 评论 (0)编辑 收藏

The analysis of MOR(MXOR) instruction implementation in MMIXWare

The analysis of MOR(MXOR) instruction implementation in MMIXWare
 -- A stupid way to understand the source code.
 the implementation of MOR(MXOR) is in file: mmix-arith.w
 436 octa bool_mult(y,z,xor)
 437   octa y,z; /* the operands */
 438   bool xor; /* do we do xor instead of or? */
 439 {
 440   octa o,x;
 441   register tetra a,b,c;
 442   register int k;
 443   for (k=0,o=y,x=zero_octa;o.h||o.l;k++,o=shift_right(o,8,1))
 444     if (o.l&0xff) {
 445       a=((z.h>>k)&0x01010101)*0xff;
 446       b=((z.l>>k)&0x01010101)*0xff;
 447       c=(o.l&0xff)*0x01010101;
 448       if (xor) x.h^=a&c, x.l^=b&c;
 449       else x.h|=a&c, x.l|=b&c;
 450     }
 451   return x;
 452 }
 It takes me several hours to understand the details.
 If we treat each octabyte as a matrix, each row corresponds to a byte, then
 y MOR z = z (matrix_mulitiply) y
 For a=((z.h>>k)&0x01010101)*0xff;
 (z.h>>k)&0x01010101 will get the four last bit in (z.h>>k). depends on the bit in last row,
 ((z.h>>k)&0x01010101)*0xff will expand the bit (either 0 or 1) into the whole row.
 *     0x01010101   
 =           ff
 =    ffffffff      
(depending on the last bit in each row of z, the result could be #ff00ff00. #ff0000ff, etc.)

similarily, b=((z.l>>k)&0x01010101)*0xff; will expand the last bit in each byte into the
whole byte.

over all, after these two step, the z becomes the replication of it's last row, since k vary
from 0 to 7, it will loop on all the rows actually.

 For c=(o.l&0xff)*0x01010101, it will get the last byte in o.l and populate it to other three byte.
 since it will not only or/xor h but also l. it is not necessary populate it to o.h.
 one example,
 let (z.h>>k)&0x01010101 = 0x01000101, then a= 0xff00ffff;
 let (z.l>>k)&0x01010101 = 0x01010001, then b= 0xffff00ff;
 let (o.l&0xff)=0xuv, then c= 0xuvuvuvuv;
  then a&c=0xuv00uvuv;
 consider the elements [i,j] in result x.  in this round, what value was accumalated in by operation
 it is the jth bit in last byte of o.l & ith bit in last column of z.(do not consider looping now.)
 in this round, the 64 combination of i and j, contirbute the value to the 64 bits in z.
 Noticed that o loop on y from last byte to first byte. There are 8 loop/rounds, in another round.
 say kth round.
 the elements[i,j] will accumuate the jth bit in last (k + 1)th row & the jth bit in last (k+1)th
 that means the jth column in y multiply the ith row in z. it conform to the definiton for
 z matrix_multiply y.

posted @ 2009-01-16 10:54 InPractice 阅读(255) | 评论 (0)编辑 收藏





posted @ 2009-01-13 19:59 InPractice 阅读(386) | 评论 (0)编辑 收藏

A piece of beautiful and trick bitwise operation code.

A detailed reading process of a piece of beautiful and trick bitwise operation code.
The following code is from MMIXWare, it is used to implement the Wyde difference between two octabyte.
     in file: "mmix-arith.w"
     423 tetra wyde_diff(y,z)
     424   tetra y,z;
     425 {
     426   register tetra a=((y>>16)-(z>>16))&0x10000;
     427   register tetra b=((y&0xffff)-(z&0xffff))&0x10000;
     428   return y-(z^((y^z)&(b-a-(b>>16))));
     429 }
It is hard to understand it without any thinking or verification, here is the process I used
to check the correctness of this algorithm.

let y = 0xuuuuvvvv;
     z = 0xccccdddd; (please note the [c]s may be different hex number.)
then y>>16 = 0x0000uuuu;
     z>>16 = 0x0000cccc;
then ((y>>16)-(z>>16)) = 0x1111gggg if #uuuu < #cccc or
     ((y>>16)-(z>>16)) = 0x0000gggg if #uuuu >= #cccc   

so variable a = 0x00010000 if #uuuu < #cccc or
   variable a = 0x00000000 if #uuuu >= #cccc
similarly, we can get
   variable b = 0x00010000 if #vvvv < #dddd or
   variable b = 0x00000000 if #vvvv >= #dddd

for (b-a-(b>>16)))), there are four different result depending on the relation between a and b.
when #uuuu >= #cccc and #vvvv >= #dddd, (b-a-(b>>16)))) = 0x00000000;
when #uuuu >= #cccc and #vvvv < #dddd, (b-a-(b>>16)))) = 0x00001111;
when #uuuu < #cccc and #vvvv >= #dddd, (b-a-(b>>16)))) = 0x11110000;
when #uuuu < #cccc and #vvvv < #dddd, (b-a-(b>>16)))) = 0x11111111;
You can see that >= map to #0000 and < map to #1111

for y-(z^((y^z)&(b-a-(b>>16)))), when (b-a-(b>>16)))) is 0x00000000, z^((y^z)&(b-a-(b>>16))) is
z^((y^z)& 0) = z^0=z, so y-(z^((y^z)&(b-a-(b>>16))))=y-z.
similarily, when (b-a-(b>>16)))) is 0x11111111, z^((y^z)&(b-a-(b>>16))) is
z^((y^z)& 1) = z^(y^z)=y, so y-(z^((y^z)&(b-a-(b>>16))))=0.

when (b-a-(b>>16)))) is 0x11110000 or 0x11110000, we can treat the y and z as two separate wydes.
each wyde in the result is correct.

You may think it is a little stupid to verify such kind of details. but for my point of view,
without such detailed analysis, I can not understand the algorithm in the code. with the hard
work like this, I successfully understand it. The pleasure deserve the effort.
I am wondering how can the author discover such a genius algorithm.

posted @ 2009-01-06 16:04 InPractice 阅读(195) | 评论 (0)编辑 收藏





posted @ 2008-11-19 10:34 InPractice 阅读(90) | 评论 (0)编辑 收藏

Create a MMIX simulator in Java

After reading the <<MMIX: A RISC Computer for the New Millennium>>, I am ispired to Create a MMIX simulator in Java.
Donald Knuth already created a high quality MMIX simulater in C, why I still bother to creating a new one in Java.
First, I want to learn more about how the computer works. I think re-implement a simulator for MMIX can
help me gain a better understanding.
Second, I want to exercise my Java skills.

After about one month's work, I realize that I can not finish it by myself. I am looking for the help.
If you are interested in MMIX and know Java, Please give me a hand.

Currently I have finished most of the instructions, but some important and complex one are not completed
I have developed a few JUnit TestCase for some instructions, but it's way far from covering all the instructions (there are 256 instructions total).
Few of the sample MMIX program in Donald Knuth's MMIXware package, such as cp.mmo, hello.mmo can be
simulated successfully, but there are much more to support.

To help on this project, first you need the access to the current source code. It's hosted on Google
code. Please follow the steps below to access the source code.

Use this command to anonymously check out the latest project source code:
# Non-members may check out a read-only working copy anonymously over HTTP.
svn checkout http://mmix.googlecode.com/svn/trunk/ mmix-read-only

If you are willint to help, please comment on this blog with your email address.

posted @ 2008-11-07 14:39 InPractice 阅读(162) | 评论 (0)编辑 收藏

My confusion about kernel and corresponding clarification.

There are many questions coming into my mind when I read the Linux kernel book and source code. As time goes by, I become more knowledgeable than before and can address those questions by myself, here is the first question addressed by myself.


Q: why kernel have to map the high memory in kernel space, why not just allocate the high memory and only map it in user process.

A: Because kernel also need to access the high memory before it returned the allocated memory to user process. For example, kernel must zero the page or initialized the page for security reason. Please refer to linux device driver page 9.

Q: why not let the clib zero the page or initialize it, it saves the kernel's effort and simplifies the kernel.

A: besides Requesting memory through clib, user program can also request memory through direct System call, in this situation, the security is not guaranteed, the information in memory will be leaked.

posted @ 2008-10-08 14:40 InPractice 阅读(145) | 评论 (0)编辑 收藏

How to substitute text in the file.

9/26/2008 8:57AM
Today I want to research the different ways to substitute text in the file. For records, I written them down.

1. use Ultra Edit, it is super easy for a Windows user if you have Ultra Edit installed.
use Ctrl + R to get Replacement Wizard and follow you intuition.

2. use VI in Unix.
will replace xx with yy.

3. use filter, such as sed, and awk in Unix.
sed -e 's/xx/yy/g' file.in > file.out
replace xx with yy in all the lines. It seem sed will not change the original input file, so I redirect the out put to file.out

posted @ 2008-09-26 10:24 InPractice 阅读(74) | 评论 (0)编辑 收藏



WYSIWYM stands for What You See is What You Mean; WYSIWYG stands for What You See is What You Get;

Microsoft -- Word is always considered as a example of WYSIWYG. Today I have a look at the tool named LyX, which is an example of WYSIWYM. From an end user's point of view, there are more similarity than difference between them.

They both display the the resulted layout on the fly; they both provide button to typeset the document.

The difference I can see between then is -- LyX use text file, while Word use binary file. But I don't think it matters.

In my humble opinion, the real difference between Word and LyX/LaTeX is as the following. In Word, you typeset in the lower level, you can control all the details but it also need more effort. In LyX/LaTex, you typeset in higher level, you only need to figure out the logic structure of the document. The resulted layout is not decided by you, you actually just share the layout developed by the expert. I think it is the key advantage of WYSIWYM.

posted @ 2008-08-15 15:24 InPractice 阅读(183) | 评论 (0)编辑 收藏

Trouble shooting - Fail to send out mail from application server

Yesterday, we found that the application can not send mail successfully; the performance of the module using email feature is also very bad. I suspect it caused by that the mail server host name can not be resolved in the application server.

I executed the following command
host <mail server host name>
It shows a strange IP. It means it can not properly resolve the mail server host name

Then I execute the command below.
man host
The output tells me to resort to /etc/resolv.conf
open it with
vi /etc/resolv.conf

The context is as following:
nameserver <name server 1>
nameserver <name server 2>
update the config with correct DNS server IP.

Everything is OK.

P.S. It seems that the ping and host commands are different. For some host name, I can ping it but I can not host it.

posted @ 2008-08-13 17:57 InPractice 阅读(98) | 评论 (0)编辑 收藏

The inelegance in Operating System

The reality is far from the idealism - the inelegance in Operating System

I am interested in Operating System, after I know more and more concepts, know more and more details, I realize that the reality is far from the idealism. The root cause is the history and to some extent, it is the back compatibility. we can not afford to make a brand new thing from scratch, we need to include many old things in any things.

Let me give some example about how the history make the current Operation System become complicated and inelegant.
1. DMA
DMA stands for Direct Memory Access, which is a way to improve the parallelism in computer system. Basically, with DMA, peripheral device can access main Memory simultaneously when CPU is running. but for historical reason, in X86 platform, some DMA device only have 24 bit address line. which limit the memory scope to 16M. since X86 platform is also lack of IO-MMU to remap the address, the memory can be used in DMA is [0,16M). It definitely complicated the memory management.

2. High Memory
Since  Linux kernel has only 1G linear address space, it can not address all the 4G physical memory in 32 bit machine. This is actual a design issue in Linux for historical reason. it does not predict that some day, the physical memory will become so large. Later in order to support more than 1 G physical memory, CONFIG_HIGHMEM compile option was added. There are also other way to fix this problem, such as 4G kernel space v.s. 4G user space.

3. PAE
PAE stands for Physical Memory Extension, PAE make it possible to support up to 64G physical memory. but to me, it is just a temporary solution, does not deserve the effort. I even do not want to have a look on the corresponding document. It does not make too much sense. I prefer to directly move to 64 bit platform. 64 bit platform has its own problems though.

the above is just some inelegant in hardware. majorly cause by historical reason. I am wondering how can we keep up the quick development under the burden of history. maybe at some point, we finally need to throw away the history and move on with a brand new start.

posted @ 2008-08-06 10:58 InPractice 阅读(121) | 评论 (0)编辑 收藏

Basics of Virtual Memory Area

Virtual Memory Area

Virtual Memory Area is also called Memory Region in some book.

In the process address spaces, there are many memory areas, contiguous addresses wil be divided into different memory area if the access right of them are different. For example, in one Java Process, there are 359 memory areas.

so the kernel need to find a effective way to insert into, remove from, search from the list of memory areas. The semantics of find_area  API is the as the following.

return null if
    1. The list itself is empty.
    2. The list is not empty, and the address is big than the last memory area.

return found area if
    1. the address is in the region of one area.
    2. the address is not in the region of any area. but is not bigger than the last area.       
        it means it is in the hole between areas. right area besides the hole is returned.

posted @ 2008-08-05 17:34 InPractice 阅读(114) | 评论 (0)编辑 收藏

How Linux kernel decrease the Kernel Stack Size in 2.6

The kernel are trying to use as little resource as possible. Here is an example, Originally, in kernerl 2.4, the size of Kernel Stack is 8K. Now, in kernel 2.6, it could be 4K, if you enable it in compilaiton time.

Why will kernel spend effort to support such a feature when most of PC have more than 1 Gigabyte memonry. I think it has something to do with the C10K probleum; C10K means Concurrent 10 Thousand Processes(Threads). considering a system with more thant 10 thousand processes, such as a WEB server, the save of 4K in every kernel stack will become 4K * 10 K = 40 M tatal save of memory, which is a big deal!

How is it possible to achieve that? originally the kernel mode stack is also used in Exception and Interrupt handling, but Exception and Interrupt handling is not specific to any process. so in 2.6, Interrupt and Exception will have their own Stack for each CPU. Kernel stack is only used by process in the kernel mode. so the acutal kernel stack did not become small.
2.4     8K Stack shared between process kernel mode, Exception, Interrupt.
2.6     4K Stack specific for process kernel mode Stack
        4K Stack specific for Exception Stack
        4K Stack specific for Interrupt Stack
Besides this, in 8K stack of 2.4, task_struct is at the bottom of stack, which may cost about 1K, in 4K stack of 2.6, only thread_info is at the bottom of stack, the task_struct is put into a per-CPU data structre, thread_info is only about 50 bytes.

posted @ 2008-08-01 09:26 InPractice 阅读(1002) | 评论 (0)编辑 收藏

High level summary of my understanding on Linux Kernel Memory Management

Here is just the high level summary of my understanding on Linux Kernel Memory Management. I think it can help achieve a better understanding of the book <<understanding linux kernel>>.

It is said, the memory management is most complex sub-system in linux kernel, at the same time, there aren't too much System Calls for it. Becuase most the the complex mechanism happens trasparently to the user process, such as COW(Copy On Write), On Demand Paging. For user process, to successfully refer to a linear memory address, the following factors are necessary:
    vm_area_struct (Virtual Memory Area, Memory Region) are set up correctly.
    Phsical memory are allocated.
    Page Global Directory, Page Table, and the corresponding entry are correclty set up according to Virtual Memory Area and Phisical Meory.
This three factors can be further simplified as
    Virtual Memory
    Phisical Memory
    Mappting between Virtual Momory and Phisical Memory.

From user process's perspective, only Virtual Memory is visible, when user process applys for memory, he got virtual memory; phisical memory may not be allocated yet. All these three factors are managed by the kernel, they can be thought of as three resource managed by the kernel. kernel not only need to manage the Virtual Memoty in user address space, but also need to manage Virtual Memory in kernel address space.

When user process try to use his virtual memory, but the phisical memory is not allocated yet. Page Exception happens, kernel take charge of it and allocate the phisical memory and set up the mapping. user process reexecute the instruction and everything go forward smoothly. It's called On Demand Paging.

Besides that there are many more concepts, such as Memory mapping, non-linear memory mapping. I will continue this article when I dig into the details.

posted @ 2008-07-29 12:20 InPractice 阅读(256) | 评论 (0)编辑 收藏

One Interesting Usage of PS command.

 ps -H -A
can show the relationship between all the processes in a tree format. it is helpful when you want to research the internals of UNIX.
we can see from the above that all the process are the children of init (directly or indirectly). especially the kernel thread are also the children of init process.
process 0 is special, it is not displayed.

From the following:
 we can see that how ssh works. actually I have created two ssh session to the server.

posted @ 2008-07-28 15:51 InPractice 阅读(107) | 评论 (0)编辑 收藏

Java 的虚拟内存分配

-Xms<size>        set initial Java heap size
-Xmx<size>        set maximum Java heap size

Java -Xms512M 应该为Java分配至少512M的内存,但是在Linux中用TOP查看,其RSS和SIZE的值远小于512M。我的理解是Java向操作系统申请内存时,用的是mmap2或者old_mmap系统调用,这两个系统调用其实都没有真正分配物理内存,而仅仅是分配了虚拟内存。所以预先分配的这些内存要到实际使用时才能落实到位。

posted @ 2008-07-17 11:21 InPractice 阅读(247) | 评论 (0)编辑 收藏

summary of regular expression grammar

There are not too much grammar. here is just the incomplete summary for the future reference.
.                                any character
|                                or
()                                grouping
[]                                character class
[^]                                negative character class

Greedy Quantifier
?                                optional
*                                any amount
+                                at least one
lazy quantifier
possessing quantifier

position related
^                                start ot the line
$                                end of the line
\<                                start of the word
\>                                end of the word
\b                                start or end of the word

non-capturing group                (?:Expression)
non-capturing atomic group        (?>Expression)
positive lookahead                (?=Expression)
negative lookahead                (?!Expression)
positive lookbehind                (?<=Expression)
negative lookbehind             (?<!Expression)

\Q start quoting
\E end quoting

mode modifier
valid modifier
i            case insensitive match mode
x             free spacing
s            dot matches all match mode
m            enhanced line-anchor match mode


posted @ 2008-07-16 17:06 InPractice 阅读(66) | 评论 (0)编辑 收藏

kernel memory mapping summay

kernel memory mapping summay

Today, finally I become clear about the relationship between
fixed mapping
permanent kernel mapping
temporary kernel mapping
noncontiguous memory area mapping
(I feel that most of the name is not appropriate, to some text, it will mislead the reader.)

4G linear virtual address space is divided into two major part.
kernel space mapping     [3G, 4G)
user space mapping        [0, 3G)

kernel space mapping is divided into more pieces
linear mapping [3G, 3G + 896M)
non linear mapping [3G + 896M + 8M, 4G)
1. Fixed Mapping (wrong name, should be compile time mapping, the virtual address is decided in compile time. )
2. Temporary mapping
3. Permanent mapping
4. noncontiguous memory area mapping (Vmalloc area)

The following is the diagram for the reference.

FIXADDR_TOP            (=0xfffff000)
                    fixed_addresses (temporary kernel mapping is part of it)
                     #define __FIXADDR_SIZE  (__end_of_permanent_fixed_addresses << PAGE_SHIFT)
                    temp fixed addresses (used in boot time)
                     #define __FIXADDR_BOOT_SIZE     (__end_of_fixed_addresses << PAGE_SHIFT)
                    Persistent kmap area (4M)

                     noncontiguous memory area mapping (Vmalloc area)
VMALLOC_START        (((unsigned long) high_memory + 2*VMALLOC_OFFSET-1) & ~(VMALLOC_OFFSET-1)) 

high_memory            MIN (896M, phisical memory size)
below the excerp of the source code.

#ifdef CONFIG_X86_PAE
#define LAST_PKMAP 512
#define LAST_PKMAP 1024
#define VMALLOC_OFFSET  (8*1024*1024)
#define VMALLOC_START   (((unsigned long) high_memory + \                         
                        2*VMALLOC_OFFSET-1) & ~(VMALLOC_OFFSET-1))   
#ifdef CONFIG_HIGHMEM                                             
# define VMALLOC_END    (FIXADDR_START-2*PAGE_SIZE)              

enum fixed_addresses {
        FIX_APIC_BASE,  /* local (CPU) APIC) -- required for SMP or not */
        FIX_CO_CPU,     /* Cobalt timer */
        FIX_CO_APIC,    /* Cobalt APIC Redirection Table */
        FIX_LI_PCIA,    /* Lithium PCI Bridge A */
        FIX_LI_PCIB,    /* Lithium PCI Bridge B */
#ifdef CONFIG_X86_F00F_BUG
        FIX_F00F_IDT,   /* Virtual mapping for IDT */
        FIX_CYCLONE_TIMER, /*cyclone timer register*/
        FIX_KMAP_BEGIN, /* reserved pte's for temporary kernel mappings */
        /* temporary boot-time mappings, used before ioremap() is functional */
#define NR_FIX_BTMAPS   16
        FIX_BTMAP_END = __end_of_permanent_fixed_addresses,

posted @ 2008-07-16 17:05 InPractice 阅读(280) | 评论 (0)编辑 收藏

Scaling your JEE application part 2 - 阅读笔记

scale up     -     vertically scale
scale out     -     horizontally scale

scale out
1. Use share nothing clustering architectures
    The session failover functionality cannot avoid errors completely when failures happen, as my article mentioned, but it will damage the performance and scalability.
2. Use scalable session replication mechanisms   
    The most scalable one is paired node replication, the least scalable solution is using database as session persistence storage.

3. Use collocated deployment instead of distributed one.

4. Shared resources and services
    Database servers, JNDI trees, LDAP Servers, and external file systems can be shared by the nodes in the cluster.

5. Memcached
    Memcached's magic lies in its two-stage hash approach. It behaves as though it were a giant hash table, looking up key = value pairs. Give it a key, and set or get some arbitrary data. When doing a memcached lookup, first the client hashes the key against the whole list of servers. Once it has chosen a server, the client then sends its request, and the server does an internal hash key lookup for the actual item data.
6. Terracotta   
    Terracotta extends the Java Memory Model of a single JVM to include a cluster of virtual machines such that threads on one virtual machine can interact with threads on another virtual machine as if they were all on the same virtual machine with an unlimited amount of heap.
7. Using unorthodox approach to achieve high scalability   

posted @ 2008-07-09 11:43 InPractice 阅读(240) | 评论 (0)编辑 收藏


今天遇到了一个奇怪的Hibernate问题。(我用得hibernate是2.1版。比较旧,不知道这个问题在hibernate 3 中是否存在。)
java.lang.ClassCastException: java.lang.Boolean  
at net.sf.hibernate.type.StringType.set(StringType.java:26)
at net.sf.hibernate.type.NullableType.nullSafeSet(NullableType.java:48)  
at net.sf.hibernate.type.NullableType.nullSafeSet(NullableType.java:35)  
at net.sf.hibernate.persister.EntityPersister.dehydrate(EntityPersister.java:393)
at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:466)  
at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:442)  
at net.sf.hibernate.impl.ScheduledInsertion.execute(ScheduledInsertion.java:29)    
at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2382)  
at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2335) 
at net.sf.hibernate.impl.SessionImpl.flush(SessionImpl.java:2204)

public class FakePO {
    String goodMan;
    public String getGoodMan() {
        return goodMan;
    public void setGoodMan(String goodMan) {
        this.goodMan = goodMan;
    public boolean isGoodMan(){
        return "Y".equalsIgnoreCase(goodMan);
怀疑可能是这个衍生的辅助方法isGoodMan()导致的问题。通过追踪Hibernate 2的源代码,发现hibernate 2是按如下方式通过反射API访问PO的。

private static Method getterMethod(Class theClass, String propertyName) {      
        Method[] methods = theClass.getDeclaredMethods();
        for (int i=0; i<methods.length; i++) {
            // only carry on if the method has no parameters
            if ( methods[i].getParameterTypes().length==0 ) {
                String methodName = methods[i].getName();
                // try "get"
                if( methodName.startsWith("get") ) {
                    String testStdMethod = Introspector.decapitalize( methodName.substring(3) );
                    String testOldMethod = methodName.substring(3);
                    if( testStdMethod.equals(propertyName) || testOldMethod.equals(propertyName) ) return methods[i];
                // if not "get" then try "is"
                /*boolean isBoolean = methods[i].getReturnType().equals(Boolean.class) ||
                if( methodName.startsWith("is") ) {
                    String testStdMethod = Introspector.decapitalize( methodName.substring(2) );
                    String testOldMethod = methodName.substring(2);
                    if( testStdMethod.equals(propertyName) || testOldMethod.equals(propertyName) ) return methods[i];
        return null;

posted @ 2008-07-07 12:14 InPractice 阅读(490) | 评论 (0)编辑 收藏

Linux 内核源码阅读 - write 系统调用的实现

这里假设最普通的情况,不考虑Direct IO 的情况。从全家的高度看,要往一个文件中写入内容,需要一下几步。
1. sys_write 将用户进程要写的内容写入到内核的文件页面缓冲中。sys_write 本身到此就结束了。
2. pdflush 内核线程(定期或者由内核阈值触发)刷新脏的页面缓冲,其实只是提交IO请求给底层的驱动。
3. IO请求并不是同步执行的,而是由底层的驱动调度执行,发出DMA操作指令。
4. 物理IO完成之后会中断并通知内核,内核负责更新IO的状态。

sys_write 的调用过程。(我的linux内核版本为2.6.24,文件系统为ext3)
asmlinkage ssize_t sys_write(unsigned int fd, const char __user * buf, size_t count)

vfs_write(file, buf, count, &pos);

file->f_op->write(file, buf, count, pos);
这里的file->fop 是在open一个文件是初始化的函数指针,ext3文件系统对应的函数为do_sync_write。
 for (;;) {
300                 ret = filp->f_op->aio_write(&kiocb, &iov, 1, kiocb.ki_pos);
301                 if (ret != -EIOCBRETRY)
302                         break;
303                 wait_on_retry_sync_kiocb(&kiocb);
304         }
306         if (-EIOCBQUEUED == ret)
307                 ret = wait_on_sync_kiocb(&kiocb);
 filp->f_op->aio_write(&kiocb, &iov, 1, kiocb.ki_pos); 是实现的核心,其函数指针指向ext3_file_write。

generic_file_aio_write(iocb, iov, nr_segs, pos);

__generic_file_aio_write_nolock(iocb, iov, nr_segs,  &iocb->ki_pos);

generic_segment_checks(iov, &nr_segs, &ocount, VERIFY_READ);

generic_file_buffered_write(iocb, iov, nr_segs, pos,ppos,count,written);

generic_file_direct_IO(WRITE, iocb, iov, pos, *nr_segs);


posted @ 2008-06-02 21:43 InPractice 阅读(2542) | 评论 (0)编辑 收藏


1. 工欲善其事,必先利其器
我开始的时候是用find xargs 和 egrep 配合来搜索关键字, 看代码的效率很低.后来装了ctags,方便多了.最初没有装ctags, 是因为觉得可能装起来费劲, 其实还是很容易装的,也就是那么几步, google一下就搞定了.

2. 要及时实践.

3. Unix下的工具看起来不如Windows的工具异用.其实不然,可能是门槛搞一些.多数人象我一样因此不敢去碰它.入门以后,会发现其实Unix下的工具真是短小精悍. 就拿VIM + Ctags 阅读源代码来说,觉得性价比高.符合80/20原则.

posted @ 2008-03-28 17:39 InPractice 阅读(184) | 评论 (0)编辑 收藏



posted @ 2007-10-20 07:56 InPractice 阅读(341) | 评论 (0)编辑 收藏


J2EE项目中基本都是遵循分层架构的,自然包结构也是基于分层的。DAO层有DAO package。service 层有service package。在这些包下面再根据模块划分子包。







posted @ 2006-04-01 09:26 InPractice 阅读(569) | 评论 (0)编辑 收藏

最近看了一些项目代码. 一点感想

1. 滥用继承。比如在类结构中已经用了模板模式,照理说子类按照需要覆盖模板中的实现即可。可是不知出于何种目的。有的子类却是抽象的,需要从该抽象子类再次扩展,导致继承树不必要的深。
2. 滥用接口。经常看到接口中定义了一堆的方法,而且该接口只有一种实现。这种接口纯粹是摆设,这样的接口根本不能指望它有稳定性。实际情况是接口将随着实现的改变而改变。你说要这样的接口干吗?
3. 喜欢抽象出框架,但是这些框架对于当前的应用来说真实不必要的复杂。事实上没有增加重用,反而降低了代码的可读性。
4. 滥用工厂模式。大家不是觉得模式很难实际运用吗。真想用模式吗?那还不简单。给每个对象都定义一个工厂类不就的了吗?说心里话,我真看不出那些工厂模式到底实现什么设计上的好处。
5. 抽象的能力不够。在一个分页的实现中。把查寻字符串抽象到了一个类中。正确的方法应该是把查询结果抽象出来。

posted @ 2006-03-31 21:36 InPractice 阅读(190) | 评论 (0)编辑 收藏



简单的说,道具就是六根长方形木条, 其中五根都是有榫头的, 另一根则是直木.

这个玩具我上个周末在黄山买的.试过四次, 一直没有成功.
今天吃过晚饭, 无意之中和老婆合作完成了.

基本上,我的分析能力还是可以的. 能够抓住要点.
但是动手能力不如我老婆, 眼看就要成功的时候,还是老婆眼疾手快, 完成最好一块.

其实刚玩的时候, 我就总结出一些基本要领.尽管不是成功的法则.但是能避免不必要的失败

1. 要以唯一的一根直木为思考的出发点.

2. 有两个方向各有两个木条和该直木相交. 其中一个方向需要居中的榫头(刚开始只是总结出这点).

3. 分析已经有的木条榫头的形状, 合理的组合不到20种.

4. 通过手工试验. 再摸索出一些规律, 养成空间的感觉. 半个小时应该可以完成.

因为我需要一些东西来证明自己的智商还在. 我说的智商是与具体知识不太相关的一种能力.
都说成功离不开自信. 自信不仅仅是一种信念. 应该通过对自己能力的检验来建立自信.

posted @ 2006-03-31 21:01 InPractice 阅读(782) | 评论 (1)编辑 收藏



有两种分析方法, 根据Developer在修复Bug时选择的CommonCause,选择比重最大的CommonCause,



(我确信这种分析方法没有太大的意义,因为缺乏对底层原因的了解. 而且Developer在选择common Cause的时候完全可能没有合适的而任选一个.经常看到的一个例子是缺乏UT. 这个就不一定是真正的原因,事实往往是做了UT却没有发现出Bug.这种分析方法是典型的不深入实际的浮夸作风, 依赖统计的数据而没有看到统计数据事实上可能存在问题. 这样的工作肯定效率不高. )



posted @ 2006-02-19 18:01 InPractice 阅读(605) | 评论 (0)编辑 收藏

Throw away unnecessary interface!

Why we need Interface? The most important benefit come from the fact: The code depend on the interface no need to care about the implementaion class. and if the implementation class is changed later, the client code no need to update.
This is the feature of Ploymophism of OOP, such as Java. 

In some projects, the struts framework was adopted, so all the field need to be persisted is in ActionForm. In order to avoid that the Service layer /DAO layer will depends on the struts. One way is to define a interface which have getter and
setter to access all the fields need to be persisted. The design is like this:

XXXActionForm --------> XXXInterface <--------------ServiceLayer/DAO Layer
                                          most of them are 
                                          getter and setter
I can understand this concern, it seems follow the paterns in Enterprise Application Architecture Pattern. but I can not agree this kinds of design. I believe this is misuse of interface.

First, in this kinds of design, if we add some fields, we need update the actionForm, them also need to update interface.
It is boring, and in this case, the interface can not provide any abstraction so the interface need to evolve as the implementation changed.

Second, there is only one  kind of implementaion in the system, so the interface can not provide the benifit from making use of polymorphism.

In a word, we can get nothing design benefit from Interface in this case, And Have burden to keep the implementaion and interface synchronized.

posted @ 2006-02-19 17:30 InPractice 阅读(252) | 评论 (0)编辑 收藏