"There are 10 kinds of people in the world ...those who understand binary and those who do not"

A primer in numbering systems

There are four basic numbering systems that are used in the human, networking, and computer science worlds. Binary, octal, decimal and hexadecimal. Decimal, the numbering system we are most familiar with has a base, or radix, of 10 and to us this is natural. Why 10? Most of us have 10 fingers and that is how it was selected as our numbering system; it's as simple as that. The Yuki people of California on the other hand (pun intended) use an octal numbering system, counting the spaces between their fingers and not the fingers themselves. Pretty cool, huh?

The other systems widely in use in the networking and computer science world are binary, octal and hexadecimal (See table below). In this tutorial we'll concern ourselves only with binary, decimal and hexadecimal, ignoring octal altogether. As a side note, one use of octal numbering is with Linux and UNIX systems to set file permissions using the chmod command.

It's all just ones and zeros - bits and nibbles and bytes oh my!

Yes, we're talking binary here. I start here by asking that you don't think of a binary number as being a just series of 1's and 0's, such as 01000001 (65 decimal), but rather as a valid numbering system. So, why binary and not decimal? Take our previous example of 01000001. It's really a sort of computer Morse code, but instead of dots and dashes, it's the presence or absence of state, on or off; whether it be electrical pulses on an Cat5 cable or the state of transistors within a CPU or memory stick. Individually a bit, that is a 1 or 0, is pretty useless. However, put them together in a byte (8 bits) for example, then you have something. With our previous example, the byte 01000001 represents the character 'A'. With the computer's ability to process and transmit millions of bytes in miliseconds it can give us letters, words, pictures, music, videos and oodles of other things almost instantly. No matter what we type or create via keyboard or mouse, eventually it all gets translated into a numerical series of 1's and 0's to represent our intent. So, you see it IS all just 1's and 0's! Oh, I almost forgot ...a nibble is half a byte, 4 bits.

Thinking in binary

With the decimal numbering system we have 10 digits to work with, 0 thru 9 and when we count we start at 0, then 1, then 2 and so on until we reach 9. Then what? Well, we've run out of digits. So, we start again at 0, but we place a 1 to the left giving us 10 (ten). And once the 1's and 10's columns reach 9 and 9 (99) they  both flip, giving us 100. Same thing goes with binary except we only have 2 digits to work with, 1 and 0.

The principal is the same we start at 0 then 1, then ...well, we've run out of digits haven't we? So, we flip that 1 to 0 and place a 1 to the left giving us 10 (two). Increment by 1 again and we have 11 (three) and once again they both flip and we move into a new column giving us 100 (four). This is probably not so bad for small numbers, but once you get past 8 or so binary digits it becomes unmanageable. I mean what does 10011100 equate to in human terms? Don't even mention that binary numbers are unwieldy; 3,000 is 101110111000 in binary ...geez! So, while it is true that we need to think in binary, we can represent these binary numbers in a more human friendly form. Anyone say decimal? Fortunately we have the ability (and absolute need) to convert between the two. 

Converting binary to decimal

For this section I'll start by examining and explaining the table below.

• First thing  to take note of is that there are 8 individual bits giving us a byte. When working with IP addresses, networking folks refer to this as an octet. Obviously, binary numbers can be larger than a simple byte. However, the byte is the most basic representation of data and as such, for the most part, we'll stay at the byte level.

• The 'Bit Set' row is for our binary digits, either 0 or 1. Here they are all 1.

• The 'Exponent Value' row is how we arrive mathematically at the 'Decimal Value' row. An exponential value such as 2³ means 2 x 2 x 2 = 8. By the way, any number with an exponent of 0 ALWAYS equates to 1 (2⁰ = 1 and 400⁰ also = 1).    

• The 'Decimal Value' row represents positionally the decimal value of the binary digit in question. It's imperative that you commit this row to memory. Practice by writing it down over and over 'till its part of your DNA :-)

• The labels 'MSB' and 'LSB' stand for 'Most Significant Bit' and 'Least Significant Bit' respectively. This lets us know which end of the binary number we are talking about. For example, if I had a binary number of 11010101 and I say 'Starting at bit 1' ...that might be confusing. However, if I say 'Starting with the bit at LSB 1'. Well, you get the picture.

Previously we talked about thinking in binary means knowing how to count in binary and that still holds valid, but now, in order to convert between the two, we are going to only concern ourselves with the position of each binary digit as it relates to its decimal equivalent. What?? OK, take for example the binary number 00001010. Referring to the table above just add the positional decimal equivalents where the 1's occur. So, 00001010 means 128 (no), 64 (no), 32 (no), 16 (no), 8 (yes), 4 (no), 2 (yes) 1 (no). Given that, we see that 8 + 2 =10. In other words 00001010 binary is 10 decimal. Here are a few more examples.

• 11000000 - 128 + 64 = 192

• 00000001 - = 1

• 00110011 - 32 + 16 + 2 + 1 = 51

• 11011011 - 128 + 64 + 16 + 8 + 2 + 1 = 219

• 00000000 - = 0

• 11111111 - 128 + 64 + 32 +16 + 8 + 4 + 2 + 1 = 255

If you encounter a binary number less than a byte such as 11001 (25 decimal), just pad the missing binary digits with 0's, making the number 00011001, a complete byte. Also, if you are presented with a binary number larger than a byte then double the decimal number with each subsequent binary position to the left. See table below. Later, when we work with subnetting, we'll use these larger values. 

At this point you should be comfortable with converting binary to decimal. Now let's go the other way, decimal to binary. I'll show you two different methods. Use the one that best works for you.

Method 1 - Process of Elimination

This method is pretty straight forward. The first thing to do is write down the decimal value row previously covered.

Next, somewhere off to the side, write down your decimal number. With a byte, or octet, the smallest number is 0 and the largest 255, so your number must be within this range. Next, follow the procedure below:

Compare the decimal number with the MSB (128). If your number is larger than or equal to 128 then place a 1 under the 128 column, subtract 128 from your number and move to the next position (64). However, if your number is less than 128 then place a 0 under the 128 column and move to the next number (64) without subtracting.

Repeat the above process with 64, 32, 16 etc. until your original number is reduced to zero. If you reach zero before reaching the LSB (1) fill in the remaining columns with 0's. below are a few examples:

Method 2 - Divide and Conquer

This method has you repeatedly dividing a decimal number by 2 and saving the remainder, either 1 or 0, as a binary digit. Keep doing this until, again, your original number has been reduced to 0. It's not as confusing as it sounds. Let's walk through an example:
Refer to the example above. Given the decimal number 41 and the divide-and-conquer method, we derive its binary equivalent with the following logic:

• 41 divided by 2 is 20, remainder 1. Place a 1 in the remainder column and 20 in the division column. This is the LSB and will be the rightmost binary digit.

• 20 divided by 2 is 10, remainder 0. Place a 0 in the remainder column and 10 in the division column.

• 10 divided by 2 is 5, remainder 0. Place a 0 in the remainder column and 5 in the division column.

• 5 divided by 2 is 2, remainder 1. Place a 1 in the remainder column and 2 in the division column.

• 2 divided by 2 is 1, remainder 0. Place a 0 in the remainder column and 1 in the division column.

• 1 divided by 2 is 0, remainder 1. Place a 1 in the remainder column and we're done ...

... well almost. Our solution came out to be only 6 binary digits long. Since we are working with bytes, just pad the left with 0's until the binary number is 8 digits in length. One more thing. Your last operation will always be 1/2 = 0, remainder 1. Here a couple more examples:

 

Binary and decimal numbers larger than a byte

At times you might have a binary or decimal that is larger than a byte. Don't panic. All you need to do is extend the binary byte row to the left, doubling as you go along.

now becomes the following if we have a 12 bit binary number:

Here is a sample conversion from binary to decimal using process of elimination:

Moving on to hexadecimal

Just when you got your head around binary and decimal, along comes hexadecimal, hex for short. Before, I mentioned that everything in the computing and networking arena is nothing more that 1's and 0's and that axiom remains unchanged. We now know how to convert back and forth between binary (what the computer speaks) and decimal (what we speak), so why do we even need hex? Besides being an efficient numbering system, hexadecimal is widely used to represent to us humans the 1's and 0's that traverse circuits and wires. A few examples where hex numbers are used are colors, memory references, IPv6 addresses and MAC addresses. Let's explore this a little more in detail.

A binary byte is exactly eight digits long, e.g. 00101110 and a hex byte number is exactly two digits in length, e.g. 6A. A decimal byte on the other hand is anywhere between one and three digits long, e.g. 4, 64 or 233. Yes, you could pad these with 0's making 004, 064 and 233, but this is still not as efficient or as practical as hex. (as we'll see when we start converting hex and binary numbers). Download myextended ASCII table; it will help with the following illustration and serve as a useful refernece for your studies:

 This screenshot is a memory dump of my computer. Imagine if those numbers were represented in binary!

Let's get on with it.

What the hex are you talking about?

Below is a hex table with decimal equivalents:

With hex, the first 10 digits, 0 thru 9, are the same as decimal. However, 10 thru 15 decimal are represented as a single hex digit A thru F. The number 5 in decimal is 5 in hex and the number 13 in decimal is D in hex. What happens when we reach F (15 decimal) and need to increment by 1 is that, just like with all other numbering systems, we flip that to 0 and place a one to the left. 0xF + 0x1 = 0x10. What is this 0x business? Placing 0x in front of a hex number is the typical way to indicate it's really hex. It makes it clear that 11 (decimal) and 0x11 (hex) are two different numbers entirely. So, from here on out I'll use the 0x to indicate hexadecimal numbers. Sometimes you'll see the hex numbers 0xA thru 0xF written in lower case (0xa thru 0xf). There's nothing wrong with that and it does not change the numerical value of the number in question. I use caps in this how-to.

Binary to hex - nibbles and bits

To convert a binary byte to hex byte, first split the binary number into two nibbles, treating them as separate numbers, and then compute the hex equivalent for each half. Finally, concatenate the two hex numbers into a single solution. This works perfectly because a nibble has a value from 0 thru 15 or 0x0 thru 0xF.

Here's some examples:

Binary to hex - bits and nibbles

The conversion process from hex to binary is not much different, we just reverse the process. A picture is worth a thousand words. Here's 4,000 of them.

Once in a while you may find yourself presented with a binary or hex number larger than a byte. The methods are the same, just expanded. If you have a binary number to convert then you may have to pad the MSB with 0's in order to have a binary number that falls on nibble boundaries, multiples of 4. For example, the binary number 1001100101 won't convert correctly, because it's ten digits in length. Add two 0's making the number 001001100101. A few illustrations:

Decimal to hex - tens to sixteens

Up to now we've been able to accomplish our conversions armed with only paper and pencil. It's time to break out your calculator for converting decimal to hex and vice versa. Take  Look at these decimal / hex equivalents:
16 = 0x10  |  34 = 0x22  |  214 = 0xD6  |  175 = 0xAF
Here's how we get from the decimal value on the left of the = sign to its hexadecimal equivalent on the right. Remember the divide and conquer method we used for converting decimal to binary? We are going to use the same method except that we are going to be dividing by 16 rather than 2. The upside is that the process is a lot shorter.

• 225 divided by 16 is 14, remainder 0x1. Place a 0x1 in the remainder column and 14 in the division column. This is our rightmost hex digit.

• 14 divided by 16 is 0, remainder 14. Since we are converting to hex we use 0xE. Place 0xE in the remainder column and we're done.

• If you have a small decimal number such as 12, you'll have your answer on the first divide. 12 / 16 = 0, remainder 12 or 0xC. Place a leading 0 in from of the C to keep things uniform. So, the answer would be 0x0C

How easy is that? Here are a few more:

It's no different with numbers larger than a byte:

Hex to decimal - sixteens to tens

Whew! Nearly done. This is our last number conversion segment, hexadecimal to decimal. With a hex number each position represents a power of 16 and what we do is calculate each position's decimal equivalent and then add them all up. We are going to use a large hex number to illustrate this concept. Given the hex number 0xA59C we find its decimal equivalent with the following logic.
(10 x 16³) + (5 x 16²) + (9 x 16¹) + (12 x 16⁰) = 43,296. Here's how it breaks down:

...And a few more:

I encourage you to practice, practice, practice. Download this worksheet to work thru various conversion exercises.

Parting shot - If you are an aspiring network or systems admin then make sure you understand without question binary and decimal conversion through and through.

Coming soon - Part II Understanding IP addresses, subnet masks and subnetting.

./Robert

 以前一直不太会用这个参数。现在认真研究了一下iostat，因为刚好有台重要的服务器压力高,所以放上来分析一下.下面这台就是IO有压力过大的服务器

rrqm/s:   每秒进行 merge 的读操作数目（当系统调用需要读取数据的时候，VFS将请求发到各个FS，如果FS发现不同的读取请求读取的是相同Block的数据，FS会将这个请求合并Merge）。即 delta(rmerge)/s
wrqm/s:  每秒进行 merge 的写操作数目。即 delta(wmerge)/s
r/s:           每秒完成的读 I/O 设备次数。即 delta(rio)/s
w/s:         每秒完成的写 I/O 设备次数。即 delta(wio)/s
rsec/s:    每秒读扇区数。即 delta(rsect)/s
wsec/s:  每秒写扇区数。即 delta(wsect)/s
rkB/s:      每秒读K字节数。是 rsect/s 的一半，因为每扇区大小为512字节。(需要计算)
wkB/s:    每秒写K字节数。是 wsect/s 的一半。(需要计算)
avgrq-sz: 平均每次设备I/O操作的数据大小 (扇区)。delta(rsect+wsect)/delta(rio+wio)
avgqu-sz: 平均I/O队列长度。即 delta(aveq)/s/1000 (因为aveq的单位为毫秒)。
await:    平均每次设备I/O操作的等待时间 (毫秒)。即 delta(ruse+wuse)/delta(rio+wio)
svctm:   平均每次设备I/O操作的服务时间 (毫秒)。即 delta(use)/delta(rio+wio)
%util:      一秒中有百分之多少的时间用于 I/O 操作，或者说一秒中有多少时间 I/O 队列是非空的。即 delta(use)/s/1000 (因为use的单位为毫秒)

如果 %util 接近 100%，说明产生的I/O请求太多，I/O系统已经满负荷，该磁盘
可能存在瓶颈。
idle小于70% IO压力就较大了,一般读取速度有较多的wait.
同时可以结合vmstat 查看查看b参数(等待资源的进程数)和wa参数(IO等待所占用的CPU时间的百分比,高过30%时IO压力高)
另外 await 的参数也要多和 svctm 来参考。差的过高就一定有 IO 的问题。
avgqu-sz 也是个做 IO 调优时需要注意的地方，这个就是直接每次操作的数据的大小，如果次数多，但数据拿的小的话，其实 IO 也会很小.如果数据拿的大，才IO 的数据会高。也可以通过 avgqu-sz × ( r/s or w/s ) = rsec/s or wsec/s.也就是讲，读定速度是这个来决定的。

另外还可以参考
svctm 一般要小于 await (因为同时等待的请求的等待时间被重复计算了)，svctm 的大小一般和磁盘性能有关，CPU/内存的负荷也会对其有影响，请求过多也会间接导致 svctm 的增加。await 的大小一般取决于服务时间(svctm) 以及 I/O 队列的长度和 I/O 请求的发出模式。如果 svctm 比较接近 await，说明 I/O 几乎没有等待时间；如果 await 远大于 svctm，说明 I/O 队列太长，应用得到的响应时间变慢，如果响应时间超过了用户可以容许的范围，这时可以考虑更换更快的磁盘，调整内核 elevator 算法，优化应用，或者升级 CPU。
队列长度(avgqu-sz)也可作为衡量系统 I/O 负荷的指标，但由于 avgqu-sz 是按照单位时间的平均值，所以不能反映瞬间的 I/O 洪水。

  别人一个不错的例子.(I/O 系统 vs. 超市排队)

举一个例子，我们在超市排队 checkout 时，怎么决定该去哪个交款台呢? 首当是看排的队人数，5个人总比20人要快吧? 除了数人头，我们也常常看看前面人购买的东西多少，如果前面有个采购了一星期食品的大妈，那么可以考虑换个队排了。还有就是收银员的速度了，如果碰上了连 钱都点不清楚的新手，那就有的等了。另外，时机也很重要，可能 5 分钟前还人满为患的收款台，现在已是人去楼空，这时候交款可是很爽啊，当然，前提是那过去的 5 分钟里所做的事情比排队要有意义 (不过我还没发现什么事情比排队还无聊的)。

I/O 系统也和超市排队有很多类似之处:

r/s+w/s 类似于交款人的总数
平均队列长度(avgqu-sz)类似于单位时间里平均排队人的个数
平均服务时间(svctm)类似于收银员的收款速度
平均等待时间(await)类似于平均每人的等待时间
平均I/O数据(avgrq-sz)类似于平均每人所买的东西多少
I/O 操作率 (%util)类似于收款台前有人排队的时间比例。

我们可以根据这些数据分析出 I/O 请求的模式，以及 I/O 的速度和响应时间。

下面是别人写的这个参数输出的分析

上面的 iostat 输出表明秒有 28.57 次设备 I/O 操作: 总IO(io)/s = r/s(读) +w/s(写) = 1.02+27.55 = 28.57 (次/秒) 其中写操作占了主体 (w:r = 27:1)。

平均每次设备 I/O 操作只需要 5ms 就可以完成，但每个 I/O 请求却需要等上 78ms，为什么? 因为发出的 I/O 请求太多 (每秒钟约 29 个)，假设这些请求是同时发出的，那么平均等待时间可以这样计算:

平均等待时间 = 单个 I/O 服务时间 * ( 1 + 2 + … + 请求总数-1) / 请求总数

应用到上面的例子: 平均等待时间 = 5ms * (1+2+…+28)/29 = 70ms，和 iostat 给出的78ms 的平均等待时间很接近。这反过来表明 I/O 是同时发起的。

每秒发出的 I/O 请求很多 (约 29 个)，平均队列却不长 (只有 2 个 左右)，这表明这 29 个请求的到来并不均匀，大部分时间 I/O 是空闲的。

一秒中有 14.29% 的时间 I/O 队列中是有请求的，也就是说，85.71% 的时间里 I/O 系统无事可做，所有 29 个 I/O 请求都在142毫秒之内处理掉了。

delta(ruse+wuse)/delta(io) = await = 78.21 => delta(ruse+wuse)/s =78.21 * delta(io)/s = 78.21*28.57 = 2232.8，表明每秒内的I/O请求总共需要等待2232.8ms。所以平均队列长度应为 2232.8ms/1000ms = 2.23，而 iostat 给出的平均队列长度 (avgqu-sz) 却为 22.35，为什么?! 因为 iostat 中有 bug，avgqu-sz 值应为 2.23，而不是 22.35。

（1）删除（delete）流程

Oracle读Block(数据块)到Buffer Cache（缓冲区）(如果该Block在Buffer中不存在)；

在Redo Log Buffer（重做日志缓冲区）中记录Oracledelete操作的细节；

在相应回滚段段头的事物表中创建一个Undo（回滚）条目；

把将要删除的记录创建前镜像，存放到Undo Block（回滚块）中；

在Buffer Cache中的相应数据块上删除记录，并且标记相应的数据块为Dirty（脏）。

（2）提交(Commit)流程

Oracle产生一个SCN；

在回滚段事物表中标记该事物状态为Commited；

LGWR（日志读写进程）Flush Log Buffer到日志文件；

如果此时数据块仍然在Buffer Cache中，那么SCN将被记录到Block Header上，这被称为快速提交；

如果Dirty Block已经被写回到磁盘，那么下一个访问这个Block的进程将会自回滚段中获取该事物的状态，确认该事物被提交。然后这个进程获得提交SCN并写回到Block Header上，这被称为延迟块清除。

转自：http://www.blogjava.net/caizh2009/articles/306531.html

一 非归档模式下

D:>sqlplus "/ as sysdba"

数据库版本为9.2.0.1.0

SQL*Plus: Release 9.2.0.1.0 - Production on 星期一 8月 14 10:20:39 2006

Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.

连接到:
Oracle9i Enterprise Edition Release 9.2.0.1.0 - Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 9.2.0.1.0 - Production

当前session产生的redo
SQL> create or replace view redo_size
2 as
3 select value
4 from v$mystat, v$statname
5 where v$mystat.statistic# = v$statname.statistic#
6 and v$statname.name = 'redo size';

视图已建立。

授权给相应数据库schema
SQL> grant select on redo_size to liyong;

授权成功。

SQL> shutdown immediate;
数据库已经关闭。
已经卸载数据库。
ORACLE 例程已经关闭。

SQL> startup mount;
ORACLE 例程已经启动。

Total System Global Area 122755896 bytes
Fixed Size 453432 bytes
Variable Size 88080384 bytes
Database Buffers 33554432 bytes
Redo Buffers 667648 bytes
数据库装载完毕。

非归档模式
SQL> alter database noarchivelog;

数据库已更改。

SQL> alter database open;

数据库已更改。

SQL> create table redo_test as
2 select * from all_objects where 1=2;

表已创建。

SQL> select * from sys.redo_size;

VALUE
----------
59488

SQL> insert into redo_test
2 select * from all_objects;

已创建28260行。

SQL> select * from sys.redo_size;

VALUE
----------
3446080

SQL> insert /*+ append */ into redo_test
2 select * from all_objects;

已创建28260行。

SQL> commit;

提交完成。

SQL> select * from sys.redo_size;

VALUE
----------
3458156

可以看到insert /*+ append */ into方式redo产生很少.
SQL> select 3446080-59488,3458156-3446080 from dual;

3446080-59488 3458156-3446080
------------- ---------------
3386592 12076

将表redo_test置为nologging状态.
SQL> alter table redo_test nologging;

表已更改。

SQL> select * from sys.redo_size;

VALUE
----------
3460052

SQL> insert into redo_test
2 select * from all_objects;

已创建28260行。

SQL> commit;

提交完成。

SQL> select * from sys.redo_size;

VALUE
----------
6805876

SQL> insert /*+ append */ into redo_test
2 select * from all_objects;

已创建28260行。

SQL> commit;

提交完成。

SQL> select * from sys.redo_size;

VALUE
----------
6818144

非归档模式下表的nologging状态对于redo影响不大
SQL> select 6805876-3460052,6818144-6805876 from dual;

6805876-3460052 6818144-6805876
--------------- ---------------
3345824 12268

结论: 在非归档模式下通过insert /*+ append */ into方式批量加载数据可以大大减少redo产生.

二 归档模式下

SQL> shutdown immediate;
数据库已经关闭。
已经卸载数据库。
ORACLE 例程已经关闭。
SQL> startup mount;
ORACLE 例程已经启动。

Total System Global Area 122755896 bytes
Fixed Size 453432 bytes
Variable Size 88080384 bytes
Database Buffers 33554432 bytes
Redo Buffers 667648 bytes
数据库装载完毕。
SQL> alter database archivelog;

数据库已更改。

SQL> alter database open;

数据库已更改。

SQL> conn liyong
请输入口令:
已连接。

将表redo_test重新置为logging
SQL> alter table redo_test logging;

表已更改。

SQL> select * from sys.redo_size;

VALUE
----------
5172

SQL> insert into redo_test
2 select * from all_objects;

已创建28260行。

SQL> commit;

提交完成。

SQL> select * from sys.redo_size;

VALUE
----------
3351344

SQL> insert /*+ append */ into redo_test
2 select * from all_objects;

已创建28260行。

SQL> commit;

提交完成。

SQL> select * from sys.redo_size;

VALUE
----------
6659932

可以看到在归档模式下，且表的logging属性为true,insert /*+ append */ into这种方式也会纪录大量redo
SQL> select 3351344-5172,6659932-3351344 from dual;

3351344-5172 6659932-3351344
------------ ---------------
3346172 3308588

将表置为nologging

SQL> alter table redo_test nologging;

表已更改。

SQL> select * from sys.redo_size;

VALUE
----------
6661820

SQL> insert into redo_test
2 select * from all_objects;

已创建28260行。

SQL> commit;

提交完成。

SQL> select * from sys.redo_size;

VALUE
----------
10008060

SQL> insert /*+ append */ into redo_test
2 select * from all_objects;

已创建28260行。

SQL> commit;

提交完成。

SQL> select * from sys.redo_size;

VALUE
----------
10022852

可以发现在归档模式，要设置表的logging属性为false，才能通过insert /*+ append */ into大大减少redo产生.
SQL> select 10008060-6661820,10022852-10008060 from dual;

10008060-6661820 10022852-10008060
---------------- -----------------
3346240 14792

结论: 在归档模式下,要设置表的logging属性为false，
才能通过insert /*+ append */ into大大减少redo.

三 下面我们再看一下在归档模式下，几种批量insert操作的效率对比.

redo_test表有45W条记录

SQL> select count(*) from redo_test;

COUNT(*)
----------
452160

1 最常见的批量数据加载 25秒

SQL> create table insert_normal as
2 select * from redo_test where 0=2;

表已创建。

SQL> set timing on

SQL> insert into insert_normal
2 select * from redo_test;

已创建452160行。

提交完成。
已用时间: 00: 00: 25.00

2 使用insert /*+ append */ into方式(这个的原理可以参见<<批量DML操作优化建议.txt>>)，但纪录redo. 17.07秒
SQL> create table insert_hwt
2 as
3 select * from redo_test where 0=2;

表已创建。
SQL> insert /*+ append */ into insert_hwt
2 select * from redo_test;

已创建452160行。

提交完成。
已用时间: 00: 00: 17.07

3 使用insert /*+ append */ into方式，且通过设置表nologging不纪录redo.

SQL> create table insert_hwt_with_nologging nologging
2 as
3 select * from redo_test where 2=0;

表已创建。

/*
或者通过
alter table table_name nologging设置
*/

SQL> insert /*+ append */ into insert_hwt_with_nologging 11.03秒
2 select * from redo_test;

已创建452160行。

提交完成。
已用时间: 00: 00: 11.03

总结:

我们看到对于批量操作，如果设置表nologging，可以大大提高性能.原因就是Oracle没有纪录DML所产生的redo.
当然，这样会影响到备份。nologging加载数据后要做数据库全备.
--------------------------------------------------------------------------------------------------------------------------------

jolly10 发表于:2008.03.18 13:19 ::分类: ( 转载学习内容 ) ::阅读:(1097次) :: 评论 (3) :: 引用 (0)

re: Oracle的Nologging何时生效 与 批量insert加载数据速度(zt) [回复]

下面我又试了试insert into XXX values (XXX)能不能少产生redo,做了试验发现,不行的,下面的过程.
SQL> select * from v$version where rownum archive log list;
Database log mode No Archive Mode
Automatic archival Disabled
Archive destination USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence 17
Current log sequence 19

SQL> create or replace view redo_size
2 as
3 select value
4 from v$mystat, v$statname
5 where v$mystat.statistic# = v$statname.statistic#
6 and v$statname.name = 'redo size';

View created.

SQL> grant select on redo_size to ljg;

SQL> conn ljg/ljg
Connected.

SQL> create table redo_test as
2 select * from all_objects where 1=2;

SQL> CREATE OR REPLACE PROCEDURE p_loging
2 as
3 CURSOR c_a IS
4 SELECT * FROM all_objects;
5
6 BEGIN
7 FOR x IN c_a LOOP
8 INSERT INTO REDO_TEST
9 VALUES(x.OWNER, x.OBJECT_NAME, x.SUBOBJECT_NAME, x.OBJECT_ID, x.DATA_OBJE CT_ID,
10 x.OBJECT_TYPE, x.CREATED, x.LAST_DDL_TIME, x.TIMESTAMP, x.STATUS, x.TEMPORARY, x.GENERATED, x.SECONDARY);
11 END LOOP;
12 COMMIT;
13
14 END;
15 /

Procedure created.

SQL> CREATE OR REPLACE PROCEDURE p_nologing
2 as
3 CURSOR c_a IS
4 SELECT * FROM all_objects;
5
6 BEGIN
7 FOR x IN c_a LOOP
8 INSERT /*+ APPEND */ INTO REDO_TEST
9 VALUES(x.OWNER, x.OBJECT_NAME, x.SUBOBJECT_NAME, x.OBJECT_ID, x.DATA_OBJECT_ID,
10 x.OBJECT_TYPE, x.CREATED, x.LAST_DDL_TIME, x.TIMESTAMP, x.STATUS, x.TEMPORARY, x.GENERATED, x.SECONDARY);
11 END LOOP;
12 COMMIT;
13
14 END;
15 /

Procedure created.

SQL> select * from sys.redo_size;

VALUE
----------
85940

SQL> exec p_loging;

PL/SQL procedure successfully completed.

SQL> select * from sys.redo_size;

VALUE
----------
15273968

SQL> exec p_nologing;

PL/SQL procedure successfully completed.

SQL> select * from sys.redo_size;

VALUE
----------
30411272

SQL> select 15273968- 85940 logging,30411272-15273968 nologging from dual;

LOGGING NOLOGGING
---------- ----------
15188028 15137304

可以看到nologging和logging产生的redo差不多.

jolly10 评论于：2008.06.05 11:07

re: Oracle的Nologging何时生效 与 批量insert加载数据速度(zt) [回复]

下面我又试了试insert into XXX values (XXX)能不能少产生redo,做了试验发现,不行的,下面的过程.
SQL> select * from v$version where rownum archive log list;
Database log mode No Archive Mode
Automatic archival Disabled
Archive destination USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence 17
Current log sequence 19

SQL> create or replace view redo_size
2 as
3 select value
4 from v$mystat, v$statname
5 where v$mystat.statistic# = v$statname.statistic#
6 and v$statname.name = 'redo size';

View created.

SQL> grant select on redo_size to ljg;

SQL> conn ljg/ljg
Connected.

SQL> create table redo_test as
2 select * from all_objects where 1=2;

SQL> CREATE OR REPLACE PROCEDURE p_loging
2 as
3 CURSOR c_a IS
4 SELECT * FROM all_objects;
5
6 BEGIN
7 FOR x IN c_a LOOP
8 INSERT INTO REDO_TEST
9 VALUES(x.OWNER, x.OBJECT_NAME, x.SUBOBJECT_NAME, x.OBJECT_ID, x.DATA_OBJE CT_ID,
10 x.OBJECT_TYPE, x.CREATED, x.LAST_DDL_TIME, x.TIMESTAMP, x.STATUS, x.TEMPORARY, x.GENERATED, x.SECONDARY);
11 END LOOP;
12 COMMIT;
13
14 END;
15 /

Procedure created.

SQL> CREATE OR REPLACE PROCEDURE p_nologing
2 as
3 CURSOR c_a IS
4 SELECT * FROM all_objects;
5
6 BEGIN
7 FOR x IN c_a LOOP
8 INSERT /*+ APPEND */ INTO REDO_TEST
9 VALUES(x.OWNER, x.OBJECT_NAME, x.SUBOBJECT_NAME, x.OBJECT_ID, x.DATA_OBJECT_ID,
10 x.OBJECT_TYPE, x.CREATED, x.LAST_DDL_TIME, x.TIMESTAMP, x.STATUS, x.TEMPORARY, x.GENERATED, x.SECONDARY);
11 END LOOP;
12 COMMIT;
13
14 END;
15 /

Procedure created.

SQL> select * from sys.redo_size;

VALUE
----------
85940

SQL> exec p_loging;

PL/SQL procedure successfully completed.

SQL> select * from sys.redo_size;

VALUE
----------
15273968

SQL> exec p_nologing;

PL/SQL procedure successfully completed.

SQL> select * from sys.redo_size;

VALUE
----------
30411272

SQL> select 15273968- 85940 logging,30411272-15273968 nologging from dual;

LOGGING NOLOGGING
---------- ----------
15188028 15137304

可以看到nologging和logging产生的redo差不多.

jolly10 评论于：2008.06.05 11:07

re: Oracle的Nologging何时生效 与 批量insert加载数据速度(zt) [回复]

在ITPUB中问到可以用BULK COLLECT 来减少insert into values的redo.
CREATE OR REPLACE PROCEDURE p_BulkAdd
AS
TYPE Tredo_test IS TABLE OF REDO_TEST%ROWTYPE;
V_REDO_TEST Tredo_test;
BEGIN
SELECT * BULK COLLECT INTO V_REDO_TEST FROM ALL_OBJECTS;
FORALL X IN V_REDO_TEST.FIRST..V_REDO_TEST.LAST
INSERT INTO REDO_TEST VALUES V_REDO_TEST(X);
END;
/

SQL> select * from sys.redo_size;

VALUE
----------
30411272

SQL> exec p_bulkadd;

PL/SQL procedure successfully completed.

SQL> select * from sys.redo_size;

VALUE
----------
35050796

SQL> select 35050796-30411272 from dual;

35050796-30411272
-----------------
4639524

这个做的确是少了很多redo.是一个方法.

局部索引local index

1.        局部索引一定是分区索引，分区键等同于表的分区键，分区数等同于表的分区说，一句话，局部索引的分区机制和表的分区机制一样。

2.        如果局部索引的索引列以分区键开头，则称为前缀局部索引。

3.        如果局部索引的列不是以分区键开头，或者不包含分区键列，则称为非前缀索引。

4.        前缀和非前缀索引都可以支持索引分区消除，前提是查询的条件中包含索引分区键。

5.        局部索引只支持分区内的唯一性，无法支持表上的唯一性，因此如果要用局部索引去给表做唯一性约束，则约束中必须要包括分区键列
           (ALTER TABLE TEST ADD CONSTRAINT PK_TEST PRIMARY KEY (SEQ_ID,PARTITION_ID) USING INDEX LOCAL;)。

6.        局部分区索引是对单个分区的，每个分区索引只指向一个表分区，全局索引则不然，一个分区索引能指向n个表分区，同时，一个表分区，也可能指向n个索引分区，对分区表中的某个分区做truncate或者move，shrink等，可能会影响到n个全局索引分区，正因为这点，局部分区索引具有更高的可用性。

7.        位图索引只能为局部分区索引。

8.        局部索引多应用于数据仓库环境中。

全局索引global index

1.        全局索引的分区键和分区数和表的分区键和分区数可能都不相同，表和全局索引的分区机制不一样。

2.        全局索引可以分区，也可以是不分区索引，全局索引必须是前缀索引，即全局索引的索引列必须是以索引分区键作为其前几列。

3.        全局分区索引的索引条目可能指向若干个分区，因此，对于全局分区索引，即使只

动，截断一个分区中的数据，都需要rebulid若干个分区甚

至是整个索引。

4.        全局索引多应用于oltp系统中。

5.        全局分区索引只按范围或者散列hash分区，hash分区是10g以后才支持。

6.        oracle9i以后对分区表做move或者truncate的时可以用update global indexes语句来同步更新全局分区索引，用消耗一定资源来换取高度的可用性。

7.        表用a列作分区，索引用b做局部分区索引，若where条件中用b来查询，那么oracle会扫描所有的表和索引的分区，成本会比分区更高，此时可以考虑用b做全局分区索引。

分区索引字典

DBA_PART_INDEXES 分区索引的概要统计信息，可以得知每个表上有哪些分区索引，分区索引的类新(local/global,)

Dba_ind_partitions每个分区索引的分区级统计信息

Dba_indexesminusdba_part_indexes，可以得到每个表上有哪些非分区索引

索引重建

Alter index idx_name rebuild partition index_partition_name [online nologging]

需要对每个分区索引做rebuild，重建的时候可以选择online(不会锁定表),或者nologging建立索引的时候不生成日志，加快速度。

Alter index rebuild idx_name [online nologging]

对非分区索引，只能整个index重建

转自
http://www.csharpwin.com/dotnetspace/8650r3516.shtml
http://hi.baidu.com/wxf966/blog/item/77954d074469e4df7a894700.html

      下面我们来谈一下Oracle中Select语句的特性。Select语句会对表中的数据进行一次扫描，但是究竟扫描多少数据存储块呢，这个并不是说数据库中有多少数据，Oracle就扫描这么大的数据块，而是Oracle会扫描高水位线以下的数据块。现在来想象一下，如果刚才是一张刚刚建立的空表，你进行了一次Select操作，那么由于高水位线HWM在最低的0位置上，所以没有数据块需要被扫描，扫描时间会极短。而如果这个时候你首先插入了一千万条数据，然后再用delete语句删除这一千万条数据。由于插入了一千万条数据，所以这个时候的高水位线就在一千万条数据这里。后来删除这一千万条数据的时候，由于delete语句不影响高水位线，所以高水位线依然在一千万条数据这里。这个时候再一次用select语句进行扫描，虽然这个时候表中没有数据，但是由于扫描是按照高水位线来的，所以需要把一千万条数据的存储空间都要扫描一次，也就是说这次扫描所需要的时间和扫描一千万条数据所需要的时间是一样多的。所以有时候有人总是经常说，怎么我的表中没有几条数据，但是还是这么慢呢，这个时候其实奥秘就是这里的高水位线了。

       那有没有办法让高水位线下降呢，其实有一种比较简单的方法，那就是采用TRUNCATE语句进行删除数据。采用TRUNCATE语句删除一个表的数据的时候，类似于重新建立了表，不仅把数据都删除了，还把HWM给清空恢复为0。所以如果需要把表清空，在有可能利用TRUNCATE语句来删除数据的时候就利用TRUNCATE语句来删除表，特别是那种数据量有可能很大的临时存储表。

        在手动段空间管理（Manual Segment Space Management）中，段中只有一个HWM，但是在Oracle9iRelease1才添加的自动段空间管理（Automatic Segment Space Management）中，又有了一个低HWM的概念出来。为什么有了HWM还又有一个低HWM呢，这个是因为自动段空间管理的特性造成的。在手段段空间管理中，当数据插入以后，如果是插入到新的数据块中，数据块就会被自动格式化等待数据访问。而在自动段空间管理中，数据插入到新的数据块以后，数据块并没有被格式化，而是在第一次在第一次访问这个数据块的时候才格式化这个块。所以我们又需要一条水位线，用来标示已经被格式化的块。这条水位线就叫做低HWM。一般来说，低HWM肯定是低于等于HWM的。

下载地址：http://cds-esd.sun.com/ESD6/JSCDL/jdk/6u23-b05/jre-6u23-linux-x64.bin?AuthParam=1292401739_2abc6c61c57eec5b100fe647086d1f33&TicketId=CJ0nhpzNkjGGwpbkW40LAgA%3D&GroupName=CDS&FilePath=/ESD6/JSCDL/jdk/6u23-b05/jre-6u23-linux-x64.bin&File=jre-6u23-linux-x64.bin

(二)   安装jre-6u23-linux-x64.bin

      将jre-6u23-linux-x64.bin上传到linux服务器

1.     在终端上，键入：
su

2.     输入超级用户口令。

3.     切换到所需的安装目录。类型：
cd <目录路径名>
例如，要在 /usr/java/ 目录中安装软件，请键入：
cd /usr/java/

关于超级用户访问权限的说明：要将 JRE 安装在系统级位置（例如 /usr/local），必须以超级用户身份登录，从而获得必要的权限。如果您不具有超级用户访问权限，请将 JRE 安装在您的主目录中，或者安装在您具有写入权限的子目录中。

4.     将您下载的文件的权限更改为可执行。类型：
chmod a+x jre-6u23-linux-x64.bin

5.      启动安装过程。键入：
./ jre-6u23-linux-x64.bin

此时将显示二进制许可协议。通读许可协议。按空格键显示下一页。读完许可协议后，输入 yes 继续安装（此步骤可能直接跳过）。

7.     JRE 安装在其自身的目录中。在本例中，它将安装在 /usr/java/ jre1.6.0_23目录中。当安装完成时，您将看到 Done（完成）字样。

1.     命令行键入“cd /etc”进入etc目录

2.     命令行键入“vi profile”打开profile文件

3.     敲击键盘ctrl+F到文件末尾

4.     在末尾处，即第一个~的地方，敲击键盘将以下内容输入到文件

JAVA_HOME=/usr/java/jre1.6.0_23

PATH=$JAVA_HOME/bin:$PATH

CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export JAVA_HOME

export PATH

export CLASSPATH

     5.     键盘敲击exc，命令行键入“:wq”，保存文件；

     6.     重新登录linux，在命令行键入“java”，如果出现帮助信息则表示配置成功。

然后，部署kettle

       1.         将pdi-ce-4.1.0-stable.zip上传到linux服务器

2.         使用unzip命令解压pdi-ce-4.1.0-stable.zip
unzip pdi-ce-4.1.0-stable.zip -d "/mnt/kettle"

3.         赋予解压目录下.sh文件的执行权力

chmod +x *.sh

4.         进入解压目录，键入“./kitchen.sh”回车，如果有帮助信息则表示kettle环境部署成功

※（重点）接着，编写执行kettle任务的shell脚本

创建test.sh，将以下内容复制粘贴到里面，然后键入“chmod +x /mnt/kettle/*.sh”，赋予test.sh执行权限。

export JAVA_HOME=/usr/java/jre1.6.0_23
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
/mnt/kettle/data-integration/kitchen.sh -file=/mnt/kettle/test.kjb > /mnt/kettle/display.txt

说明：
前3行，设置java环境变量的语句必不可少，
因为crond是个守护进程，它不归属于任何用户，虽然之前以root身份配置了java的环境变量，但是crond一样找不到java命令，
所以，当crond执行kettle任务时，需要动态设置java环境变量，crond才能找到java命令。

最后，使用crontab命令添加定时任务

       1.在终端上，键入“crontab -e”，进入定时任务文件
       2.键入“a”，对文件进行编辑
          输入如下内容： 

23 11 10 02 * /mnt/kettle/test.sh

        我在原有基础上加了个BOOL变量canChangeDirection，来控制在定时器每次时间间隔中，只能改变一次蛇的移动方向，这样就修复了以上BUG。

        以下为1.01版本的可执行文件及源代码的下载
        MySnake_v1.01

public class StrBinaryTurn {
    //将字符串转换成二进制字符串，以空格相隔
    public String toBinary(String str){
        char[] strChar = str.toCharArray();
        String result = "";
        for(int i = 0; i < strChar.length; i++){
            
            result += Integer.toBinaryString(strChar[i]) + " ";
        }
        return result;
        
    }
    //将二进制字符串转换成Unicode字符串
    public String toStr(String binStr){
        String[] tempStr = StrToStrArray(binStr);
        char[] tempChar = new char[tempStr.length];
        for(int i = 0; i < tempStr.length; i++){
            tempChar[i] = toChar(tempStr[i]);
        }
        return String.valueOf(tempChar);
    }
    //将二进制字符串转换为char
    private char toChar(String binStr){
        int[] temp = binStrToIntArray(binStr);
        int sum = 0;
        
        for(int i = 0; i < temp.length; i++){

            sum += temp[temp.length - 1 - i] << i;

        }
    
        return (char)sum;
        
    }
    //将初始二进制字符串转换成字符串数组，以空格相隔
    private String[] StrToStrArray(String str){
        return str.split(" ");
    }
    

    //将二进制字符串转换成int数组
    private int[] binStrToIntArray(String binStr){
        
        char[] temp = binStr.toCharArray();
        int[] result = new int[temp.length];
    
        for(int i = 0; i < temp.length; i++){
            result[i] = temp[i] - 48;
        }
        return result;
        
    }
    
    public static void main(String[] args){
        
        StrBinaryTurn cTob = new StrBinaryTurn();
        System.out.println(cTob.toBinary("橘子，好吃！aaa"));
        System.out.println();
        System.out.println(cTob.toBinary("999111"));
        System.out.println();
        System.out.println(cTob.toBinary("What a nice day!"));
        System.out.println();
        System.out.println(cTob.toBinary("^@^ - -! （）★"));
        
        System.out.println();
        System.out.println();
        
        System.out.println(cTob.toStr("110101001011000 101101101010000 " +
                "1111111100001100 101100101111101 " +
                "101010000000011 1111111100000001 " +
                "1100001 1100001 1100001 "));
        System.out.println(cTob.toStr("111001 111001 111001 " +
                "110001 110001 110001"));
        System.out.println(cTob.toStr("1010111 1101000 1100001 " +
                "1110100 100000 1100001 100000 1101110 " +
                "1101001 1100011 1100101 100000 1100100 " +
                "1100001 1111001 100001 "));
        System.out.println(cTob.toStr("1011110 1000000 1011110 " +
                "100000 101101 100000 101101 100001 100000 " +
                "1111111100001000 1111111100001001 10011000000101 "));
    }
}

110101001011000 101101101010000 1111111100001100 101100101111101 101010000000011 1111111100000001 1100001 1100001 1100001

111001 111001 111001 110001 110001 110001

1010111 1101000 1100001 1110100 100000 1100001 100000 1101110 1101001 1100011 1100101 100000 1100100 1100001 1111001 100001

1011110 1000000 1011110 100000 101101 100000 101101 100001 100000 1111111100001000 1111111100001001 10011000000101

橘子，好吃！aaa
999111
What a nice day!
^@^ - -! （）★

public enum MoveSuit {
        MOVELEFT,
        MOVERIGHT,
        MOVEDOWN,
        ROTATE,
        FALL
    }

    //移动方法,通过移动类型决定如何移动
    public synchronized void move(MoveSuit type, TetrisBoard tetrisBoard){
        
        switch(type){
        
        case MOVELEFT:
            moveLeft();
            break;
        case MOVERIGHT:
            moveRight();
            break;
        case MOVEDOWN:
            if(tetrisBoard.canMoveDown(this))
                moveDown();
            break;
        case ROTATE:
            clear();
            rotateClock().draw();
            break;
        case FALL:
            fall(tetrisBoard);
            break;
            
        }
    }

3.如果GameOver一次以后再TryAgain,游戏等级和方块下落间隔会在分数到达100分左右时突然变到刚才GameOver时的最高等级和方块下落间隔.

修正方法:
游戏等级和方块下落间隔都是通过消行总数判断的,之前我忘记再每次游戏开始时将消行总数设为0,所以只需要在TetrisBoard类中加一个setRemovedLines方法,在Game类中的startGame方法中调用setRemovedLines(0)即可.

4.每个小方块的边框原来我设为了白色,在两个大方块相临时会将其中一个大方块的一部分边框擦除,比较难看,如果把小方块长宽缩小一个象素,当两个大方块相临时,衔接处就会有两个边框,也不是很好看.

修正方法:将边框颜色设为背景色黑色,这样边框被擦除也看不出来,两个大方块衔接处只有一个边框,可以两全其美.虽然这个方法不太好...


BUG说完了,该说说新功能了,呵呵~

我又添加了一个可以记录并显示最好成绩信息的功能,玩游戏总得有个奔头么,嘿嘿!
第一次玩游戏时,最高分为0,最高等级为1,创造者姓名为"Are you the winner?"
当游戏gameover时,如果当前游戏分数大于最高分,就让玩家输入自己姓名,并将其和最高分数,等级加密后存入myTetris.dat.
重新打开游戏时,会从myTetris.dat中读取信息并解密,并将最高分,最高等级和最高分创造者显示在游戏界面上

最后添张图:


最后把jar包和源代码放出来!希望大家给我留言,互相交流,嘿嘿!
以前忘说了,如果想让帮助正常显示的话,需要在和jar包同级目录下新建个文件夹叫pics,然后在里面放一个叫logo的图片.
myTetris下载地址
谢谢大家对我的支持!