BlogJava--随笔分类-数据库

关于oracle中的timestamp和date类型

tacy lee — Tue, 25 Dec 2007 16:42:00 GMT

之前一直认为类似：where timestamp>date 这种子句是不走索引的

下面简单做一个验证：

c:>sqlplus / as sysdba
sys@EOS >create table test as select table_name,to_timestamp(last_analyzed) date_test from dba_tables;

表已创建。

sys@EOS> create index idx_test_date on test (date_test);

索引已创建。

sys@EOS> desc test
名称                                                  是否为空? 类型
----------------------------------------------------- -------- ----------------
--------------------
TABLE_NAME                                            NOT NULL VARCHAR2(30)
DATE_TEST                                                      TIMESTAMP(0)

sys@EOS> select date_test from test where date_test > TO_DATE('2007-11-5 00:00:00','yyyy-MM-dd HH24:mi:ss');

执行计划
----------------------------------------------------------
Plan hash value: 944171586

-------------------------------------------------------------------------------- --
| Id | Operation        | Name          | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------- --
|   0 | SELECT STATEMENT |               |     1 |    22 |     1   (0)| 00:00:01 |
|* 1 | INDEX RANGE SCAN| IDX_TEST_DATE |     1 |    22 |     1   (0)| 00:00:01 |
-------------------------------------------------------------------------------- --

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - access("DATE_TEST">TIMESTAMP'2007-11-05 00:00:00')

Note
-----
   - dynamic sampling used for this statement

统计信息
----------------------------------------------------------
          7 recursive calls
          0 db block gets
         18 consistent gets
          0 physical reads
          0 redo size
        280 bytes sent via SQL*Net to client
        374 bytes received via SQL*Net from client
          1 SQL*Net roundtrips to/from client
          0 sorts (memory)
          0 sorts (disk)
          0 rows processed

从上面可以清楚看到，timestamp>date情况下，走索引

纠正我之前的认识。

另外再补充一下，date这个数据类型一般情况下很少用，建议产品里面所有的date数据类型全部改为timestamp

tacy lee 2007-12-26 00:42 发表评论

db2诊断系列之---捕获sql执行情况

tacy lee — Sun, 25 Nov 2007 06:51:00 GMT

作者：tacy lee

在应用使用过程中，我们经常会碰到应用响应时间很慢，甚至没有响应，但是应用服务器可能并不是很繁忙，cpu利用率也非常低，引起这种状况的原因有很多种，比如环境问题，应用资源泄漏，数据库原因等等，本文主要是从一次应用性能诊断过程来谈谈如何通过数据库诊断应用性能问题。

问题：

测试过程中发现应用中某个跳转页面执行时间比较长，系统压力不大，cpu利用很低，该页面需要从cache中取数据，第一次的时候加载cache（从数据库中查询回数据并cache）。

诊断：

页面逻辑比较简单，我们先用loadrunner模拟并发测试一下这个页面，然后再数据库端捕获sql执行情况。

1、打开db2监控开关

#db2 connect to eos

#db2 update monitor switches using statement on

#db2 reset monitor all

2、几分钟之后，我们收集sql统计快照

#db2 get snapshot for dynamic sql on eos > dysqlstatus.out

现在统计信息已经存放在dysqlstatus.out中，你可以使用任意方便的文本处理工具查看，我一般用windows上的gvim来处理，打开dysqlstatus.out

Number of executions               = 1

Number of compilations             = 1

Worst preparation time (ms)        = 2

Best preparation time (ms)         = 2

Internal rows deleted              = 0

Internal rows inserted             = 0

Rows read                          = 2

Internal rows updated              = 0

Rows written                       = 0

Statement sorts                    = 0

Statement sort overflows           = 0

Total sort time                    = 0

Buffer pool data logical reads     = Not Collected

Buffer pool data physical reads    = Not Collected

Buffer pool temporary data logical reads   = Not Collected

Buffer pool temporary data physical reads  = Not Collected

Buffer pool index logical reads    = Not Collected

Buffer pool index physical reads   = Not Collected

Buffer pool temporary index logical reads  = Not Collected

Buffer pool temporary index physical reads = Not Collected

Total execution time (sec.ms)      = 0.000377

Total user cpu time (sec.ms)       = 0.010000

Total system cpu time (sec.ms)     = 0.000000

Statement text                     = select ACTIVITYDEFID,ACTIVITYINSTID from wfworkitem  where 
PROCESSINSTID=104199 and CURRENTSTATE = 4


......

简单说一下vi中的处理

:g!/Total execution time/d

只保留文本中的sql执行时间，我们要按照执行时间来排序



通过vim的visual功能选择执行时间块（等号后面的数字），然后排序

Total execution time (sec.ms)      = 0.050590

Total execution time (sec.ms)      = 0.000170

Total execution time (sec.ms)      = 0.000247

Total execution time (sec.ms)      = 0.000292

Total execution time (sec.ms)      = 0.000474

Total execution time (sec.ms)      = 0.000330

Total execution time (sec.ms)      = 0.000348

Total execution time (sec.ms)      = 0.000279

Total execution time (sec.ms)      = 0.000385

Total execution time (sec.ms)      = 0.000296

Total execution time (sec.ms)      = 0.000261

Total execution time (sec.ms)      = 0.000195

Total execution time (sec.ms)      = 0.000226

Total execution time (sec.ms)      = 0.000227

Total execution time (sec.ms)      = 0.000193

......

:'<,'>!sort



排序后的结果（部分）

Total execution time (sec.ms)      = 2.027776

Total execution time (sec.ms)      = 2.203624

Total execution time (sec.ms)      = 2.504677

Total execution time (sec.ms)      = 2.951256

Total execution time (sec.ms)      = 3.119875

Total execution time (sec.ms)      = 3.303277

Total execution time (sec.ms)      = 3.303517

Total execution time (sec.ms)      = 4.017133

Total execution time (sec.ms)      = 4.043329

Total execution time (sec.ms)      = 4.252125

Total execution time (sec.ms)      = 4.400952

Total execution time (sec.ms)      = 4.606765

Total execution time (sec.ms)      = 5.208087

Total execution time (sec.ms)      = 5.778598

Total execution time (sec.ms)      = 8.117470

Total execution time (sec.ms)      = 9797.905136

可以看到最长时间的sql total执行时间耗费了3797.905123s.

现在我们到dysqlstatus.out中去找这条语句

Number of executions               = 4602

Number of compilations             = 4294967295

Worst preparation time (ms)        = 2

Best preparation time (ms)         = 2

Internal rows deleted              = 0

Internal rows inserted             = 0

Rows read                          = 2963688

Internal rows updated              = 0

Rows written                       = 0

Statement sorts                    = 0

Statement sort overflows           = 0

Total sort time                    = 0

Buffer pool data logical reads     = Not Collected

Buffer pool data physical reads    = Not Collected

Buffer pool temporary data logical reads   = Not Collected

Buffer pool temporary data physical reads  = Not Collected

Buffer pool index logical reads    = Not Collected

Buffer pool index physical reads   = Not Collected

Buffer pool temporary index logical reads  = Not Collected

Buffer pool temporary index physical reads = Not Collected

Total execution time (sec.ms)      = 9797.905136

Total user cpu time (sec.ms)       = 9.290000

Total system cpu time (sec.ms)     = 1.230000

Statement text                     = select * from XXXX_T_CNFACTIVITYDEF

这条语句总共执行了4602次，平均每次的执行时间2S，而且这些数据应该是被cache起来的；）

总结：

上面的方法简单总结了从数据库层面对应用的性能问题诊断，希望对大家有所帮助，对于数据库快照诊断问题的思路对于任意数据库通用

补充一个unix上脚本处理方式：

sqlsort.sh

awk 'BEGIN{RS="";FS="\n";ORS="\n"};/Statement text/{print $1, $21, $24}' $1 | awk '$5 > 0 {print "AvgTime:", $11/$5, "\t", $0}'| sort -n | head -n $2|awk '{print $0, "\n"}'

使用：#sqlsort.sh dysqlstate.out 10（显示Top ten）

del.icio.us Tags: db2,java,tuning,diagnostic

tacy lee 2007-11-25 14:51 发表评论

db2诊断系列之---定位锁等待问题

tacy lee — Sat, 24 Nov 2007 13:18:00 GMT

作者：tacy lee

在应用中，我们经常会碰到sql执行很慢，但是数据库cpu和内存使用率又不高的情况，类似的问题基本上由于锁，排序等原因造成，本文主要描述如何去定位锁等待问题，谁在锁等待？等待谁持有的锁？锁在那个表？

一、测试准备

1、先在session1执行如下操作，创建测试表

#db2 connect to eos
#export DB2OPTIONS=+C
#db2 "create table tacy_test (a int not null primary key,b varchar(10))"
#db2 "insert into tacy_test values(1,'a')"
#db2 "insert into tacy_test values(2,'a')"
#db2 "insert into tacy_test values(3,'a')"
#db2 "insert into tacy_test values(4,'a')"
#db2 commit

2、在session2执行如下操作

#db2 connect to eos
#export DB2OPTIONS=+C

二、产生一个lock wait

在session1做一个表更新：

#db2 "update tacy_test set b='b' where a=4"

sql执行成功

在session2做同样更新操作：

#db2 "update tacy_test set b='c' where a=4"

进程被挂起等待

三、定位锁等待

1、先来看看应用的情况：

#db2pd -db eos -applications

Database Partition 0 -- Database EOS -- Active -- Up 0 days 07:37:37

Applications:
Address    AppHandl [nod-index] NumAgents  CoorPid    Status                  C-AnchID C-StmtUID  L-AnchID L-StmtUID  Appid                           
0x10140040 8        [000-00008] 1          8425       Lock-wait               80       2          66       1          *LOCAL.db2inst1.071124043739    
0x100CE540 7        [000-00007] 1          8358       UOW-Waiting             0        0          80       2          *LOCAL.db2inst1.071124043708

可以看到有一个应用的状态处于Lock-wait

2、现在我们来看看应用在等什么

#db2pd -db eos -locks showlock wait

Database Partition 0 -- Database EOS -- Active -- Up 0 days 07:42:56

Locks:
Address    TranHdl    Lockname                   Type       Mode Sts Owner      Dur HldCnt     Att Rlse
0x2C8E0760 3          02001806078066020000000052 Row        ..X  W   2          1   0          0   0x0  TbspaceID 2 TableID 1560 RecordID 0x2668007

锁的类型为Row（行锁），X锁（排他锁），下面是我们最关心的锁的位置

TbspaceID 2 TableID 1560 RecordID 0x2668007

其中TbspaceID为表空间ID，TableID为表的ID，RecordID代表具体位置，全部应该是0x0266807,其中前面三个字节为page number，为0x02668，后面一个字节代表solt identifier，为0x07

3、找到相应的表

#db2 "select tbspace,tabschema,tabname,tableid,tbspaceid from syscat.tables where tbspaceid=2 and tableid=1560"

TBSPACE       TABSCHEMA   TABNAME    TABLEID TBSPACEID
------------  ----------- ---------- ------- ---------
USERSPACE1    DB2INST1    TACY_TEST     1560         2

  1 record(s) selected.

4、根据RecordID找到锁在哪行

db2提供了一个强大的数据分析工具db2dart，可以dump出相应的page数据

#db2dart eos /dd /tsi 2 /oi 1560 /ps 157312p /np 1 /v y

Warning: The database state is not consistent.

Warning: Reorg rows MAY be due to the inconsistent state of the database.
                  DB2DART Processing completed with warning(s)!
                        Complete DB2DART report found in:
/home/db2inst1/sqllib/db2dump/DART0000/EOS.RPT

其中tsi为表空间id(2)，oi为表id(1560)，ps为page number(0x0266807)，需要转换为十进制，在结尾必须加p，np代表你要获取的页数，v为是否详细输出

现在我们来看看EOS.RPT

______________________________________________________________________________

        _______                    DART                   _______ 

   D a t a b a s e   A n a l y s i s   a n d   R e p o r t i n g   T o o l

                           IBM    DB2    6000
______________________________________________________________________________

DART (V8.1.0)  Report:
2007-11-24-20.59.51.355893

            Database Name: EOS
            Report name: EOS.RPT
            Old report back-up: EOS.BAK
            Database Subdirectory: /opt/db2/db2inst1/NODE0000/SQL00001
            Operational Mode: Database Inspection Only (INSPECT)

______________________________________________________________________________
------------------------------------------------------------------------------


Action option: DD 
Table-object-ID: 1560; Tablespace-ID: 2; First-page: 157312p; Number-pages: 1; Verbose: y

Warning: The database state is not consistent.

Warning: Reorg rows MAY be due to the inconsistent state of the database.
Connecting to Buffer Pool Services...

   Table object report phase start.
   Dump format is verbose.

                           ______________________________________

         Page 0 of object 1560 from table space 2.

         BPS Page Header:

                     Page Data Offset = 48
                     Page Data Length = 4048
                             Page LSN = 0000 AE97 AE41
                   Object Page Number = 0
                     Pool Page Number = 157312
                            Object ID = 1560
                          Object Type = Data Object

               Data Page Header:

                           Slot Count = 8
                     Total Free Space = 2784
                  Total Reserve Space = 0
               Youngest Reserve Space = n/a
                         Youngest TID = n/a
                    Free Space Offset = 2799
                  Maximum Record Size = 23

               Data Records:


            Slot 0:

               Offset Location = 3996  (xF9C)
               Record Length = 32  (x20)

               Record Type = Data Object Header Control Record

                  Page count = 1
         Object Creation LSN = 0000 AE97 800C
                Object State = x0000
          UDI Since Runstats = 0
                  DART Field = x00000000

            Slot 1:

               Offset Location = 2992  (xBB0)
               Record Length = 1004  (x3EC)

               Record Type = Free Space Control Record

               Free space entries:
                 0:  2884 (x0B44),  4028 (x0FBC),  4028 (x0FBC),  4028 (x0FBC)
                 4:  4028 (x0FBC),  4028 (x0FBC),  4028 (x0FBC),  4028 (x0FBC)
                 8:  4028 (x0FBC),  4028 (x0FBC),  4028 (x0FBC),  4028 (x0FBC)
               省略。。。
                  492:  4028 (x0FBC),  4028 (x0FBC),  4028 (x0FBC),  4028 (x0FBC)
                  496:  4028 (x0FBC),  4028 (x0FBC),  4028 (x0FBC),  4028 (x0FBC)

            Slot 2:

               Offset Location = 2916  (xB64)
               Record Length = 76  (x4C)

               Record Type = Table Directory Record

                  MetaIndex Root Page = 157377
                  Index Type = 2
                  Table Descriptor Pointer  --  Page 157312  Slot 3
                  Max Insert Search = 0
                  Flags = x02000200
                     bit representation = 00000010 00000000 00000010 00000000
                  Check pending info:
                     Constraint status    = x00
                     Constraint RID       = Page 0 Slot 0
                     last BID          = x00000000

            Slot 3:

               Offset Location = 2892  (xB4C)
               Record Length = 24  (x18)

               Record Type = Table Description Record

                  Number of Columns = 2


                  Column 1:
                  Type is Long Integer
                  Length = 4
                  Prohibits NULLs
                  Prohibits Default
            Fixed offset: 0

                  Column 2:
                  Type is Fixed Length Character String
                  Length = 10
                  Allows NULLs
                  Prohibits Default
            Fixed offset: 4

            Slot 4:

               Offset Location = 2869  (xB35)
               Record Length = 23  (x17)

               Record Type = Table Data Record (FIXEDVAR)

               Fixed part length value = 15

                  Column 1:
            Fixed offset: 0
                  Type is Long Integer
                  Value = 1

                  Column 2:
            Fixed offset: 4
                  Type is Fixed Length Character String
                      61202020 20202020 2020                 a               


            Slot 5:

               Offset Location = 2846  (xB1E)
               Record Length = 23  (x17)

               Record Type = Table Data Record (FIXEDVAR)

               Fixed part length value = 15

                  Column 1:
            Fixed offset: 0
                  Type is Long Integer
                  Value = 2

                  Column 2:
            Fixed offset: 4
                  Type is Fixed Length Character String
                      61202020 20202020 2020                 a               


            Slot 6:

               Offset Location = 2823  (xB07)
               Record Length = 23  (x17)

               Record Type = Table Data Record (FIXEDVAR)

               Fixed part length value = 15

                  Column 1:
            Fixed offset: 0
                  Type is Long Integer
                  Value = 3

                  Column 2:
            Fixed offset: 4
                  Type is Fixed Length Character String
                      61202020 20202020 2020                 a               


            Slot 7:

               Offset Location = 2800  (xAF0)
               Record Length = 23  (x17)

               Record Type = Table Data Record (FIXEDVAR)

               Fixed part length value = 15

                  Column 1:
            Fixed offset: 0
                  Type is Long Integer
                  Value = 4

                  Column 2:
            Fixed offset: 4
                  Type is Fixed Length Character String
                      61202020 20202020 2020                 a               


         Slots Summary:  Total=8,  In-use=8,  Deleted=0.

      
   Table object report phase end.
                     ______________________________________

                  DB2DART Processing completed with warning(s)!
                     Warning(s) detected during processing.
                     ______________________________________

                        Complete DB2DART report found in:
/home/db2inst1/sqllib/db2dump/DART0000/EOS.RPT
    _______    D A R T    P R O C E S S I N G    C O M P L E T E    _______

找到Solt 7 (0x07)，ok，你现在可以清楚的知道应用等待的Row为(4,a)

总结

通过上面的方法，我们简单描述了一个db2锁问题的定位方法，希望能给大家在分析和定位应用性能问题的时候起到一定的帮助

del.icio.us Tags: db2,lock,tuning,diagnostic

tacy lee 2007-11-24 21:18 发表评论

捕获DB2 sql的执行快照

tacy lee — Tue, 30 Oct 2007 04:59:00 GMT

先建立一个监控器

db2 "create event monitor SQLCOST for statements write to file '/home/db2inst1'"

再设置事务状态为打开

db2 "set event monitor SQLCOST state=1"

注：1为打开，0为关闭，收集数据之后，记得关闭你的监控器，否则。。。

跑你的测试后，在你的/home/db2inst1目录下会生成一些evm文件

用下面指令获取诊断信息：

db2evmon -db eos51 -evm SQLCOST>sqlcost1.txt

完成之后删除你的监控器

db2 "drop event monitor SQLCOST"

生成的采样例子，从下面的例子中，你可以清除的看到SQL执行的时间，CPU消耗情况，排序是否溢出，BufferPool的使用情况，根据这些信息，SQL的执行效率一目了然：

26) Statement Event ...

Appl Handle: 336

Appl Id: C0A80421.O905.0ABDA5065446

Appl Seq number: 0657

Record is the result of a flush: FALSE

-------------------------------------------

Type : Dynamic

Operation: Execute

Section : 7

Creator : NULLID

Package : SYSSN300

Consistency Token : SYSLVL01

Package Version ID :

Cursor : SQL_CURSN300C7

Cursor was blocking: FALSE

Text : update WFProcessInst set relateData=? where processInstID= ?

-------------------------------------------

Start Time: 04/25/2007 14:57:19.402248

Stop Time: 04/25/2007 14:57:19.409622

Exec Time: 0.007374 seconds

Number of Agents created: 1

User CPU: 0.000000 seconds

System CPU: 0.000000 seconds [licl1]

Fetch Count: 0

Sorts: 0

Total sort time: 0

Sort overflows: 0 [licl2]

Rows read: 1

Rows written: 1

Internal rows deleted: 0

Internal rows updated: 0

Internal rows inserted: 0

Bufferpool data logical reads: 9

Bufferpool data physical reads: 0

Bufferpool temporary data logical reads: 0

Bufferpool temporary data physical reads: 0

Bufferpool index logical reads: 3

Bufferpool index physical reads: 0

Bufferpool temporary index logical reads: 0

Bufferpool temporary index physical reads: 0 [licl3]

SQLCA:

sqlcode: 0

sqlstate: 00000

[licl1]SQL执行时间和CPU消耗情况

[licl2]SQL的排序情况，可以看到这个SQL没有排序，当然也没有排序溢出

[licl3]Bufferpool的使用情况，逻辑读和物理读的对比

del.icio.us Tags: database , db2 , tuning

tacy lee 2007-10-30 12:59 发表评论

认识oracle数据类型

tacy lee — Fri, 07 Sep 2007 09:10:00 GMT

摘要: 作者：tacy lee

在设计数据库的时候，选择正确的数据类型，往往可以避免很多的问题，正确理解数据库的类型，对于存储空间规划，应用性能调整都会很有帮助，下面是我个人的一点总结阅读全文

tacy lee 2007-09-07 17:10 发表评论