BlogJava-miaoyachun-随笔分类-MySQL

SQL连接查询

Milo的海域 — Wed, 04 Jul 2012 01:35:00 GMT

关于SQL连接查询的图形表示SQL连接查询

Milo的海域 2012-07-04 09:35 发表评论

About cannot core dump after setuid on linux server

Milo的海域 — Wed, 30 May 2012 07:32:00 GMT

In many cases, binaries can no longer dump core after calling setuid(). 
Under Linux it is possible to re-enable this with a system call.

e.g.

+#ifdef __linux__
+#include <sys/prctl.h>
+#endif
+
#ifdef HAVE_purify
#define IF_PURIFY(A,B) (A)
#else
@@ -1362,6 +1366,10 @@
     sql_perror("setuid");
     unireg_abort(1);
   }
+#ifdef __linux__
+  /* inform kernel that process is dumpable */
+  prctl(PR_SET_DUMPABLE,1,0,0,0);
+#endif /* __linux__ */
#endif

Manual of prctl

       PR_SET_DUMPABLE
              (Since Linux 2.4) Set the state of the flag determining whether core dumps are produced for this process  upon  delivery  of  a
              signal  whose  default  behaviour  is  to  produce a core dump.  (Normally this flag is set for a process by default, but it is
              cleared when a set-user-ID or set-group-ID program is executed and also by various system calls that  manipulate  process  UIDs
              and  GIDs).  In kernels up to and including 2.6.12, arg2 must be either 0 (process is not dumpable) or 1 (process is dumpable).
              Since kernel 2.6.13, the value 2 is also permitted; this causes any binary which normally would not  be  dumped  to  be  dumped
              readable by root only.  (See also the description of /proc/sys/fs/suid_dumpable in proc(5).)



Ref: http://bugs.mysql.com/bug.php?id=21723
Some files about linux core dump:

/proc/sys/fs/suid_dumpable
/etc/profile
/etc/security/limits.conf
/proc/sys/kernel/core_pattern

Milo的海域 2012-05-30 15:32 发表评论

sysbench 0.5介绍

Milo的海域 — Mon, 14 May 2012 08:10:00 GMT

0.5版本的sysbench已经支持multi-table

Download src code

bzr branch lp:sysbench

make后就可以执行了

Percona对这个版本的sysbench的参数有个不错的wiki page:
http://www.percona.com/docs/wiki/benchmark:sysbench:olpt.lua

下面这个文章对0.5的sysbench有个比较全面的介绍:

./sysbench --mysql-host=$host1 --mysql-port=3306 --mysql-user=*** --mysql-password=*** --test=/path/to/sysbench/tests/db/oltp.lua --oltp-tables-count=$oltp_table_num --num-threads=$num_thread --max-requests=0 --max-time=$max_time prepare

./sysbench --mysql-host=$host1 --mysql-port=3306 --mysql-user=*** --mysql-password=*** --test=/path/to/sysbench/tests/db/oltp.lua --oltp-tables-count=$oltp_table_num --num-threads=$num_thread --max-requests=0 --max-time=$max_time run

补充下，如果要使用 --max-time这个参数，需要配合--max-requests=0。

--max-requests=N limit for total number of requests [10000]

如果N=0将取消max-requests的限制

Milo的海域 2012-05-14 16:10 发表评论

About mysqldump of MySQL 5.5 backup information_schema & performance_schema databases

Milo的海域 — Fri, 11 May 2012 06:01:00 GMT

MySQL 5.5多出了两个新数据库 information_schema ＆ performance_schema

但是用mysqldump去备份这两个数据库的时候都会报错

mysqldump: Got error: 1044: Access denied for user 'root'@'localhost' to database 'information_schema' when using LOCK TABLES

mysqldump: Got error: 1142: SELECT,LOCK TABL command denied to user 'root'@'localhost' for table 'cond_instances' when using LOCK TABLES

用--all-databases备份时也不会备份这两个数据库
官方文档里解释到：

http://dev.mysql.com/doc/refman/5.5/en/mysqldump.html

总结一下，如果一定要用mysqldump去备份这两个数据库不是不可以，但是得disable lock tables。我用的是：

--database information_schema --lock-tables=0

Milo的海域 2012-05-11 14:01 发表评论

几个ssh管道用法

Milo的海域 — Wed, 09 May 2012 09:13:00 GMT

原来ssh可以这样用

1. remote file copy

[root@xen74v01 ~]# cat test.pl
#!/usr/bin/perl
print "eth0.74"=~/(\w+)/;
print "\n";
[root@xen74v01 ~]# cat test.pl | ssh 10.1.74.76 'cat - > /tmp/test.pl'

拷贝文件时，如果文件很大，又不想影响网络IO可以用pv工具进行流量控制

pv -L10m test.pl | ssh 10.1.74.76 'cat - > /tmp/test.pl'

这里pv的行为跟cat比较类似，但是支持IO流量控制，这里设置10M/s.

2. local script remote execute

[root@xen74v01 ~]# cat test.pl
#!/usr/bin/perl
print "eth0.74"=~/(\w+)/;
print "\n";
[root@xen74v01 ~]# perl test.pl
eth0
[root@xen74v01 ~]# cat test.pl | ssh 10.1.74.76 'perl'
eth0
[root@xen74v01 ~]# ssh 10.1.74.76 'perl' < test.pl
eth0

这样就不用把脚本拷贝到远端去执行了

参考：

http://linux.icydog.net/ssh/piping.php

http://www.ivarch.com/programs/quickref/pv.shtml

http://www.mysqlperformanceblog.com/2009/05/20/hint-throttling-xtrabackup/

Milo的海域 2012-05-09 17:13 发表评论

innobackupex & FLUSH TABLES WITH READ LOCK

Milo的海域 — Tue, 10 Apr 2012 09:19:00 GMT

当innobackupex 做全备的时候(my version 1.6.5), 当备份到MyISAM数据时, innobackupex 会flush tables with read lock, 来禁止MyISAM的写操作. (假设没有--no-lock选项)

sub backup {

    if (!$option_incremental && !$option_no_lock) {
        # make a prep copy before locking tables, if using rsync
        backup_files(1);

        # flush tables with read lock
        mysql_lockall();
    }

    if ($option_slave_info) {
        write_slave_info();
    }

}

sub mysql_lockall {

    if (compare_versions($mysql_server_version, '4.0.22') == 0
        || compare_versions($mysql_server_version, '4.1.7') == 0) {
        # MySQL server version is 4.0.22 or 4.1.7
        mysql_send "COMMIT;";
        mysql_send "FLUSH TABLES WITH READ LOCK;";
    } else {
        # MySQL server version is other than 4.0.22 or 4.1.7
        mysql_send "FLUSH TABLES WITH READ LOCK;";
        mysql_send "COMMIT;";
    }
    write_binlog_info;

}

但是如果备份的时候还有很重的workload, "flush tables with read lock" 可能会比较耗时. 这里参考了：

http://www.mysqlperformanceblog.com/2010/04/24/how-fast-is-flush-tables-with-read-lock/

看了下--no-lock的选项说明：

    --no-lock
        Use this option to disable table lock with "FLUSH TABLES WITH READ
        LOCK". Use it only if ALL your tables are InnoDB and you DO NOT CARE
        about the binary log position of the backup.

如果我们能保证workload仅仅是innodb相关的，我们可以使用这个选项。

记得在1.5版本的时候，使用--no-lock选项会导致xtrabackup_slave_info没有保存备份时的logfile & pos. 这个问题在1.6.5被解决了

    if ($option_slave_info) {
        write_slave_info();
    }

xtrabackup_slave_info & xtrabackup_binlog_info文件在1.5版本是在mysql_lockall函数里更新的。但是新版本已经把write_slave_info提到mysql_lockall外面了。

Milo的海域 2012-04-10 17:19 发表评论

innobackupex 1.6.4 need empty datadir when copy-back

Milo的海域 — Tue, 03 Apr 2012 05:01:00 GMT

When restore by xtrabackup '--copy-back' with version 1.6.4, you may get error:

IMPORTANT: Please check that the copy-back run completes successfully.
At the end of a successful copy-back run innobackupex-1.6.4
prints "completed OK!".

Original data directory is not empty! at innobackupex-1.6.4 line 544.

when read those lines we found that:

    # check that original data directories exist and they are empty
    if_directory_exists_and_empty($orig_datadir, "Original data");
    if_directory_exists_and_empty($orig_ibdata_dir, "Original InnoDB data");
    if_directory_exists_and_empty($orig_iblog_dir, "Original InnoDB log");

Google for reason for this check and find it mentioned in:

http://www.mysqlperformanceblog.com/2011/12/19/percona-xtrabackup-1-6-4/

innobackupex did not check that MySQL datadir was empty before –copy-back was run. With this bug fix,
innobackupex will now error out of the –copy-back operation if the destination is not empty, avoiding
potential data loss or a strang combination of a restored backup and previous data. Bug Fixed: #737569 (Valentine Gostev)

Milo的海域 2012-04-03 13:01 发表评论

Setup MySQL CDT DEV Enviroment

Milo的海域 — Fri, 23 Mar 2012 09:49:00 GMT

1. install bzr tool, ref http://dev.mysql.com/doc/refman/5.1/en/installing-development-tree.html
2. download mysql 5.1 code from trunk
3. autoreconf --force --install
4. ./configure --with-debug --without-libedit --with-plugins=innobase
5. make & sudo make install
6. setup eclipse cdt env
   a. startup eclipse by sudo cmd or root user
   b. build project
   c. set debug diag, ref http://forge.mysql.com/wiki/Eclipse/CDT_on_Linux_and_Mac_OS_X, here is my "program parameters" when startup mysqld instance:

--datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --port=3306 --socket=/tmp/mysql.sock --default-storage-engine=innodb

Milo的海域 2012-03-23 17:49 发表评论

How to use sysbench test mysql

Milo的海域 — Mon, 19 Mar 2012 08:35:00 GMT

# parepare test database
createDB="drop database if exists $database; create database if not exists $database;"
mysql -h $vip --port=$port -uadmin -padmin -e "$createDB"

# prepare data
prepare="$sysbench --test=oltp --mysql-table-engine=innodb --oltp-table-size=$row"
prepare="$prepare --mysql-port=$port --mysql-host=$vip --mysql-db=$database"
prepare="$prepare --mysql-password=admin --mysql-user=admin prepare"
$prepare

# run sysbench
# http://sysbench.sourceforge.net/docs/#database_mode see this link for more options
run="$sysbench --test=oltp --mysql-table-engine=innodb --init-rng --oltp-table-size=$row"
run="$run --max-requests=0 --max-time=900 --num-threads=128 --oltp-test-mode=complex"
run="$run --oltp-point-selects=2 --oltp-simple-ranges=1 --oltp-sum-ranges=2"
run="$run --oltp-index-updates=10 --oltp-non-index-updates=5 --mysql-port=$port"
run="$run --mysql-db=$database --mysql-host=$vip --mysql-password=admin --mysql-user=admin"

# support oltp-user-delay-min to add delay for each sysbench request
if [[ "$lag" != "nolag" ]]
then
run="$run --oltp-user-delay-min=$lag"
fi
run="$run run"

Milo的海域 2012-03-19 16:35 发表评论

innodb_flush_log_at_trx_commit option

Milo的海域 — Thu, 23 Feb 2012 08:28:00 GMT

Official ref: innodb_flush_log_at_trx_commit

这个参数的配置涉及trax提交写trax log文件的行为

innodb_flush_log_at_trx_commit = 0
# 每秒写一次trax log,并执行fsync

innodb_flush_log_at_trx_commit = 1
# 每次trax 提交的时候写一次trax log, 并执行fsync

innodb_flush_log_at_trx_commit = 2
# 每次trax 提交的时候写一次trax log, 不会执行fsync

Milo的海域 2012-02-23 16:28 发表评论

指定mysql默认字符集

Milo的海域 — Tue, 31 Jan 2012 05:18:00 GMT

从源码安装mysql时指定默认字符集：

./configure ... '--with-charset=utf8' '--with-collation=utf8_general_ci' ...
make & make install
start mysql instance

验证：

也可以在my.cnf里面修改默认字符集：

default-character-set=latin1

Milo的海域 2012-01-31 13:18 发表评论