2008年1月31日
#
rsync -avz -e ssh bak@192.168.0.138:/home/res /home/res/bak
1. 使用Replication进行实时备份
参考mysql docs
2. 定期在slave上使用shell,mysqldump,sftp,crontab进行永久备份
#!/bin/sh
export d=`date +%Y%m%d%H%M%S`
mkdir -p /data/dbbak/baktables/$d
for i in `echo "show tables" | mysql -u bak -ppassword db|grep -v Tables`;
do
echo $i; mysqldump --add-drop-table --allow-keywords -q -a -c -u bak -ppasswod mbook_hd $i > backup/$d/$i.sql
done
tar czf backup/$d.tar.gz backup/$d/
rm -rf backup/$d/
lftp -e "cd /dbbak/; mput -c $d.tar.gz; quit" -u bak,password sftp://192.168.0.138
参考http://ocaoimh.ie/simple-mysql-backup/
10G级别的可以每天作一次备份，100G级别看带宽和硬盘，T级没有经验
#!/usr/bin/env python
import string, os, sys
dir = '/home/waterye/works/'
files = os.listdir(dir)
for f in files:
if not os.path.isfile(dir+os.sep+f): continue
linecount = 0
fp = open(dir+os.sep+f,'r')
for line in fp:
linecount += 1
fp = open(dir+os.sep+f,'r')
for i,line in enumerate(fp):
if (i==linecount-1 and line.endswith('\n')):
print f
print "--------------------------------"
* bash太难理解，还是用py看上去舒服，虽然不专业，但能完成任务就行
"perldoc DateTime" would have told you how to format milliseconds.
use DateTime;
use Time::HiRes qw(time);
my $dt = DateTime->from_epoch( epoch => time() );
print $dt->strftime('%Y-%m-%d %H-%M-%S-%3N'), "\n";
* sudo apt-get install libdatetime-perl
perl真麻烦
1. http://www.whalin.com/memcached/ v2.0.1
2. http://code.google.com/p/spymemcached/ v2.2
a. 使用whalin版会导致File Descriptor leak,而使用spy版则不会，原因是whalin版没有使用selector管理socketchannel.
Thread.sleep(1000 * 30);
System.out.println("begin
");
Selector selector = null;
SocketChannel channel = null;
try {
String host = "192.168.0.74";
int port = 11211;
int timeout = 1000 * 60;
selector = Selector.open();
channel = SocketChannel.open();
channel.configureBlocking(false);
channel.connect(new InetSocketAddress(host, port));
channel.register(selector, channel.validOps());
try {
selector.select();
} catch (IOException e) {
e.printStackTrace();
}
Iterator it = selector.selectedKeys().iterator();
int i = 0;
while (it.hasNext()) {
i++;
System.out.println(i);
SelectionKey selKey = (SelectionKey) it.next();
it.remove();
try {
processSelectionKey(selKey);
} catch (IOException e) {
e.printStackTrace();
selKey.cancel();
}
}
System.out.println("unclose");
Thread.sleep(1000 * 30);
} catch (Exception e) {
e.printStackTrace();
} finally {
if (channel != null && channel.isOpen()) {
try {
channel.close();
} catch (Exception e) {
e.printStackTrace();
}
}
if (selector!=null) {
selector.close(); // not fd leak
}
}
System.out.println("end");
Thread.sleep(1000 * 30 * 1);
通过lsof -p pid | grep pipe可以观察是否有fd leak.
b. w版set 1000000 object 需要600s左右，s版只需150s左右
192.168.0.74 [root avi]$ python /root/memcached/cacti-memcached-1.0/memcached.py localhost
total_items:2002344 get_hits:3 uptime:50291 cmd_get:3 time:1224521129 bytes:58888890 curr_connections:1 connection_structures:8 bytes_written:16167008 limit_maxbytes:402653184 cmd_set:2002344 curr_items:1000000 rusage_user:101.730357 get_misses:0 rusage_system:199.092442 bytes_read:63849044 total_connections:314
1. iftop
sudo apt-get install iftop
sudo iftop -i ppp0
2. slurm
sudo apt-get install slurm
slurm -i ppp0
3. vnstat
sudo apt-get install vnstat
sudo chmod o+x /usr/bin/vnstat
sudo chmod o+wx /var/lib/vnstat/
vnstat -u -i ppp0
vnstat -i ppp0
vnstat -l -i ppp0
因为不是server就没有用复杂的mrtg
vi,gedit会在最后一行加上\n,实在找不到好的解决方法,只好自己写个shell script删掉
#!/usr/bin/env python
import sys
oldfile=sys.argv[1]
newfile=sys.argv[2]
print oldfile,newfile,
print
linecount = 0
str = ''
f = open(oldfile,'r')
for line in f:
linecount += 1
f = open(oldfile,'r')
for i,line in enumerate(f):
if (i==linecount-1):
str += line.replace('\n','')
# print line.replace('\n','')
else:
str += line
# print line,
#print len(str),str
f2 = open(newfile, 'w')
f2.write(str)
quit()
free -lmt
cat /proc/meminfo
dmesg |grep [mM][eE][mM]
cat /proc/pid/status
1. find . -type f -exec rm -v {} \;
2. nice -n 19 rm -rf directory
3. ls | xargs rm
4. find . -mtime +90 -type f -exec rm -v {} \;
采用第四种最好，把一些旧的文件先删除
shell
mysql -h 192.168.0.136 -P 3306 -N -u root -p'abcd' -e 'show full processlist' | egrep -v 'Sleep|show full processlist'
jsp
String sql = " show full processlist ";
rs = stmt.executeQuery(sql);
while (rs.next()) {
String id = rs.getString("Id");
String user = rs.getString("User");
String host = rs.getString("Host");
String db = rs.getString("db");
String command = rs.getString("Command");
String info = rs.getString("Info");
String state = rs.getString("State");
if (command.equalsIgnoreCase("Sleep")) continue;
if (info.equalsIgnoreCase("show full processlist")) continue;
if (info.length()==0 && state.length()==0) continue;
int time = rs.getInt("Time");
// print
}
nbtstat.exe -a 192.168.0.171
nbtscan-1.0.35.exe -f 192.168.0.1/24
http://www.unixwiz.net/tools/nbtscan.html
在/root/.bashrc增加一行
export PS1="\[ \033[0;32;40m\u@\h:\w\$ \033[0m \]"
---------------------------------------------------------------------
首先使用一个例子来说明如何实现Shell彩色：
PS1="\[ \033[0;32;40m\u@\h:\w\$ \033[0m \]"
在上面命令中，“\033[0;32;40m”表示定义后面文本的颜色，“\033”表示启动一个转义序列，“[”定义开始颜色定义，“0”定义默认的字体颜色，其它部分稍后定义。
“32”定义文本的前景色，这里32表示绿色；“40”定义文本的背景色，40表示黑色。
在字符串的最后定义了“ \033[0m”，它是用来恢复了默认的文本颜色设置，这样就只会得到一个彩色提示符，而不会影响命令和其输出的颜色显示（即黑底白字）。
我们一共有8种字体颜色可供选择，它们分别是30 (黑色)、31 (红色)、32 (绿色)、33 (黄色)、34 (蓝色)、35 ( 紫红色)、36 (青色)和37 (白色)。
对于底色也有8种颜色可供选择，只需要将字体颜色的3修改为4即可，例如40、41、42、43、44、45、46、47。
文本属性
我们前面提到，转义序列符后面的“0”表示定义文本的颜色设置。
除了颜色设置以外，还可以设置文本的其它属性。
转义序列符后可以跟以下数值：0、1、22、4、24、5、25、7、27，分别定义颜色、黑体、非黑体、下画线、非下画线、闪烁、非闪烁、翻转、非翻转。
出处: http://www.cppblog.com/Bugs/archive/2008/06/24/54442.html
CustomLog "|/usr/local/apache2/bin/rotatelogs /usr/local/apache2/logs/access.log.%Y.%m.%d 86400 480" common # one day per file
java.awt.image.BufferedImage bufi = javax.imageio.ImageIO.read(bais);
javax.imageio.ImageIO.write(bufi, "jpg", baos);
check_mysql!-H 192.168.0.88 -P 3308 -u check -p checkpwd
check_mysql!-H 192.168.0.99 -P 3308 -u check -p checkpwd -S ; check mysql slave server
check_mysql_query!-H 192.168.0.99 -P 3308 -u check -p checkpwd -q "select count(*) from table_name" -w 100 -c 200
check_http!-p 8080
check_http!-u http://192.168.0.188:8080/a.jsp
/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/sda1
/usr/local/nagios/libexec/check_users -w 5 -c 10
/usr/local/nagios/libexec/check_procs -w 150 -c 200
/usr/local/nagios/libexec/check_procs -a 'instance_name=app.task' -c 1:1
/usr/local/nagios/libexec/check_mrtgtraf -F /home/mrtg/logs/192.168.0.188_1.log -a AVG -w 1000000,2000000 -c 5000000,5000000 -e 10
Map<String, Integer> m = new TreeMap<String, Integer>();
m.put(key, value);
Set<Map.Entry<String, Integer>> treeSet = new TreeSet<Map.Entry<String, Integer>>(
new Comparator<Map.Entry<String, Integer>>() {
public int compare(Map.Entry<String, Integer> o1, Map.Entry<String, Integer> o2) {
Integer d1 = o1.getValue();
Integer d2 = o2.getValue();
int r = d2.compareTo(d1);
if (r!=0) return r;
else return o2.getKey().compareTo(o1.getKey());
}
});
treeSet.addAll(m.entrySet());
for (Map.Entry me : treeSet) {
System.out.println(me.getKey() + "," + me.getValue());
}
awk '{ if (index($4,"ip")>0) { split($4,a,","); if (substr(a[2],11)>0) print substr(a[2],11) } }' access.log.txt.2008-07-26 > ip.20080726.log
awk '{a[$0]++}END{for(i in a){print a[i] " " i}}' ip.20080726.log | sort -rn | head -n 300
awk '{a[$0]++}END{for(i in a){ j++;} } END { print j }' ip.20080726.log
The GNU Awk User's Guide
:%s/oldstr/newstr/g 替换所有
:$ 跳到文件最后一行
yy命令复制当前整行的内容到vi缓冲区
粘贴缓冲区中的内容，用p
key_buffer_size + tmp_table_size + (read_buffer_size + sort_buffer_size + read_rnd_buffer_size +join_buffer_size )*max_connections = 21215973024
大于物理内存16G,所以导致out of memory
最近mysql(5.0.45)偶尔出现"Got error 127 when reading table"的错误.原来是bug,在5.0.51已fix.
Release Notes for MySQL Community Server 5.0.51
If one thread was performing concurrent inserts, other threads reading from the same table using equality key searches could see the index values for new rows before the data values had been written, leading to reports of table corruption. (
Bug#29838)
tail -f sql.log | grep 'table_name', 非常快
tail -f sql.log | grep 'table_name' | awk '{print substr($0,0,150)}',非常慢
select table_name, engine, row_format, table_rows, data_length, index_length, max_data_length, data_free
from information_schema.tables
order by table_rows desc, data_length desc limit 100
mysql 5.x 增加information_schema
VisualVM is a visual tool that integrates several existing JDK software tools and lightweight memory and CPU profiling capabilities. This tool is designed for both production and development time use and further enhances the capability of monitoring and performance analysis for the Java SE platform.
目前还是用jconsole远程观察jvm. visualvm不知是否像听上去那么好.此项目还在beta2状态.好像dev.java.net的项目都不怎么样.
Adobe AIR is a cross-operating system runtime that enables you to use your existing HTML/Ajax, Flex, or Flash web development skills and tools to build and deploy rich Internet applications to the desktop.
air就像客户端的jvm,现在大家都喜欢提供api给developer,希望air能更加open一些,或许能获得像jvm一样的成功.
fisheye: 代码仓库查看工具. changelog非常方便查看团队成员提交的代码. 当然还有rss功能. search也非常强大. 不过statistic功能就比较简单些.
show variables like 'table_cache'
show variables like 'key_buffer_size'
show variables like 'thread_cache_size'
show variables like 'tmp_table_size'
SHOW STATUS LIKE 'open%tables%';
SHOW STATUS LIKE '%key_read%';
SHOW STATUS LIKE '%qc%';
SHOW STATUS LIKE '%Threads_created%';
SHOW STATUS LIKE '%Created_tmp_disk_tables%';
set GLOBAL max_connections = 2048
set GLOBAL table_cache=2048
set GLOBAL key_buffer_size=4294967296;
set GLOBAL query_cache_size=536870912;
set GLOBAL thread_cache_size=400;
http://trac.symfony-project.com/wiki/MysqlOptimization