waterye

2008年10月21日 #

使用rsync+ssh同步n个文件

rsync -avz -e ssh bak@192.168.0.138:/home/res /home/res/bak

posted @ 2009-12-29 20:24 waterye 阅读(509) | 评论 (0)编辑 收藏

mysql备份

1. 使用Replication进行实时备份
参考mysql docs
2. 定期在slave上使用shell,mysqldump,sftp,crontab进行永久备份
#!/bin/sh
export d
=`date +%Y%m%d%H%M%S`
mkdir 
-/data/dbbak/baktables/$d
for i in `echo "show tables" | mysql -u bak -ppassword db|grep -v Tables`;
do    
  echo $i; mysqldump 
--add-drop-table --allow-keywords ----u bak -ppasswod mbook_hd $i > backup/$d/$i.sql
done

tar czf backup
/$d.tar.gz backup/$d/
rm 
-rf backup/$d/

lftp 
-"cd /dbbak/; mput -c $d.tar.gz; quit" -u bak,password sftp://192.168.0.138
参考http://ocaoimh.ie/simple-mysql-backup/
10G级别的可以每天作一次备份,100G级别看带宽和硬盘,T级没有经验

posted @ 2009-12-29 20:17 waterye 阅读(1040) | 评论 (0)编辑 收藏

检查哪些文件以\n结束

#!/usr/bin/env python
import string, os, sys  
   
dir 
= '/home/waterye/works/'  

files 
= os.listdir(dir)
for f in files:
    
if not os.path.isfile(dir+os.sep+f): continue
    linecount 
= 0
    fp 
= open(dir+os.sep+f,'r')
    
for line in fp:
        linecount 
+= 1
    fp 
= open(dir+os.sep+f,'r')
    
for i,line in enumerate(fp):
        
if (i==linecount-1 and line.endswith('\n')):
            
print f
            
print "--------------------------------"
* bash太难理解,还是用py看上去舒服,虽然不专业,但能完成任务就行

posted @ 2009-02-19 22:33 waterye 阅读(892) | 评论 (0)编辑 收藏

perl milliseconds

"perldoc DateTime" would have told you how to format milliseconds.

  
use DateTime;
  
use Time::HiRes qw(time);
  
my $dt = DateTime->from_epoch( epoch => time() );
  
print $dt->strftime('%Y-%m-%d %H-%M-%S-%3N'), "\n";

* sudo apt-get install libdatetime-perl

perl真麻烦

posted @ 2008-12-04 23:33 waterye 阅读(1059) | 评论 (0)编辑 收藏

memcached java client

1. http://www.whalin.com/memcached/ v2.0.1
2. http://code.google.com/p/spymemcached/ v2.2

a. 使用whalin版会导致File Descriptor leak,而使用spy版则不会,原因是whalin版没有使用selector管理socketchannel.
       Thread.sleep(1000 * 30);
        System.out.println(
"begin");

        Selector selector 
= null;
        SocketChannel channel 
= null;
        
try {
            String host 
= "192.168.0.74";
            
int port = 11211;
            
int timeout = 1000 * 60;

            selector 
= Selector.open();
            channel 
= SocketChannel.open();
            channel.configureBlocking(
false);

            channel.connect(
new InetSocketAddress(host, port));
            channel.register(selector, channel.validOps());

            
try {
                selector.select();
            } 
catch (IOException e) {
                e.printStackTrace();
            }

            Iterator it 
= selector.selectedKeys().iterator();
            
int i = 0;
            
while (it.hasNext()) {
                i
++;
                System.out.println(i);
                SelectionKey selKey 
= (SelectionKey) it.next();
                it.remove();

                
try {
                    processSelectionKey(selKey);
                } 
catch (IOException e) {
                    e.printStackTrace();
                    selKey.cancel();
                }
            }

            System.out.println(
"unclose");
            Thread.sleep(
1000 * 30);
        } 
catch (Exception e) {
            e.printStackTrace();
        } 
finally {
            
if (channel != null && channel.isOpen()) {
                
try {
                    channel.close();
                } 
catch (Exception e) {
                    e.printStackTrace();
                }
            }
            
if (selector!=null) {
                selector.close();      
// not fd leak
            }
        }

        System.out.println(
"end");
        Thread.sleep(
1000 * 30 * 1);
通过lsof -p pid | grep pipe可以观察是否有fd leak.

b. w版set 1000000 object 需要600s左右,s版只需150s左右

posted @ 2008-10-31 19:55 waterye 阅读(1533) | 评论 (2)编辑 收藏

cacti-memcached

192.168.0.74 [root avi]$ python /root/memcached/cacti-memcached-1.0/memcached.py localhost
total_items:2002344 get_hits:3 uptime:50291 cmd_get:3 time:1224521129 bytes:58888890 curr_connections:1 connection_structures:8 bytes_written:16167008 limit_maxbytes:402653184 cmd_set:2002344 curr_items:1000000 rusage_user:101.730357 get_misses:0 rusage_system:199.092442 bytes_read:63849044 total_connections:314

posted @ 2008-10-21 00:50 waterye 阅读(981) | 评论 (0)编辑 收藏