随笔-193  评论-715  文章-1  trackbacks-0

What is Memcached?

Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

Memcached is simple yet powerful. Its simple design promotes quick deployment, ease of development, and solves many problems facing large data caches.

How to install Memcached?

First, you must install libevent:

1, download libevent library from URL: http://www.monkey.org/~provos/libevent/

2, execute following commands:

tar zxvf libevent-1.4.14b-stable.tar.gz

cd libevent-1.4.14b-stable

./configure --prefix=/usr

make && make install

Second, you could install memcached by source package or deb file package (in Debian system):

1, install Memcached by source package:

1) Download source package from official web site:

       http://memcached.org/

2) Execute following commands:

tar zxvf memcached-1.4.5.tar.gz

cd memcached-1.4.5

./configure --prefix=/usr

make && make install

2, install Memcached by deb binary package:

1) Download binary package for Debian from following URL:

              http://packages.debian.org/lenny/i386/memcached/download

       2) Execute following command:

dpkg -i memcached_1.2.2-1+lenny1_i386.deb

How to install Windows version?

It has Windows version, you could download it from this URL:

       http://jehiah.cz/projects/memcached-win32/

To install memcached as a service, follow the next steps:

1, Unzip the binaries in your desired directory (eg. c:"memcached)

2, Install the service using the command: 'c:"memcached"memcached.exe -d install' from either the command line

3, Start the server from the Microsoft Management Console or by running the following command: 'c:"memcached"memcached.exe -d start'

4, Use the server, by default listening to port 11211

How to startup/stop Memcached?

1, use below command to startup Memcached

/usr/bin/memcached -m 64 -p 11211 -u nobody –d

       Command usages:

       -p <num>      TCP port number to listen on (default: 11211)

-U <num>      UDP port number to listen on (default: 0, off)

-s <file>     unix socket path to listen on (disables network support)

-l <ip_addr> interface to listen on, default is INDRR_ANY

-d            run as a daemon

-r            maximize core file limit

-u <username> assume identity of <username> (only when run as root)

-m <num>      max memory to use for items in megabytes, default is 64 MB

-M            return error on memory exhausted (rather than removing items)

-c <num>      max simultaneous connections, default is 1024

-k            lock down all paged memory

-v            verbose (print errors/warnings while in event loop)

-vv           very verbose (also print client commands/reponses)

-h            print this help and exit

-i            print memcached and libevent license

-b            run a managed instanced (mnemonic: buckets)

-P <file>     save PID in <file>, only used with -d option

-f <factor>   chunk size growth factor, default 1.25

-n <bytes>    minimum space allocated for key+value+flags, default 48

2, you can use following commands to check running:

netstat -na|grep 11211

tcp        0      0 127.0.0.1:11211         0.0.0.0:*               LISTEN    

ps -ef|grep memcached

nobody   13305  1 0 10:54 pts/0    00:00:00 /usr/bin/memcached -m 64 -p 11211 -u nobody

3, use below command to stop Memcached:

ps -ef|grep memcached|grep -v grep|awk '{print $2}'|xargs kill -9

Java client usages

Gwhalin is the most popular client for Java platform, its official website:

http://github.com/gwhalin/Memcached-Java-Client

1, Basic

Let’s say you have 3 servers. Server 1 and server 2 have 3GB of space and server 3 has 2GB of space for cache. Here is how I would set up my client.

import com.danga.MemCached.*;

public class MyClass {

// create a static client as most installs only need

// a single instance

protected static MemCachedClient mcc = new MemCachedClient();

// set up connection pool once at class load

static {

// server list and weights

String[] servers =

{

server1.mydomain.com:1624”,

server2.mydomain.com:1624”,

server3.mydomain.com:1624”

};

Integer[] weights = { 3, 3, 2 };

// grab an instance of our connection pool

SockIOPool pool = SockIOPool.getInstance();

// set the servers and the weights

pool.setServers( servers );

pool.setWeights( weights );

// set some basic pool settings

// 5 initial, 5 min, and 250 max conns

// and set the max idle time for a conn

// to 6 hours

pool.setInitConn( 5 );

pool.setMinConn( 5 );

pool.setMaxConn( 250 );

pool.setMaxIdle( 1000 * 60 * 60 * 6 );

// set the sleep for the maint thread

// it will wake up every x seconds and

// maintain the pool size

pool.setMaintSleep( 30 );

// set some TCP settings

// disable nagle

// set the read timeout to 3 secs

// and don’t set a connect timeout

pool.setNagle( false );

pool.setSocketTO( 3000 );

pool.setSocketConnectTO( 0 );

// initialize the connection pool

pool.initialize();

// lets set some compression on for the client

// compress anything larger than 64k

mcc.setCompressEnable( true );

mcc.setCompressThreshold( 64 * 1024 );

}

// from here on down, you can call any of the client calls

public static void examples() {

mcc.set( “foo”, “This is a test String” );

String bar = mcc.get( “foo” );

}

}

2, Multi-client

If you need to support multiple clients (i.e. Java, PHP, Perl, etc.) you need to make a few changes when you are setting things up:

// use a compatible hashing algorithm

pool.setHashingAlg( SockIOPool.NEW_COMPAT_HASH );

// store primitives as strings

// the java client serializes primitives

//

// note: this will not help you when it comes to

// storing non primitives

mcc.setPrimitiveAsString( true );

// don’t url encode keys

// by default the java client url encodes keys

// to sanitize them so they will always work on the server

// however, other clients do not do this

mcc.setSanitizeKeys( false );

3, Failover/Failback Notes

By default the java client will failover to a new server when a server dies. It will also failback to the original if it detects that the server comes back (it checks the server in a falling off pattern).

If you want to disable this (useful if you have flapping servers), there are two settings to handle this.

pool.setFailover( false );

pool.setFailback( false );

Client has a socket connection pool, it will be establishing multi connections at program initialization, following are connection details example:

------------------------------------------------------------------------------------------------------------------

devwh:~# netstat -na|grep 11211

tcp        0      0 0.0.0.0:11211           0.0.0.0:*               LISTEN    

tcp        0      0 192.168.1.13:11211      192.168.1.100:53228     ESTABLISHED

tcp        0      0 192.168.1.13:11211      192.168.1.100:53226     ESTABLISHED

tcp        0      0 192.168.1.13:11211      192.168.1.100:53227     ESTABLISHED

tcp        0      0 192.168.1.13:11211      192.168.1.100:53224     ESTABLISHED

tcp        0      0 192.168.1.13:11211      192.168.1.100:53225     ESTABLISHED

Memcached monitor tool

A fan has wrote a tool for monitor Memcached, it is a Perl script, you could use it to investigate cache server’s status. Following are script source and usage:
        

#!/usr/bin/perl
#
# memcached-tool:
#   stats/management tool for memcached.
#
# Author:
#   Brad Fitzpatrick <brad@danga.com>
#
# License:
#   public domain.  I give up all rights to this
#   tool.  modify and copy at will.
#


use strict;
use IO::Socket::INET;

my $host = shift;
my $mode = shift || "display";
my ($from, $to);

if ($mode eq "display") {
    
undef $mode if @ARGV;
elsif ($mode eq "move") {
    
$from = shift;
    
$to = shift;
    
undef $mode if $from < 6 || $from > 17;
    
undef $mode if $to   < 6 || $to   > 17;
    
print STDERR "ERROR: parameters out of range\n\n" unless $mode;
elsif ($mode eq 'dump') {
    ;
elsif ($mode eq 'stats') {
    ;
else {
    
undef $mode;
}

undef $mode if @ARGV;

die 
"Usage: memcached-tool <host[:port]> [mode]\n
       memcached-tool 10.0.0.5:11211 display    # shows slabs
       memcached-tool 10.0.0.5:11211            # same.  (default is display)
       memcached-tool 10.0.0.5:11211 stats      # shows general stats
       memcached-tool 10.0.0.5:11211 move 7 9   # takes 1MB slab from class #7
                                                # to class #9.

You can only move slabs around once memory is totally allocated, and only
once the target class is full.  (So you can't move from #6 to #9 and #7
to #9 at the same itme, since you'd have to wait for #9 to fill from
the first reassigned page)
" unless $host && $mode;

$host .= ":11211" unless $host =~ /:\d+/;

my $sock = IO::Socket::INET->new(PeerAddr => $host,
                 Proto    
=> 'tcp');
die "Couldn't connect to $host\n" unless $sock;


if ($mode eq "move") {
    
my $tries = 0;
    
while (1) {
    
print $sock "slabs reassign $from $to\r\n";
    
my $res = <$sock>;
    
$res =~ s/\s+//;
    
if ($res eq "DONE") {
        
print "Success.\n";
        
exit 0;
    } 
elsif ($res eq "CANT") {
        
print "Error: can't move from $from to $to.  Destination not yet full?  See usage docs.\n";
        
exit;
    } 
elsif ($res eq "BUSY") {
        
if (++$tries == 3) {
        
print "Failed to move after 3 tries.  Try again later.\n";
        
exit;
        }

        
print "Page busy, retrying\n";
        
sleep 1;
    }
    }

    
exit;
}

if ($mode eq 'dump') {
    
my %items;
    
my $totalitems;

    
print $sock "stats items\r\n";

    
while (<$sock>) {
        
last if /^END/;
        
if (/^STAT items:(\d*):number (\d*)/) {
            
$items{$1= $2;
            
$totalitems += $2;
        }
    }
    
print STDERR "Dumping memcache contents\n";
    
print STDERR "  Number of buckets: " . scalar(keys(%items)) . "\n";
    
print STDERR "  Number of items  : $totalitems\n";

    
foreach my $bucket (sort(keys(%items))) {
        
print STDERR "Dumping bucket $bucket - " . $items{$bucket. " total items\n";
        
print $sock "stats cachedump $bucket $items{$bucket} 1\r\n";
        
my %keyexp;
        
while (<$sock>) {
            
last if /^END/;
            
# return format looks like this
            # ITEM foo [6 b; 1176415152 s]

            if (/^ITEM (\S+\[.* (\d+) s\]/) {
                
$keyexp{$1= $2;
            }
        }

        
foreach my $k (keys(%keyexp)) {
            
my $val;
            
print $sock "get $k\r\n";
            
my $response = <$sock>;
            
$response =~ /VALUE (\S+) (\d+) (\d+)/;
            
my $flags = $2;
            
my $len = $3;
            
read $sock, $val , $len;
            
# get the END
            $_ = <$sock>;
            
$_ = <$sock>;
            
print "add $k $flags $keyexp{$k} $len\r\n$val\r\n";
        }
    }
    
exit;
}

if ($mode eq 'stats') {
    
my %items;

    
print $sock "stats\r\n";

    
while (<$sock>) {
        
last if /^END/;
        
chomp;
        
if (/^STAT\s+(\S*)\s+(.*)/) {
            
$items{$1= $2;
        }
    }
    
printf ("#%-17s %5s %11s\n", $host, "Field", "Value");
    
foreach my $name (sort(keys(%items))) {
      
printf ("%24s %12s\n", $name, $items{$name});
      
    }
    
exit;
}

# display mode:

my %items;  # class -> { number, age, chunk_size, chunks_per_page,
            #            total_pages, total_chunks, used_chunks,
            #            free_chunks, free_chunks_end }


print $sock "stats items\r\n";
while (<$sock>) {
    
last if /^END/;
    
if (/^STAT items:(\d+):(\w+) (\d+)/) {
    
$items{$1}{$2= $3;
    }
}

print $sock "stats slabs\r\n";
while (<$sock>) {
    
last if /^END/;
    
if (/^STAT (\d+):(\w+) (\d+)/) {
    
$items{$1}{$2= $3;
    }
}

print "  #  Item_Size   Max_age  1MB_pages Count   Full?\n";
foreach my $n (1..40) {
    
my $it = $items{$n};
    
next if (0 == $it->{total_pages});
    
my $size = $it->{chunk_size} < 1024 ? "$it->{chunk_size} B " : 
    
sprintf("%.1f kB", $it->{chunk_size} / 1024.0);
    
my $full = $it->{free_chunks_end} == 0 ? "yes" : " no";
    
printf "%3d   %8s %7d s %7d %7d %7s\n",
                        
$n, $size, $it->{age}, $it->{total_pages},
                        
$it->{number}, $full;
}



Usage:

    ./memcachedTool.perl 127.0.0.1:11211

Follow is result:

--------------------------------------------------------------------------------

 # Item_Size   Max_age 1MB_pages Count   Full?

 1      80 B   274880 s       1     101      no


How to use Memcached as Hibernate 2nd level cache

The Hibernate-memcached is a library for using Memcached as a second level distributed cache in Hibernate.

l         Based on the excellent spymemcached client

l         Includes support for the Whalin (danga) memcached client

l         Supports entity and query caching.

The official website is: http://code.google.com/p/hibernate-memcached/

 

1, adding hibernate-memcached and dependencies libraries to application, you will need…

n         Hibernate-memcached-1.2.2.jar

n         Slf4j-1.5.0.jar

n         An slf4j implementation for your preferred logging framework, for log4j you can use slf4j-log4j12-1.5.0.jar.

n         java_memcached-release_2.5.1.jar( optional, if you want to use Whalin client)

n         Memcached-2.3.1.jar( optional, if you want to use Spymemcached client)

2, configuring Hibernate:

The first thing to do is to tell hibernate which cache provider, you would like to use, and to enable the 2nd level cache.

       <property name="hibernate.cache.provider_class">com.googlecode.hibernate.memcached.MemcachedCacheProvider</property>

The Hibernate-memcached liberary supports query caching of Hibernate, you have to enable query caching:

    <property name="hibernate.cache.use_query_cache">true</property>

Configure it to use Whalin to connect Memcached server, adding below configuration:

<property name="hibernate.memcached.memcacheClientFactory">

com.googlecode.hibernate.memcached.dangamemcached.DangaMemcacheClientFactory</property>

3, Cache Wide Settings

The cache wide settings all have defaults can mostly be overwritten at a cache-region level.

Property

Default

Description

hibernate.memcached.servers

localhost:11211

Space delimited list of memcached instances in host:port format

hibernate.memcached.cacheTimeSeconds

300

The default number of seconds items should be cached. Can be overriden at the region level.

hibernate.memcached.keyStrategy

HashCodeKeyStrategy

Sets the strategy class to to use for generating cache keys. Must provide a class name that implements KeyStrategy

hibernate.memcached.readBufferSize

DefaultConnectionFactory.DEFAULT_READ_BUFFER_SIZE

The read buffer size for each server connection from this factory

hibernate.memcached.operationQueueLength

DefaultConnectionFactory.DEFAULT_OP_QUEUE_LEN

Maximum length of the operation queue returned by this connection factory

hibernate.memcached.operationTimeout

DefaultConnectionFactory.DEFAULT_OPERATION_TIMEOUT

Default operation timeout in milliseconds

hibernate.memcached.hashAlgorithm

HashAlgorithm.NATIVE_HASH

Which hash algorithm to use when adding items to the cache. Prior to hibernate-memcached 1.2 this defaulted to HashAlgorithm.KETAMA_HASH

hibernate.memcached.connectionFactory (since 1.2)

DefaultConnectionFactory

The "simple" name of the ConnectionFactory class to use from spymemcached. Must be one of DefaultConnectionFactory, KetamaConnectionFactory, or BinaryConnectionFactory

hibernate.memcached.clearSupported

false

Enables support for the MemcachedCache.clear() method for all cache regions. The way clear is implemented for memcached is expensive and adds overhead to all get/set operations. It is not recommended for production use.

4, Cache Region Settings

Cache region properties are set by giving your cached data a "region name" in hibernate. You can tune the MemcachedCache instance for your region using the following properties. These properties essentially override the cache-wide properties above. Note: that the square brackets are there to denote the text you need to replace, they are not part of the property name.

Property

Default

Description

hibernate.memcached.YOUR_REGION_NAME.cacheTimeSeconds

none, see hibernate.memcached.cacheTimeSeconds

Set the cache time for this cache region, overriding the cache-wide setting.

hibernate.memcached.YOUR_REGION_NAME.keyStrategy

none, see hibernate.memcached.keyStrategy

Overrides the strategy class to to use for generating cache keys in this cache region. Must provide a class name that implements KeyStrategy

hibernate.memcached.YOUR_REGION_NAME.clearSupported

none, see hibernate.memcached.clearSupported

Enables clear() operations for this cache region only. Again, the clear operation incurs cost on every get/set operation.

posted on 2010-07-21 11:01 Robin's Programming World 阅读(2572) 评论(4)  编辑  收藏 所属分类: Java

评论:
# re: Memcached Study Notes 2010-07-21 16:04 | ed hardy swimwear
hibernate.memcached.cacheTimeSeconds  回复  更多评论
  
# re: Memcached Study Notes 2010-07-21 16:04 | true religion perfume
The Hibernate-memcached liberary supports query caching of Hibernate, you have to enable query caching:

<property name="hibernate.cache.use_query_cache">true</property>  回复  更多评论
  
# re: Memcached Study Notes 2010-12-01 14:06 | wow account cheap

Son flacon s'apparente à une oeuvre d'art. Un reflet bleu fascinant pour une sculpture futuriste, des formes sensuelles, envoûtantes, mythiques. De la matière noble (or, argent, cristal) qui épouse une technologie exceptionnelle.   回复  更多评论
  
# re: Memcached Study Notes 2010-12-07 16:06 | christian louboutin shoes
Passed without incident.  回复  更多评论
  

只有注册用户登录后才能发表评论。


网站导航: