"ping: unknown host www.baidu.com"
解决方案:
如果某台Linux服务器ping不通域名, 如下提示:
# ping www.baidu.com
ping: unknown host www.baidu.com
如果确定网络没问题的情况下, 可以通过如下步骤寻找解决办法:
1) 确定设置了域名服务器, 没有的话, 建议设置Google的公共DNS服务, 它应该不会出问题的
# cat /etc/resolv.conf
-------------------------------------------------------------------
nameserver 8.8.8.8
nameserver 8.8.4.4
-------------------------------------------------------------------
2) 确保网关已设置
# grep GATEWAY /etc/sysconfig/network-scripts/ifcfg*
-------------------------------------------------------------------
/etc/sysconfig/network-scripts/ifcfg-eth0:GATEWAY=192.168.40.1
-------------------------------------------------------------------
如果未设置, 则通过如下方式增加网关:
# route add default gw 192.168.40.1
或者手工编写/etc/sysconfig/network-scripts/ifcfg*文件后, 重启network服务:
# service network restart
3) 确保可用dns解析
# grep hosts /etc/nsswitch.conf
-------------------------------------------------------------------
hosts: files dns
-------------------------------------------------------------------
如果以上哪个有问题, 修正后, 再测试, 应该就没问题了:
#ping -c 3 www.baidu.com
PING www.a.shifen.com (220.181.6.175) 56(84) bytes of data.
64 bytes from 220.181.6.175: icmp_seq=0 ttl=50 time=9.51 ms
64 bytes from 220.181.6.175: icmp_seq=1 ttl=50 time=8.45 ms
64 bytes from 220.181.6.175: icmp_seq=2 ttl=50 time=8.97 ms
--- www.a.shifen.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 8.450/8.977/9.511/0.446 ms, pipe 2
0. 简介
1. 基础知识
1.1. 介绍
1.2. 安装
1.3. Hello World
1.4. 配置语法
2. 输入插件(Input)
2.1. 标准输入(Stdin)
2.2. 读取文件(File)
2.3. 读取网络数据(TCP)
2.4. 读取 Syslog 数据
2.5. 读取 Redis 数据
3. 编码插件(Codec)
3.1. 采用 JSON 编码
3.2. 合并多行数据(Multiline)
4. 过滤器插件(Filter)
4.1. Grok 正则捕获
4.2. 时间处理(Date)
4.3. 数据修改(Mutate)
4.4. GeoIP 查询归类
4.5. UserAgent 匹配归类
4.6. Key-Value 切分
4.7. 随心所欲的 Ruby 处理
4.8. 数值统计(Metrics)
5. 输出插件(Output)
5.1. 标准输出(Stdout)
5.2. 保存成文件(File)
5.3. 保存进 Elasticsearch
5.4. 输出到 Redis
5.5. 输出到 Statsd
5.6. 报警到 Nagios
5.7. 发送邮件(Email)
5.8. 调用命令执行(Exec)
6. 尚未进入官方库的常用插件
6.1. Kafka
6.2. HDFS
6.3. Scribe
7. 深入了解
7.1. 自己写一个插件
7.2. 为什么用 JRuby? 能用 MRI 运
7.3. 其他类似项目
在这里对jedis关于事务、管道和分布式的调用方式做一个简单的介绍和对比:
一、普通同步方式
最简单和基础的调用方式,
@Test public void test1Normal() { Jedis jedis = new Jedis("localhost"); long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { String result = jedis.set("n" + i, "n" + i); } long end = System.currentTimeMillis(); System.out.println("Simple SET: " + ((end - start)/1000.0) + " seconds"); jedis.disconnect(); }
很简单吧,每次set
之后都可以返回结果,标记是否成功。
二、事务方式(Transactions)
redis的事务很简单,他主要目的是保障,一个client发起的事务中的命令可以连续的执行,而中间不会插入其他client的命令。
看下面例子:
@Test public void test2Trans() { Jedis jedis = new Jedis("localhost"); long start = System.currentTimeMillis(); Transaction tx = jedis.multi(); for (int i = 0; i < 100000; i++) { tx.set("t" + i, "t" + i); } List<Object> results = tx.exec(); long end = System.currentTimeMillis(); System.out.println("Transaction SET: " + ((end - start)/1000.0) + " seconds"); jedis.disconnect(); }
我们调用jedis.watch(…)
方法来监控key,如果调用后key值发生变化,则整个事务会执行失败。另外,事务中某个操作失败,并不会回滚其他操作。这一点需要注意。还有,我们可以使用discard()
方法来取消事务。
三、管道(Pipelining)
有时,我们需要采用异步方式,一次发送多个指令,不同步等待其返回结果。这样可以取得非常好的执行效率。这就是管道,调用方法如下:
@Test public void test3Pipelined() { Jedis jedis = new Jedis("localhost"); Pipeline pipeline = jedis.pipelined(); long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { pipeline.set("p" + i, "p" + i); } List<Object> results = pipeline.syncAndReturnAll(); long end = System.currentTimeMillis(); System.out.println("Pipelined SET: " + ((end - start)/1000.0) + " seconds"); jedis.disconnect(); }
四、管道中调用事务
就Jedis提供的方法而言,是可以做到在管道中使用事务,其代码如下:
@Test public void test4combPipelineTrans() { jedis = new Jedis("localhost"); long start = System.currentTimeMillis(); Pipeline pipeline = jedis.pipelined(); pipeline.multi(); for (int i = 0; i < 100000; i++) { pipeline.set("" + i, "" + i); } pipeline.exec(); List<Object> results = pipeline.syncAndReturnAll(); long end = System.currentTimeMillis(); System.out.println("Pipelined transaction: " + ((end - start)/1000.0) + " seconds"); jedis.disconnect(); }
但是经测试(见本文后续部分),发现其效率和单独使用事务差不多,甚至还略微差点。
五、分布式直连同步调用
@Test public void test5shardNormal() { List<JedisShardInfo> shards = Arrays.asList( new JedisShardInfo("localhost",6379), new JedisShardInfo("localhost",6380)); ShardedJedis sharding = new ShardedJedis(shards); long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { String result = sharding.set("sn" + i, "n" + i); } long end = System.currentTimeMillis(); System.out.println("Simple@Sharing SET: " + ((end - start)/1000.0) + " seconds"); sharding.disconnect(); }
这个是分布式直接连接,并且是同步调用,每步执行都返回执行结果。类似地,还有异步管道调用。
六、分布式直连异步调用
@Test public void test6shardpipelined() { List<JedisShardInfo> shards = Arrays.asList( new JedisShardInfo("localhost",6379), new JedisShardInfo("localhost",6380)); ShardedJedis sharding = new ShardedJedis(shards); ShardedJedisPipeline pipeline = sharding.pipelined(); long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { pipeline.set("sp" + i, "p" + i); } List<Object> results = pipeline.syncAndReturnAll(); long end = System.currentTimeMillis(); System.out.println("Pipelined@Sharing SET: " + ((end - start)/1000.0) + " seconds"); sharding.disconnect(); }
七、分布式连接池同步调用
如果,你的分布式调用代码是运行在线程中,那么上面两个直连调用方式就不合适了,因为直连方式是非线程安全的,这个时候,你就必须选择连接池调用。
@Test public void test7shardSimplePool() { List<JedisShardInfo> shards = Arrays.asList( new JedisShardInfo("localhost",6379), new JedisShardInfo("localhost",6380)); ShardedJedisPool pool = new ShardedJedisPool(new JedisPoolConfig(), shards); ShardedJedis one = pool.getResource(); long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { String result = one.set("spn" + i, "n" + i); } long end = System.currentTimeMillis(); pool.returnResource(one); System.out.println("Simple@Pool SET: " + ((end - start)/1000.0) + " seconds"); pool.destroy(); }
上面是同步方式,当然还有异步方式。
八、分布式连接池异步调用
@Test public void test8shardPipelinedPool() { List<JedisShardInfo> shards = Arrays.asList( new JedisShardInfo("localhost",6379), new JedisShardInfo("localhost",6380)); ShardedJedisPool pool = new ShardedJedisPool(new JedisPoolConfig(), shards); ShardedJedis one = pool.getResource(); ShardedJedisPipeline pipeline = one.pipelined(); long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { pipeline.set("sppn" + i, "n" + i); } List<Object> results = pipeline.syncAndReturnAll(); long end = System.currentTimeMillis(); pool.returnResource(one); System.out.println("Pipelined@Pool SET: " + ((end - start)/1000.0) + " seconds"); pool.destroy(); }
九、需要注意的地方
事务和管道都是异步模式。在事务和管道中不能同步查询结果。比如下面两个调用,都是不允许的:
Transaction tx = jedis.multi(); for (int i = 0; i < 100000; i++) { tx.set("t" + i, "t" + i); } System.out.println(tx.get("t1000").get()); //不允许 List<Object> results = tx.exec(); … … Pipeline pipeline = jedis.pipelined(); long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { pipeline.set("p" + i, "p" + i); } System.out.println(pipeline.get("p1000").get()); //不允许 List<Object> results = pipeline.syncAndReturnAll();
事务和管道都是异步的,个人感觉,在管道中再进行事务调用,没有必要,不如直接进行事务模式。
分布式中,连接池的性能比直连的性能略好(见后续测试部分)。
分布式调用中不支持事务。
因为事务是在服务器端实现,而在分布式中,每批次的调用对象都可能访问不同的机器,所以,没法进行事务。
十、测试
运行上面的代码,进行测试,其结果如下:
Simple SET: 5.227 seconds Transaction SET: 0.5 seconds Pipelined SET: 0.353 seconds Pipelined transaction: 0.509 seconds Simple@Sharing SET: 5.289 seconds Pipelined@Sharing SET: 0.348 seconds Simple@Pool SET: 5.039 seconds Pipelined@Pool SET: 0.401 seconds
另外,经测试分布式中用到的机器越多,调用会越慢。上面是2片,下面是5片:
Simple@Sharing SET: 5.494 seconds Pipelined@Sharing SET: 0.51 seconds Simple@Pool SET: 5.223 seconds Pipelined@Pool SET: 0.518 seconds
下面是10片:
Simple@Sharing SET: 5.9 seconds Pipelined@Sharing SET: 0.794 seconds Simple@Pool SET: 5.624 seconds Pipelined@Pool SET: 0.762 seconds
下面是100片:
Simple@Sharing SET: 14.055 seconds Pipelined@Sharing SET: 8.185 seconds Simple@Pool SET: 13.29 seconds Pipelined@Pool SET: 7.767 seconds
分布式中,连接池方式调用不但线程安全外,根据上面的测试数据,也可以看出连接池比直连的效率更好。
十一、完整的测试代码
package com.example.nosqlclient; import java.util.Arrays; import java.util.List; import org.junit.AfterClass; import org.junit.BeforeClass; import org.junit.Test; import redis.clients.jedis.Jedis; import redis.clients.jedis.JedisPoolConfig; import redis.clients.jedis.JedisShardInfo; import redis.clients.jedis.Pipeline; import redis.clients.jedis.ShardedJedis; import redis.clients.jedis.ShardedJedisPipeline; import redis.clients.jedis.ShardedJedisPool; import redis.clients.jedis.Transaction; import org.junit.FixMethodOrder; import org.junit.runners.MethodSorters; @FixMethodOrder(MethodSorters.NAME_ASCENDING) public class TestJedis { private static Jedis jedis; private static ShardedJedis sharding; private static ShardedJedisPool pool; @BeforeClass public static void setUpBeforeClass() throws Exception { List<JedisShardInfo> shards = Arrays.asList( new JedisShardInfo("localhost",6379), new JedisShardInfo("localhost",6379)); //使用相同的ip:port,仅作测试 jedis = new Jedis("localhost"); sharding = new ShardedJedis(shards); pool = new ShardedJedisPool(new JedisPoolConfig(), shards); } @AfterClass public static void tearDownAfterClass() throws Exception { jedis.disconnect(); sharding.disconnect(); pool.destroy(); } @Test public void test1Normal() { long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { String result = jedis.set("n" + i, "n" + i); } long end = System.currentTimeMillis(); System.out.println("Simple SET: " + ((end - start)/1000.0) + " seconds"); } @Test public void test2Trans() { long start = System.currentTimeMillis(); Transaction tx = jedis.multi(); for (int i = 0; i < 100000; i++) { tx.set("t" + i, "t" + i); } //System.out.println(tx.get("t1000").get()); List<Object> results = tx.exec(); long end = System.currentTimeMillis(); System.out.println("Transaction SET: " + ((end - start)/1000.0) + " seconds"); } @Test public void test3Pipelined() { Pipeline pipeline = jedis.pipelined(); long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { pipeline.set("p" + i, "p" + i); } //System.out.println(pipeline.get("p1000").get()); List<Object> results = pipeline.syncAndReturnAll(); long end = System.currentTimeMillis(); System.out.println("Pipelined SET: " + ((end - start)/1000.0) + " seconds"); } @Test public void test4combPipelineTrans() { long start = System.currentTimeMillis(); Pipeline pipeline = jedis.pipelined(); pipeline.multi(); for (int i = 0; i < 100000; i++) { pipeline.set("" + i, "" + i); } pipeline.exec(); List<Object> results = pipeline.syncAndReturnAll(); long end = System.currentTimeMillis(); System.out.println("Pipelined transaction: " + ((end - start)/1000.0) + " seconds"); } @Test public void test5shardNormal() { long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { String result = sharding.set("sn" + i, "n" + i); } long end = System.currentTimeMillis(); System.out.println("Simple@Sharing SET: " + ((end - start)/1000.0) + " seconds"); } @Test public void test6shardpipelined() { ShardedJedisPipeline pipeline = sharding.pipelined(); long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { pipeline.set("sp" + i, "p" + i); } List<Object> results = pipeline.syncAndReturnAll(); long end = System.currentTimeMillis(); System.out.println("Pipelined@Sharing SET: " + ((end - start)/1000.0) + " seconds"); } @Test public void test7shardSimplePool() { ShardedJedis one = pool.getResource(); long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { String result = one.set("spn" + i, "n" + i); } long end = System.currentTimeMillis(); pool.returnResource(one); System.out.println("Simple@Pool SET: " + ((end - start)/1000.0) + " seconds"); } @Test public void test8shardPipelinedPool() { ShardedJedis one = pool.getResource(); ShardedJedisPipeline pipeline = one.pipelined(); long start = System.currentTimeMillis(); for (int i = 0; i < 100000; i++) { pipeline.set("sppn" + i, "n" + i); } List<Object> results = pipeline.syncAndReturnAll(); long end = System.currentTimeMillis(); pool.returnResource(one); System.out.println("Pipelined@Pool SET: " + ((end - start)/1000.0) + " seconds"); } }