少年阿宾

那些青春的岁月

  BlogJava :: 首页 :: 联系 :: 聚合  :: 管理
  497 Posts :: 0 Stories :: 135 Comments :: 0 Trackbacks

2017年8月3日 #

Go语言没有沿袭传统面向对象编程中的诸多概念,比如继承、虚函数、构造函数和析构函数、隐藏的this指针等。

 

方法

Go 语言中同时有函数和方法。方法就是一个包含了接受者(receiver)的函数,receiver可以是内置类型或者结构体类型的一个值或者是一个指针。所有给定类型的方法属于该类型的方法集。

如下面的这个例子,定义了一个新类型Integer,它和int一样,只是为它内置的int类型增加了个新方法Less()

复制代码
type Integer int   func (a Integer) Less(b Integer) bool {     return a < b  }  func main() {     var a Integer = 1       if a.Less(2) {         fmt.Println("less then 2")     }    }
复制代码

可以看出,Go语言在自定义类型的对象中没有C++/Java那种隐藏的this指针,而是在定义成员方法时显式声明了其所属的对象。

 

method的语法如下:

func (r ReceiverType) funcName(parameters) (results)

当调用method时,会将receiver作为函数的第一个参数:

funcName(r, parameters);

所以,receiver是值类型还是指针类型要看method的作用。如果要修改对象的值,就需要传递对象的指针。

指针作为Receiver会对实例对象的内容发生操作,而普通类型作为Receiver仅仅是以副本作为操作对象,并不对原实例对象发生操作。

复制代码
func (a *Ingeger) Add(b Integer) {     *a += b }  func main() {     var a Integer = 1      a.Add(3)     fmt.Println("a =", a)     //  a = 4 }
复制代码

如果Add方法不使用指针,则a返回的结果不变,这是因为Go语言函数的参数也是基于值传递。

注意:当方法的接受者是指针时,即使用值类型调用那么方法内部也是对指针的操作。

 

之前说过,Go语言没有构造函数的概念,通常使用一个全局函数来完成。例如:

复制代码
func NewRect(x, y, width, height float64) *Rect {     return &Rect{x, y, width, height} }     func main() {     rect1 := NewRect(1,2,10,20)     fmt.Println(rect1.width) }
复制代码

 

 


匿名组合

Go语言提供了继承,但是采用了组合的语法,我们将其称为匿名组合,例如:

复制代码
type Base struct {     name string }  func (base *Base) Set(myname string) {     base.name = myname }  func (base *Base) Get() string {     return base.name }  type Derived struct {     Base     age int  }  func (derived *Derived) Get() (nm string, ag int) {     return derived.name, derived.age }   func main() {     b := &Derived{}      b.Set("sina")     fmt.Println(b.Get()) }
复制代码

例子中,在Base类型定义了get()和set()两个方法,而Derived类型继承了Base类,并改写了Get()方法,在Derived对象调用Set()方法,会加载基类对应的方法;而调用Get()方法时,加载派生类改写的方法。

 

组合的类型和被组合的类型包含同名成员时, 会不会有问题呢?可以参考下面的例子:

复制代码
type Base struct {     name string     age int }  func (base *Base) Set(myname string, myage int) {     base.name = myname     base.age = myage }  type Derived struct {     Base     name string }  func main() {     b := &Derived{}      b.Set("sina", 30)     fmt.Println("b.name =",b.name, "\tb.Base.name =", b.Base.name)     fmt.Println("b.age =",b.age, "\tb.Base.age =", b.Base.age) }
复制代码

 

 

 


值语义和引用语义

值语义和引用语义的差别在于赋值,比如

b = a b.Modify()

如果b的修改不会影响a的值,那么此类型属于值类型;如果会影响a的值,那么此类型是引用类型。

Go语言中的大多数类型都基于值语义,包括:

  • 基本类型,如byte、int、bool、float32、string等;
  • 复合类型,如arry、struct、pointer等;

 

C语言中的数组比较特别,通过函数传递一个数组的时候基于引用语义,但是在结构体定义数组变量的时候基于值语义。而在Go语言中,数组和基本类型没有区别,是很纯粹的值类型,例如:

var a = [3] int{1,2,3} var b = a b[1]++ fmt.Println(a, b)   // [1 2 3] [1 3 3]

从结果看,b=a赋值语句是数组内容的完整复制,要想表达引用,需要用指针:

var a = [3] int{1,2,3} var b = &a    // 引用语义 b[1]++ fmt.Println(a, b)   // [1 3 3] [1 3 3]

 

 


接口

Interface 是一组抽象方法(未具体实现的方法/仅包含方法名参数返回值的方法)的集合,如果实现了 interface 中的所有方法,即该类/对象就实现了该接口。

Interface 的声明格式:

type interfaceName interface {       //方法列表   }  

Interface 可以被任意对象实现,一个类型/对象也可以实现多个 interface;
interface的变量可以持有任意实现该interface类型的对象。

 如下面的例子:

复制代码
package main      import "fmt"      type Human struct {         name string         age int         phone string     }      type Student struct {         Human //匿名字段         school string         loan float32     }      type Employee struct {         Human //匿名字段         company string         money float32     }      //Human实现SayHi方法     func (h Human) SayHi() {         fmt.Printf("Hi, I am %s you can call me on %s\n", h.name, h.phone)     }      //Human实现Sing方法     func (h Human) Sing(lyrics string) {         fmt.Println("La la la la...", lyrics)     }      //Employee重载Human的SayHi方法     func (e Employee) SayHi() {         fmt.Printf("Hi, I am %s, I work at %s. Call me on %s\n", e.name,             e.company, e.phone)         }      // Interface Men被Human,Student和Employee实现     // 因为这三个类型都实现了这两个方法     type Men interface {         SayHi()         Sing(lyrics string)     }      func main() {         mike := Student{Human{"Mike", 25, "222-222-XXX"}, "MIT", 0.00}         paul := Student{Human{"Paul", 26, "111-222-XXX"}, "Harvard", 100}         sam := Employee{Human{"Sam", 36, "444-222-XXX"}, "Golang Inc.", 1000}         tom := Employee{Human{"Tom", 37, "222-444-XXX"}, "Things Ltd.", 5000}          //定义Men类型的变量i         var i Men          //i能存储Student         i = mike             fmt.Println("This is Mike, a Student:")         i.SayHi()         i.Sing("November rain")          //i也能存储Employee         i = tom         fmt.Println("This is tom, an Employee:")         i.SayHi()         i.Sing("Born to be wild")          //定义了slice Men         fmt.Println("Let's use a slice of Men and see what happens")         x := make([]Men, 3)         //这三个都是不同类型的元素,但是他们实现了interface同一个接口         x[0], x[1], x[2] = paul, sam, mike          for _, value := range x{             value.SayHi()         }     }
复制代码

 

空接口

空interface(interface{})不包含任何的method,正因为如此,所有的类型都实现了空interface。空interface对于描述起不到任何的作用(因为它不包含任何的method),但是空interface在我们需要存储任意类型的数值的时候相当有用,因为它可以存储任意类型的数值。它有点类似于C语言的void*类型。

复制代码
// 定义a为空接口     var a interface{}     var i int = 5     s := "Hello world"     // a可以存储任意类型的数值     a = i     a = s
复制代码

 

interface的变量里面可以存储任意类型的数值(该类型实现了interface),那么我们怎么反向知道这个interface变量里面实际保存了的是哪个类型的对象呢?目前常用的有两种方法:switch测试、Comma-ok断言。

 

switch测试如下:

复制代码
type Element interface{} type List [] Element  type Person struct {     name string     age int  }  //打印 func (p Person) String() string {     return "(name: " + p.name + " - age: "+strconv.Itoa(p.age)+ " years)" }  func main() {     list := make(List, 3)     list[0] = 1 //an int      list[1] = "Hello" //a string     list[2] = Person{"Dennis", 70}       for index, element := range list{         switch value := element.(type) {             case int:                 fmt.Printf("list[%d] is an int and its value is %d\n", index, value)             case string:                 fmt.Printf("list[%d] is a string and its value is %s\n", index, value)             case Person:                 fmt.Printf("list[%d] is a Person and its value is %s\n", index, value)             default:                 fmt.Println("list[%d] is of a different type", index)         }        }    }
复制代码

 

如果使用Comma-ok断言的话:

复制代码
func main() {     list := make(List, 3)     list[0] = 1 // an int     list[1] = "Hello" // a string     list[2] = Person{"Dennis", 70}      for index, element := range list {         if value, ok := element.(int); ok {             fmt.Printf("list[%d] is an int and its value is %d\n", index, value)         } else if value, ok := element.(string); ok {             fmt.Printf("list[%d] is a string and its value is %s\n", index, value)         } else if value, ok := element.(Person); ok {             fmt.Printf("list[%d] is a Person and its value is %s\n", index, value)         } else {             fmt.Printf("list[%d] is of a different type\n", index)         }     } }
复制代码

 

 

嵌入接口

正如struct类型可以包含一个匿名字段,interface也可以嵌套另外一个接口。

如果一个interface1作为interface2的一个嵌入字段,那么interface2隐式的包含了interface1里面的method。

 

 

反射

所谓反射(reflect)就是能检查程序在运行时的状态。

使用reflect一般分成三步,下面简要的讲解一下:要去反射是一个类型的值(这些值都实现了空interface),首先需要把它转化成reflect对象(reflect.Type或者reflect.Value,根据不同的情况调用不同的函数)。这两种获取方式如下:

 t := reflect.TypeOf(i)    //得到类型的元数据,通过t我们能获取类型定义里面的所有元素  v := reflect.ValueOf(i)   //得到实际的值,通过v我们获取存储在里面的值,还可以去改变值

 

转化为reflect对象之后我们就可以进行一些操作了,也就是将reflect对象转化成相应的值,例如

tag := t.Elem().Field(0).Tag  //获取定义在struct里面的标签 name := v.Elem().Field(0).String()  //获取存储在第一个字段里面的值

 

获取反射值能返回相应的类型和数值

var x float64 = 3.4 v := reflect.ValueOf(x) fmt.Println("type:", v.Type()) fmt.Println("kind is float64:", v.Kind() == reflect.Float64) fmt.Println("value:", v.Float())

 

最后,反射的话,那么反射的字段必须是可修改的,我们前面学习过传值和传引用,这个里面也是一样的道理。反射的字段必须是可读写的意思是,如果下面这样写,那么会发生错误

var x float64 = 3.4 v := reflect.ValueOf(x) v.SetFloat(7.1)

 

如果要修改相应的值,必须这样写

var x float64 = 3.4 p := reflect.ValueOf(&x) v := p.Elem() v.SetFloat(7.1)

上面只是对反射的简单介绍,更深入的理解还需要自己在编程中不断的实践。

 

 

参考文档:

http://se77en.cc/2014/05/05/methods-interfaces-and-embedded-types-in-golang/

http://se77en.cc/2014/05/04/choose-whether-to-use-a-value-or-pointer-receiver-on-methods/

 http://www.cnblogs.com/chenny7/p/4497969.html





posted @ 2017-08-03 11:34 abin 阅读(36) | 评论 (0)编辑 收藏

2017年8月2日 #

不可或缺的函数,在Go中定义函数的方式如下:

func (p myType ) funcName ( a, b int , c string ) ( r , s int ) {     return } 

通过函数定义,我们可以看到Go中函数和其他语言中的共性和特性

共性

  • 关键字——func
  • 方法名——funcName
  • 入参——— a,b int,b string
  • 返回值—— r,s int
  • 函数体—— {}

特性

Go中函数的特性是非常酷的,给我们带来不一样的编程体验。

为特定类型定义函数,即为类型对象定义方法

在Go中通过给函数标明所属类型,来给该类型定义方法,上面的 p myType 即表示给myType声明了一个方法, p myType 不是必须的。如果没有,则纯粹是一个函数,通过包名称访问。packageName.funcationName

如:

//定义新的类型double,主要目的是给float64类型扩充方法 type double float64  //判断a是否等于b func (a double) IsEqual(b double) bool {     var r = a - b     if r == 0.0 {         return true     } else if r < 0.0 {         return r > -0.0001     }     return r < 0.0001 }  //判断a是否等于b func IsEqual(a, b float64) bool {     var r = a - b     if r == 0.0 {         return true     } else if r < 0.0 {         return r > -0.0001     }     return r < 0.0001 }  func main() {     var a double = 1.999999     var b double = 1.9999998     fmt.Println(a.IsEqual(b))     fmt.Println(a.IsEqual(3))     fmt.Println( IsEqual( (float64)(a), (float64)(b) ) )  } 

上述示例为 float64 基本类型扩充了方法IsEqual,该方法主要是解决精度问题。 其方法调用方式为: a.IsEqual(double) ,如果不扩充方法,我们只能使用函数IsEqual(a, b float64)

入参中,如果连续的参数类型一致,则可以省略连续多个参数的类型,只保留最后一个类型声明。

如 func IsEqual(a, b float64) bool 这个方法就只保留了一个类型声明,此时入参a和b均是float64数据类型。 这样也是可以的: func IsEqual(a, b float64, accuracy int) bool

变参:入参支持变参,即可接受不确定数量的同一类型的参数

如 func Sum(args ...int) 参数args是的slice,其元素类型为int 。经常使用的fmt.Printf就是一个接受任意个数参数的函数 fmt.Printf(format string, args ...interface{})

支持多返回值

前面我们定义函数时返回值有两个r,s 。这是非常有用的,我在写C#代码时,常常为了从已有函数中获得更多的信息,需要修改函数签名,使用out ,ref 等方式去获得更多返回结果。而现在使用Go时则很简单,直接在返回值后面添加返回参数即可。

如,在C#中一个字符串转换为int类型时逻辑代码

int v=0;  if ( int.TryPase("123456",out v) ) {     //code } 

而在Go中,则可以这样实现,逻辑精简而明确

if v,isOk :=int.TryPase("123456") ; isOk {     //code } 

同时在Go中很多函数充分利用了多返回值

  • func (file *File) Write(b []byte) (n int, err error)
  • func Sincos(x float64) (sin, cos float64)

那么如果我只需要某一个返回值,而不关心其他返回值的话,我该如何办呢? 这时可以简单的使用符号下划线”_“ 来忽略不关心的返回值。如:

_, cos = math.Sincos(3.1415) //只需要cos计算的值 

命名返回值

前面我们说了函数可以有多个返回值,这里我还要说的是,在函数定义时可以给所有的返回值分别命名,这样就能在函数中任意位置给不同返回值复制,而不需要在return语句中才指定返回值。同时也能增强可读性,也提高godoc所生成文档的可读性

如果不支持命名返回值,我可能会是这样做的

func ReadFull(r Reader, buf []byte) (int, error) {     var n int     var err error      for len(buf) > 0  {         var nr int         nr, err = r.Read(buf)          n += nr         if err !=nil {             return n,err         }         buf = buf[nr:]     }     return n,err } 

但支持给返回值命名后,实际上就是省略了变量的声明,return时无需写成return n,err 而是将直接将值返回

func ReadFull(r Reader, buf []byte) (n int, err error) {     for len(buf) > 0 && err == nil {         var nr int         nr, err = r.Read(buf)         n += nr         buf = buf[nr:]     }     return } 

函数也是“值”

和Go中其他东西一样,函数也是值,这样就可以声明一个函数类型的变量,将函数作为参数传递。

声明函数为值的变量(匿名函数:可赋值个变量,也可直接执行)

//赋值 fc := func(msg string) {     fmt.Println("you say :", msg) } fmt.Printf("%T \n", fc) fc("hello,my love") //直接执行 func(msg string) {     fmt.Println("say :", msg) }("I love to code") 

输出结果如下,这里表明fc 的类型为:func(string)

func(string)  you say : hello,my love say : I love to code 

将函数作为入参(回调函数),能带来便利。如日志处理,为了统一处理,将信息均通过指定函数去记录日志,且是否记录日志还有开关

func Log(title string, getMsg func() string) {     //如果开启日志记录,则记录日志     if true {         fmt.Println(title, ":", getMsg())     } } //---------调用-------------- count := 0 msg := func() string {     count++     return "您没有即使提醒我,已触犯法律" } Log("error", msg) Log("warring", msg) Log("info", msg) fmt.Println(count) 

这里输出结果如下,count 也发生了变化

error : 您没有即使提醒我,已触犯法律 warring : 您没有即使提醒我,已触犯法律 info : 您没有即使提醒我,已触犯法律 3 

函数也是“类型”

你有没有注意到上面示例中的 fc := func(msg string)... ,既然匿名函数可以赋值给一个变量,同时我们经常这样给int赋值 value := 2 ,是否我们可以声明func(string) 类型 呢,当然是可以的。

//一个记录日志的类型:func(string) type saveLog func(msg string)  //将字符串转换为int64,如果转换失败调用saveLog func stringToInt(s string, log saveLog) int64 {      if value, err := strconv.ParseInt(s, 0, 0); err != nil {         log(err.Error())         return 0     } else {         return value     } }  //记录日志消息的具体实现 func myLog(msg string) {     fmt.Println("Find Error:", msg) }  func main() {     stringToInt("123", myLog) //转换时将调用mylog记录日志     stringToInt("s", myLog) } 

这里我们定义了一个类型,专门用作记录日志的标准接口。在stringToInt函数中如果转换失败则调用我自己定义的接口函数进行日志处理,至于最终执行的哪个函数,则无需关心。

defer 延迟函数

defer 又是一个创新,它的作用是:延迟执行,在声明时不会立即执行,而是在函数return后时按照后进先出的原则依次执行每一个defer。这样带来的好处是,能确保我们定义的函数能百分之百能够被执行到,这样就能做很多我们想做的事,如释放资源,清理数据,记录日志等

这里我们重点来说明下defer的执行顺序

func deferFunc() int {     index := 0      fc := func() {          fmt.Println(index, "匿名函数1")         index++          defer func() {             fmt.Println(index, "匿名函数1-1")             index++         }()     }      defer func() {         fmt.Println(index, "匿名函数2")         index++     }()      defer fc()      return func() int {         fmt.Println(index, "匿名函数3")         index++         return index     }() }  func main() {     deferFunc() } 

这里输出结果如下,

0 匿名函数3 1 匿名函数1 2 匿名函数1-1 3 匿名函数2 

有如下结论:

  • defer 是在执行完return 后执行
  • defer 后进先执行

另外,我们常使用defer去关闭IO,在正常打开文件后,就立刻声明一个defer,这样就不会忘记关闭文件,也能保证在出现异常等不可预料的情况下也能关闭文件。而不像其他语言:try-catch 或者 using() 方式进行处理。

file , err :=os.Open(file) if err != nil {     return err } defer file.Close()  //dosomething with file 

后续,我将讨论: 作用域、传值和传指针 以及 保留函数init(),main()

本笔记中所写代码存储位置:

posted @ 2017-08-02 16:39 abin 阅读(48) | 评论 (0)编辑 收藏

2016年8月18日 #

MySQL触发器Trigger实例篇
发表于668 天前 ⁄ IT技术 ⁄ 暂无评论

以前关注的数据存储过程不太懂其中奥妙,最近遇到跨数据库,同时对多个表进行CURD(Create增、Update改、Read读、Delete删),怎么才能让繁琐的数据CURD同步变得更容易呢?相信很多人会首先想到了MySQL存储过程、触发器,这种想法确实不错。于是饶有兴趣地亲自写了CUD(增、改、删)触发器的实例,用触发器实现多表数据同步更新。

MySQL触发器Trigger实例篇

定义: 何为MySQL触发器?

在MySQL Server里面也就是对某一个表的一定的操作,触发某种条件(Insert,Update,Delete 等),从而自动执行的一段程序。从这种意义上讲触发器是一个特殊的存储过程。下面通过MySQL触发器实例,来了解一下触发器的工作过程吧!

一、创建MySQL实例数据表:

在mysql的默认的测试test数据库下,创建两个表t_a与t_b:



    /*Table structure for table `t_a` */
    DROP TABLE IF EXISTS `t_a`;
    CREATE TABLE `t_a` (
      `id` smallint(1) unsigned NOT NULL AUTO_INCREMENT,
      `username` varchar(20) DEFAULT NULL,
      `groupid` mediumint(8) unsigned NOT NULL DEFAULT '0',
      PRIMARY KEY (`id`)
    ) ENGINE=MyISAM AUTO_INCREMENT=16 DEFAULT CHARSET=latin1;
     
    /*Data for the table `t_a` */
    LOCK TABLES `t_a` WRITE;
    UNLOCK TABLES;
     
    /*Table structure for table `t_b` */
    DROP TABLE IF EXISTS `t_b`;
    CREATE TABLE `t_b` (
      `id` smallint(1) unsigned NOT NULL AUTO_INCREMENT,
      `username` varchar(20) DEFAULT NULL,
      `groupid` mediumint(8) unsigned NOT NULL DEFAULT '0',
      PRIMARY KEY (`id`)
    ) ENGINE=MyISAM AUTO_INCREMENT=57 DEFAULT CHARSET=latin1;
     
    /*Data for the table `t_b` */
    LOCK TABLES `t_b` WRITE;
    UNLOCK TABLES;

在t_a表上分创建一个CUD(增、改、删)3个触发器,将t_a的表数据与t_b同步实现CUD,注意创建触发器每个表同类事件有且仅有一个对应触发器,为什么只能对一个触发器,不解释啦,看MYSQL的说明帮助文档吧。

二、创建MySQL实例触发器:

在实例数据表t_a上依次按照下面步骤创建tr_a_insert、tr_a_update、tr_a_delete三个触发器

1、创建INSERT触发器trigger_a_insert:



    DELIMITER $$
     
    USE `test`$$
     
    --判断数据库中是否存在tr_a_insert触发器
    DROP TRIGGER /*!50032 IF EXISTS */ `tr_a_insert`$$
    --不存在tr_a_insert触发器,开始创建触发器
    --Trigger触发条件为insert成功后进行触发
    CREATE
        /*!50017 DEFINER = 'root'@'localhost' */
        TRIGGER `tr_a_insert` AFTER INSERT ON `t_a`
        FOR EACH ROW BEGIN
            --Trigger触发后,同时对t_b新增同步一条数据
            INSERT INTO `t_b` SET username = NEW.username, groupid=NEW.groupid;
        END;
    $$
     
    DELIMITER;
2、创建UPDATE触发器trigger_a_update:


    DELIMITER $$
     
    USE `test`$$
    --判断数据库中是否存在tr_a_update触发器
    DROP TRIGGER /*!50032 IF EXISTS */ `tr_a_update`$$
    --不存在tr_a_update触发器,开始创建触发器
    --Trigger触发条件为update成功后进行触发
    CREATE
        /*!50017 DEFINER = 'root'@'localhost' */
        TRIGGER `tr_a_update` AFTER UPDATE ON `t_a`
        FOR EACH ROW BEGIN
        --Trigger触发后,当t_a表groupid,username数据有更改时,对t_b表同步一条更新后的数据
          IF new.groupid != old.groupid OR old.username != new.username THEN
            UPDATE `t_b` SET groupid=NEW.groupid,username=NEW.username WHEREusername=OLD.username AND groupid=OLD.groupid;
          END IF;
              
        END;
    $$
     
    DELIMITER ;
3、创建DELETE触发器trigger_a_delete:


    DELIMITER $$
     
    USE `test`$$
    --判断数据库中是否存在tr_a_delete触发器
    DROP TRIGGER /*!50032 IF EXISTS */ `tr_a_delete`$$
    --不存在tr_a_delete触发器,开始创建触发器
    --Trigger触发条件为delete成功后进行触发
    CREATE
        /*!50017 DEFINER = 'root'@'localhost' */
        TRIGGER `tr_a_delete` AFTER DELETE ON `t_a`
        FOR EACH ROW BEGIN
            --t_a表数据删除后,t_b表关联条件相同的数据也同步删除
            DELETE FROM `t_b` WHERE username=Old.username AND groupid=OLD.groupid;
        END;
    $$
     
    DELIMITER ;

三、测试MySQL实例触发器:

分别测试实现t_a与t_b实现数据同步CUD(增、改、删)3个Triggers

1、测试MySQL的实例tr_a_insert触发器:

在t_a表中新增一条数据,然后分别查询t_a/t_b表的数据是否数据同步,测试触发器成功标志,t_a表无论在何种情况下,新增了一条或多条记录集时,没有t_b表做任何数据insert操作,它同时新增了一样的多条记录集。

下面来进行MySQL触发器实例测试:



    --t_a表新增一条记录集
        INSERT INTO `t_a` (username,groupid) VALUES ('sky54.net',123)
       
        --查询t_a表
        SELECT id,username,groupid FROM `t_a`
       
        --查询t_b表
        SELECT id,username,groupid FROM `t_b`

2、测试MySQL的实例tr_a_update、tr_a_delete触发器:

这两个MySQL触发器测试原理、步骤与tr_a_insert触发器一样的,先修改/删除一条数据,然后分别查看t_a、t_b表的数据变化情况,数据变化同步说明Trigger实例成功,否则需要逐步排查错误原因。

世界上任何一种事物都其其优点和缺点,优点与缺点是自身一个相对立的面。当然这里不是强调“世界非黑即白”式的“二元论”,“存在即合理”嘛。当然 MySQL触发器的优点不说了,说一下不足之处,MySQL Trigger没有很好的调试、管理环境,难于在各种系统环境下测试,测试比MySQL存储过程要难,所以建议在生成环境下,尽量用存储过程来代替 MySQL触发器。

本篇结束前再强调一下,支持触发器的MySQL版本需要5.0以上,5.0以前版本的MySQL升级到5.0以后版本方可使用触发器哦!








http://blog.csdn.net/hireboy/article/details/18079183



posted @ 2016-08-18 17:25 abin 阅读(268) | 评论 (0)编辑 收藏

2016年6月14日 #

     摘要: 在开发高并发系统时有三把利器用来保护系统:缓存、降级和限流。缓存的目的是提升系统访问速度和增大系统能处理的容量,可谓是抗高并发流量的银弹;而降级是当服务出问题或者影响到核心流程的性能则需要暂时屏蔽掉,待高峰或者问题解决后再打开;而有些场景并不能用缓存和降级来解决,比如稀缺资源(秒杀、抢购)、写服务(如评论、下单)、频繁的复杂查询(评论的最后几页),因此需有一种手段来限制这些场景的并发/请求量,即限...  阅读全文
posted @ 2016-06-14 13:38 abin 阅读(1356) | 评论 (1)编辑 收藏

2016年5月13日 #

Install the Command Line Client

If you prefer command line client, then you can install it on your Linux with the following command.

Debian

sudo apt-get install python-pip sudo pip install shadowsocks

Ubuntu

Yes, you can use the above commands to install shadowsocks client on ubuntu. But it will install it under ~/.local/bin/ directory and it causes loads of trouble. So I suggest using su to become root first and then issue the following two commands.

apt-get install python-pip pip install shadowsocks

Fedora/Centos

sudo yum install python-setuptools   or   sudo dnf install python-setuptools sudo easy_install pip sudo pip install shadowsocks

OpenSUSE

sudo zypper install python-pip sudo pip install shadowsocks

Archlinux

sudo pacman -S python-pip sudo pip install shadowsocks

As you can see the command of installing shadowsocks client is the same to the command of installing shadowsocks server, because the above command will install both the client and the server. You can verify this by looking at the installation script output

Downloading/unpacking shadowsocks Downloading shadowsocks-2.8.2.tar.gz Running setup.py (path:/tmp/pip-build-PQIgUg/shadowsocks/setup.py) egg_info for package shadowsocks  Installing collected packages: shadowsocks Running setup.py install for shadowsocks  Installing sslocal script to /usr/local/bin Installing ssserver script to /usr/local/bin Successfully installed shadowsocks Cleaning up...

sslocal is the client software and ssserver is the server software. On some Linux distros such as ubuntu, the shadowsocks client sslocal is installed under /usr/local/bin. On Others such as Archsslocal is installed under /usr/bin/. Your can use whereis command to find the exact location.

user@debian:~$ whereis sslocal sslocal: /usr/local/bin/sslocal

Create a Configuration File

we will create a configuration file under /etc/

sudo vi /etc/shadowsocks.json

Put the following text in the file. Replace server-ip with your actual IP and set a password.

{
"server":"server-ip",
"server_port":8000,
"local_address": "127.0.0.1",
"local_port":1080,
"password":"your-password",
"timeout":600,
"method":"aes-256-cfb"
}

Save and close the file. Next start the client using command line

sslocal -c /etc/shadowsocks.json

To run in the background

sudo sslocal -c /etc/shadowsocks.json -d start

Auto Start the Client on System Boot

Edit /etc/rc.local file

sudo vi /etc/rc.local

Put the following line above the exit 0 line:

sudo sslocal -c /etc/shadowsocks.json -d start

Save and close the file. Next time you start your computer, shadowsocks client will automatically start and connect to your shadowsocks server.

Check if It Works

After you rebooted your computer, enter the following command in terminal:

sudo systemctl status rc-local.service

If your sslocal command works then you will get this ouput:


● rc-local.service - /etc/rc.local 

Compatibility Loaded: loaded (/etc/systemd/system/rc-local.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2015-11-27 03:19:25 CST; 2min 39s ago
Process: 881 ExecStart=/etc/rc.local start (code=exited, status=0/SUCCESS)
CGroup: /system.slice/rc-local.service
├─ 887 watch -n 60 su matrix -c ibam
└─1112 /usr/bin/python /usr/local/bin/sslocal -c /etc/shadowsocks....

As you can see from the last line, the sslocal command created a process whose pid is 1112 on my machine. It means shadowsocks client is running smoothly. And of course you can tell your browser to connect through your shadowsocks client to see if everything goes well.

If for some reason your /etc/rc.local script won’t run, then check the following post to find the solution.

How to enable /etc/rc.local with SystemdInstall the Command Line Client

If you prefer command line client, then you can install it on your Linux with the following command.

Debian

sudo apt-get install python-pip
sudo pip install shadowsocks

Ubuntu

Yes, you can use the above commands to install shadowsocks client on ubuntu. But it will install it under ~/.local/bin/ directory and it causes loads of trouble. So I suggest using su to become root first and then issue the following two commands.

apt-get install python-pip
pip install shadowsocks

Fedora/Centos

sudo yum install python-setuptools   or   sudo dnf install python-setuptools
sudo easy_install pip
sudo pip install shadowsocks

OpenSUSE

sudo zypper install python-pip
sudo pip install shadowsocks

Archlinux

sudo pacman -S python-pip
sudo pip install shadowsocks

As you can see the command of installing shadowsocks client is the same to the command of installing shadowsocks server, because the above command will install both the client and the server. You can verify this by looking at the installation script output

Downloading/unpacking shadowsocks
Downloading shadowsocks-2.8.2.tar.gz
Running setup.py (path:/tmp/pip-build-PQIgUg/shadowsocks/setup.py) egg_info for package shadowsocks

Installing collected packages: shadowsocks
Running setup.py install for shadowsocks

Installing sslocal script to /usr/local/bin
Installing ssserver script to /usr/local/bin
Successfully installed shadowsocks
Cleaning up...

sslocal is the client software and ssserver is the server software. On some Linux distros such as ubuntu, the shadowsocks client sslocal is installed under /usr/local/bin. On Others such as Archsslocal is installed under /usr/bin/. Your can use whereis command to find the exact location.

user@debian:~$ whereis sslocal
sslocal: /usr/local/bin/sslocal

Create a Configuration File

we will create a configuration file under /etc/

sudo vi /etc/shadowsocks.json

Put the following text in the file. Replace server-ip with your actual IP and set a password.

{
"server":"server-ip",
"server_port":8000,
"local_address": "127.0.0.1",
"local_port":1080,
"password":"your-password",
"timeout":600,
"method":"aes-256-cfb"
}

Save and close the file. Next start the client using command line

sslocal -c /etc/shadowsocks.json

To run in the background

sudo sslocal -c /etc/shadowsocks.json -d start

Auto Start the Client on System Boot

Edit /etc/rc.local file

sudo vi /etc/rc.local

Put the following line above the exit 0 line:

sudo sslocal -c /etc/shadowsocks.json -d start

Save and close the file. Next time you start your computer, shadowsocks client will automatically start and connect to your shadowsocks server.

Check if It Works

After you rebooted your computer, enter the following command in terminal:

sudo systemctl status rc-local.service

If your sslocal command works then you will get this ouput:

● rc-local.service - /etc/rc.local Compatibility
Loaded: loaded (/etc/systemd/system/rc-local.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2015-11-27 03:19:25 CST; 2min 39s ago
Process: 881 ExecStart=/etc/rc.local start (code=exited, status=0/SUCCESS)
CGroup: /system.slice/rc-local.service
├─ 887 watch -n 60 su matrix -c ibam
└─1112 /usr/bin/python /usr/local/bin/sslocal -c /etc/shadowsocks....

As you can see from the last line, the sslocal command created a process whose pid is 1112 on my machine. It means shadowsocks client is running smoothly. And of course you can tell your browser to connect through your shadowsocks client to see if everything goes well.

If for some reason your /etc/rc.local script won’t run, then check the following post to find the solution.

How to enable /etc/rc.local with Systemd




posted @ 2016-05-13 22:56 abin 阅读(589) | 评论 (0)编辑 收藏

2016年4月27日 #

废话少说,直接上代码,以前都是调用别人写好的,现在有时间自己弄下,具体功能如下:
1、httpClient+http+线程池:
2、httpClient+https(单向不验证证书)+线程池:

https在%TOMCAT_HOME%/conf/server.xml里面的配置文件
<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true" 
     maxThreads="150" scheme="https" secure="true" 
     clientAuth="false" keystoreFile="D:/tomcat.keystore" 
     keystorePass="heikaim" sslProtocol="TLS"  executor="tomcatThreadPool"/> 
其中 clientAuth="false"表示不开启证书验证,只是单存的走https



package com.abin.lee.util;

import org.apache.commons.collections4.MapUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.*;
import org.apache.http.client.HttpRequestRetryHandler;
import org.apache.http.client.config.CookieSpecs;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.ConnectTimeoutException;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.message.BasicHeader;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.protocol.HttpContext;
import org.apache.http.util.EntityUtils;

import javax.net.ssl.*;
import java.io.IOException;
import java.io.InterruptedIOException;
import java.net.UnknownHostException;
import java.nio.charset.Charset;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import java.util.*;

/**
* Created with IntelliJ IDEA.
* User: abin
* Date: 16-4-18
* Time: 上午10:24
* To change this template use File | Settings | File Templates.
*/
public class HttpClientUtil {
private static CloseableHttpClient httpsClient = null;
private static CloseableHttpClient httpClient = null;

static {
httpClient = getHttpClient();
httpsClient = getHttpsClient();
}

public static CloseableHttpClient getHttpClient() {
try {
httpClient = HttpClients.custom()
.setConnectionManager(PoolManager.getHttpPoolInstance())
.setConnectionManagerShared(true)
.setDefaultRequestConfig(requestConfig())
.setRetryHandler(retryHandler())
.build();
} catch (Exception e) {
e.printStackTrace();
}
return httpClient;
}


public static CloseableHttpClient getHttpsClient() {
try {
//Secure Protocol implementation.
SSLContext ctx = SSLContext.getInstance("SSL");
//Implementation of a trust manager for X509 certificates
TrustManager x509TrustManager = new X509TrustManager() {
public void checkClientTrusted(X509Certificate[] xcs,
String string) throws CertificateException {
}
public void checkServerTrusted(X509Certificate[] xcs,
String string) throws CertificateException {
}
public X509Certificate[] getAcceptedIssuers() {
return null;
}
};
ctx.init(null, new TrustManager[]{x509TrustManager}, null);
//首先设置全局的标准cookie策略
// RequestConfig requestConfig = RequestConfig.custom().setCookieSpec(CookieSpecs.STANDARD_STRICT).build();
ConnectionSocketFactory connectionSocketFactory = new SSLConnectionSocketFactory(ctx, hostnameVerifier);
Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.INSTANCE)
.register("https", connectionSocketFactory).build();
// 设置连接池
httpsClient = HttpClients.custom()
.setConnectionManager(PoolsManager.getHttpsPoolInstance(socketFactoryRegistry))
.setConnectionManagerShared(true)
.setDefaultRequestConfig(requestConfig())
.setRetryHandler(retryHandler())
.build();
} catch (Exception e) {
e.printStackTrace();
}
return httpsClient;
}

// 配置请求的超时设置
//首先设置全局的标准cookie策略
public static RequestConfig requestConfig(){
RequestConfig requestConfig = RequestConfig.custom()
.setCookieSpec(CookieSpecs.STANDARD_STRICT)
.setConnectionRequestTimeout(20000)
.setConnectTimeout(20000)
.setSocketTimeout(20000)
.build();
return requestConfig;
}

public static HttpRequestRetryHandler retryHandler(){
//请求重试处理
HttpRequestRetryHandler httpRequestRetryHandler = new HttpRequestRetryHandler() {
public boolean retryRequest(IOException exception,int executionCount, HttpContext context) {
if (executionCount >= 5) {// 如果已经重试了5次,就放弃
return false;
}
if (exception instanceof NoHttpResponseException) {// 如果服务器丢掉了连接,那么就重试
return true;
}
if (exception instanceof SSLHandshakeException) {// 不要重试SSL握手异常
return false;
}
if (exception instanceof InterruptedIOException) {// 超时
return false;
}
if (exception instanceof UnknownHostException) {// 目标服务器不可达
return false;
}
if (exception instanceof ConnectTimeoutException) {// 连接被拒绝
return false;
}
if (exception instanceof SSLException) {// ssl握手异常
return false;
}

HttpClientContext clientContext = HttpClientContext.adapt(context);
HttpRequest request = clientContext.getRequest();
// 如果请求是幂等的,就再次尝试
if (!(request instanceof HttpEntityEnclosingRequest)) {
return true;
}
return false;
}
};
return httpRequestRetryHandler;
}



//创建HostnameVerifier
//用于解决javax.net.ssl.SSLException: hostname in certificate didn't match: <123.125.97.66> != <123.125.97.241>
static HostnameVerifier hostnameVerifier = new NoopHostnameVerifier(){
@Override
public boolean verify(String s, SSLSession sslSession) {
return super.verify(s, sslSession);
}
};


public static class PoolManager {
public static PoolingHttpClientConnectionManager clientConnectionManager = null;
private static int maxTotal = 200;
private static int defaultMaxPerRoute = 100;

private PoolManager(){
clientConnectionManager.setMaxTotal(maxTotal);
clientConnectionManager.setDefaultMaxPerRoute(defaultMaxPerRoute);
}

private static class PoolManagerHolder{
public static PoolManager instance = new PoolManager();
}

public static PoolManager getInstance() {
if(null == clientConnectionManager)
clientConnectionManager = new PoolingHttpClientConnectionManager();
return PoolManagerHolder.instance;
}

public static PoolingHttpClientConnectionManager getHttpPoolInstance() {
PoolManager.getInstance();
// System.out.println("getAvailable=" + clientConnectionManager.getTotalStats().getAvailable());
// System.out.println("getLeased=" + clientConnectionManager.getTotalStats().getLeased());
// System.out.println("getMax=" + clientConnectionManager.getTotalStats().getMax());
// System.out.println("getPending="+clientConnectionManager.getTotalStats().getPending());
return PoolManager.clientConnectionManager;
}


}

public static class PoolsManager {
public static PoolingHttpClientConnectionManager clientConnectionManager = null;
private static int maxTotal = 200;
private static int defaultMaxPerRoute = 100;

private PoolsManager(){
clientConnectionManager.setMaxTotal(maxTotal);
clientConnectionManager.setDefaultMaxPerRoute(defaultMaxPerRoute);
}

private static class PoolsManagerHolder{
public static PoolsManager instance = new PoolsManager();
}

public static PoolsManager getInstance(Registry<ConnectionSocketFactory> socketFactoryRegistry) {
if(null == clientConnectionManager)
clientConnectionManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);
return PoolsManagerHolder.instance;
}

public static PoolingHttpClientConnectionManager getHttpsPoolInstance(Registry<ConnectionSocketFactory> socketFactoryRegistry) {
PoolsManager.getInstance(socketFactoryRegistry);
// System.out.println("getAvailable=" + clientConnectionManager.getTotalStats().getAvailable());
// System.out.println("getLeased=" + clientConnectionManager.getTotalStats().getLeased());
// System.out.println("getMax=" + clientConnectionManager.getTotalStats().getMax());
// System.out.println("getPending="+clientConnectionManager.getTotalStats().getPending());
return PoolsManager.clientConnectionManager;
}

}

public static String httpPost(Map<String, String> request, String httpUrl){
String result = "";
CloseableHttpClient httpClient = getHttpClient();
try {
if(MapUtils.isEmpty(request))
throw new Exception("请求参数不能为空");
HttpPost httpPost = new HttpPost(httpUrl);
List<NameValuePair> nvps = new ArrayList<NameValuePair>();
for(Iterator<Map.Entry<String, String>> iterator=request.entrySet().iterator(); iterator.hasNext();){
Map.Entry<String, String> entry = iterator.next();
nvps.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
}
httpPost.setEntity(new UrlEncodedFormEntity(nvps, Consts.UTF_8));
System.out.println("Executing request: " + httpPost.getRequestLine());
CloseableHttpResponse response = httpClient.execute(httpPost);
result = EntityUtils.toString(response.getEntity());
System.out.println("Executing response: "+ result);
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
try {
httpClient.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return result;
}

public static String httpPost(String json, String httpUrl, Map<String, String> headers){
String result = "";
CloseableHttpClient httpClient = getHttpClient();
try {
if(StringUtils.isBlank(json))
throw new Exception("请求参数不能为空");
HttpPost httpPost = new HttpPost(httpUrl);
for(Iterator<Map.Entry<String, String>> iterator=headers.entrySet().iterator();iterator.hasNext();){
Map.Entry<String, String> entry = iterator.next();
Header header = new BasicHeader(entry.getKey(), entry.getValue());
httpPost.setHeader(header);
}
httpPost.setEntity(new StringEntity(json, Charset.forName("UTF-8")));
System.out.println("Executing request: " + httpPost.getRequestLine());
CloseableHttpResponse response = httpClient.execute(httpPost);
result = EntityUtils.toString(response.getEntity());
System.out.println("Executing response: "+ result);
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
try {
httpClient.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return result;
}

public static String httpGet(String httpUrl, Map<String, String> headers) {
String result = "";
CloseableHttpClient httpClient = getHttpClient();
try {
HttpGet httpGet = new HttpGet(httpUrl);
System.out.println("Executing request: " + httpGet.getRequestLine());
for(Iterator<Map.Entry<String, String>> iterator=headers.entrySet().iterator();iterator.hasNext();){
Map.Entry<String, String> entry = iterator.next();
Header header = new BasicHeader(entry.getKey(), entry.getValue());
httpGet.setHeader(header);
}
CloseableHttpResponse response = httpClient.execute(httpGet);
result = EntityUtils.toString(response.getEntity());
System.out.println("Executing response: "+ result);
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
try {
httpClient.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return result;
}


public static String httpGet(String httpUrl) {
String result = "";
CloseableHttpClient httpClient = getHttpClient();
try {
HttpGet httpGet = new HttpGet(httpUrl);
System.out.println("Executing request: " + httpGet.getRequestLine());
CloseableHttpResponse response = httpClient.execute(httpGet);
result = EntityUtils.toString(response.getEntity());
System.out.println("Executing response: "+ result);
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
try {
httpClient.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return result;
}





maven依赖:
  <!--httpclient-->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpcore</artifactId>
            <version>4.4.4</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpmime</artifactId>
            <version>4.5.2</version>
        </dependency>

<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>4.1</version>
</dependency>
posted @ 2016-04-27 19:04 abin 阅读(808) | 评论 (0)编辑 收藏

2015年11月3日 #

1、twemproxy explore

      当我们有大量 Redis 或 Memcached 的时候,通常只能通过客户端的一些数据分配算法(比如一致性哈希),来实现集群存储的特性。虽然Redis 2.6版本已经发布Redis Cluster,但还不是很成熟适用正式生产环境。 Redis 的 Cluster 方案还没有正式推出之前,我们通过 Proxy 的方式来实现集群存储

       Twitter,世界最大的Redis集群之一部署在Twitter用于为用户提供时间轴数据。Twitter Open Source部门提供了Twemproxy。

     Twemproxy,也叫nutcraker。是一个twtter开源的一个redis和memcache代理服务器。 redis作为一个高效的缓存服务器,非常具有应用价值。但是当使用比较多的时候,就希望可以通过某种方式 统一进行管理。避免每个应用每个客户端管理连接的松散性。同时在一定程度上变得可以控制。

      Twemproxy是一个快速的单线程代理程序,支持Memcached ASCII协议和更新的Redis协议:

     它全部用C写成,使用Apache 2.0 License授权。项目在Linux上可以工作,而在OSX上无法编译,因为它依赖了epoll API.

      Twemproxy 通过引入一个代理层,可以将其后端的多台 Redis 或 Memcached 实例进行统一管理与分配,使应用程序只需要在 Twemproxy 上进行操作,而不用关心后面具体有多少个真实的 Redis 或 Memcached 存储。 

2、twemproxy特性:

    • 支持失败节点自动删除

      • 可以设置重新连接该节点的时间
      • 可以设置连接多少次之后删除该节点
      • 该方式适合作为cache存储
    • 支持设置HashTag

      • 通过HashTag可以自己设定将两个KEYhash到同一个实例上去。
    • 减少与redis的直接连接数

      • 保持与redis的长连接
      • 可设置代理与后台每个redis连接的数目
    • 自动分片到后端多个redis实例上

      • 多种hash算法:能够使用不同的策略和散列函数支持一致性hash。
      • 可以设置后端实例的权重
    • 避免单点问题

      • 可以平行部署多个代理层.client自动选择可用的一个
    • 支持redis pipelining request

           支持请求的流式与批处理,降低来回的消耗

    • 支持状态监控

      • 可设置状态监控ip和端口,访问ip和端口可以得到一个json格式的状态信息串
      • 可设置监控信息刷新间隔时间
    • 高吞吐量

      • 连接复用,内存复用。
      • 将多个连接请求,组成reids pipelining统一向redis请求。

     另外可以修改redis的源代码,抽取出redis中的前半部分,作为一个中间代理层。最终都是通过linux下的epoll 事件机制提高并发效率,其中nutcraker本身也是使用epoll的事件机制。并且在性能测试上的表现非常出色。

3、twemproxy问题与不足


Twemproxy 由于其自身原理限制,有一些不足之处,如: 
  • 不支持针对多个值的操作,比如取sets的子交并补等(MGET 和 DEL 除外)
  • 不支持Redis的事务操作
  • 出错提示还不够完善
  • 也不支持select操作

4、安装与配置 

具体的安装步骤可用查看github:https://github.com/twitter/twemproxy
Twemproxy 的安装,主要命令如下: 
apt-get install automake  
apt-get install libtool  
git clone git://github.com/twitter/twemproxy.git  
cd twemproxy  
autoreconf -fvi  
./configure --enable-debug=log  
make  
src/nutcracker -h

通过上面的命令就算安装好了,然后是具体的配置,下面是一个典型的配置 
    redis1:  
      listen: 127.0.0.1:6379 #使用哪个端口启动Twemproxy  
      redis: true #是否是Redis的proxy  
      hash: fnv1a_64 #指定具体的hash函数  
      distribution: ketama #具体的hash算法  
      auto_eject_hosts: true #是否在结点无法响应的时候临时摘除结点  
      timeout: 400 #超时时间(毫秒)  
      server_retry_timeout: 2000 #重试的时间(毫秒)  
      server_failure_limit: 1 #结点故障多少次就算摘除掉  
      servers: #下面表示所有的Redis节点(IP:端口号:权重)  
       - 127.0.0.1:6380:1  
       - 127.0.0.1:6381:1  
       - 127.0.0.1:6382:1  
      
    redis2:  
      listen: 0.0.0.0:10000  
      redis: true  
      hash: fnv1a_64  
      distribution: ketama  
      auto_eject_hosts: false  
      timeout: 400  
      servers:  
       - 127.0.0.1:6379:1  
       - 127.0.0.1:6380:1  
       - 127.0.0.1:6381:1  
       - 127.0.0.1:6382:1 

你可以同时开启多个 Twemproxy 实例,它们都可以进行读写,这样你的应用程序就可以完全避免所谓的单点故障。


http://blog.csdn.net/hguisu/article/details/9174459/
posted @ 2015-11-03 19:30 abin 阅读(458) | 评论 (0)编辑 收藏

2015年11月1日 #

Linux is a powerhouse when it comes to networking, and provides a full featured and high performance network stack. When combined with web front-ends such asHAProxylighttpdNginxApache or your favorite application server, Linux is a killer platform for hosting web applications. Keeping these applications up and operational can sometimes be a challenge, especially in this age of horizontally scaled infrastructure and commodity hardware. But don't fret, since there are a number of technologies that can assist with making your applications and network infrastructure fault tolerant.

One of these technologies, keepalived, provides interface failover and the ability to perform application-layer health checks. When these capabilities are combined with the Linux Virtual Server (LVS) project, a fault in an application will be detected by keepalived, and the virtual interfaces that are accessed by clients can be migrated to another available node. This article will provide an introduction to keepalived, and will show how to configure interface failover between two or more nodes. Additionally, the article will show how to debug problems with keepalived and VRRP.

What Is Keepalived?


The keepalived project provides a keepalive facility for Linux servers. This keepalive facility consists of a VRRP implementation to manage virtual routers (aka virtual interfaces), and a health check facility to determine if a service (web server, samba server, etc.) is up and operational. If a service fails a configurable number of health checks, keepalived will fail a virtual router over to a secondary node. While useful in its own right, keepalived really shines when combined with the Linux Virtual Server project. This article will focus on keepalived, and a future article will show how to integrate the two to create a fault tolerant load-balancer.

Installing KeepAlived From Source Code


Before we dive into configuring keepalived, we need to install it. Keepalived is distributed as source code, and is available in several package repositories. To install from source code, you can execute wget or curl to retrieve the source, and then run "configure", "make" and "make install" compile and install the software:

$ wget http://www.keepalived.org/software/keepalived-1.1.17.tar.gz  $ tar xfvz keepalived-1.1.17.tar.gz   $ cd keepalived-1.1.17  $ ./configure --prefix=/usr/local  $ make && make install 

In the example above, the keepalived daemon will be compiled and installed as /usr/local/sbin/keepalived.

Configuring KeepAlived


The keepalived daemon is configured through a text configuration file, typically named keepalived.conf. This file contains one or more configuration stanzas, which control notification settings, the virtual interfaces to manage, and the health checks to use to test the services that rely on the virtual interfaces. Here is a sample annotated configuration that defines two virtual IP addresses to manage, and the individuals to contact when a state transition or fault occurs:

# Define global configuration directives global_defs {     # Send an e-mail to each of the following     # addresses when a failure occurs    notification_email {        matty@prefetch.net        operations@prefetch.net    }    # The address to use in the From: header    notification_email_from root@VRRP-director1.prefetch.net     # The SMTP server to route mail through    smtp_server mail.prefetch.net     # How long to wait for the mail server to respond    smtp_connect_timeout 30     # A descriptive name describing the router    router_id VRRP-director1 }  # Create a VRRP instance  VRRP_instance VRRP_ROUTER1 {      # The initial state to transition to. This option isn't     # really all that valuable, since an election will occur     # and the host with the highest priority will become     # the master. The priority is controlled with the priority     # configuration directive.     state MASTER      # The interface keepalived will manage     interface br0      # The virtual router id number to assign the routers to     virtual_router_id 100      # The priority to assign to this device. This controls     # who will become the MASTER and BACKUP for a given     # VRRP instance.     priority 100      # How many seconds to wait until a gratuitous arp is sent     garp_master_delay 2      # How often to send out VRRP advertisements     advert_int 1      # Execute a notification script when a host transitions to     # MASTER or BACKUP, or when a fault occurs. The arguments     # passed to the script are:     #  $1 - "GROUP"|"INSTANCE"     #  $2 = name of group or instance     #  $3 = target state of transition     # Sample: VRRP-notification.sh VRRP_ROUTER1 BACKUP 100     notify "/usr/local/bin/VRRP-notification.sh"      # Send an SMTP alert during a state transition     smtp_alert      # Authenticate the remote endpoints via a simple      # username/password combination     authentication {         auth_type PASS         auth_pass 192837465     }     # The virtual IP addresses to float between nodes. The     # label statement can be used to bring an interface      # online to represent the virtual IP.     virtual_ipaddress {         192.168.1.100 label br0:100         192.168.1.101 label br0:101     } } 

The configuration file listed above is self explanatory, so I won't go over each directive in detail. I will point out a couple of items:

  • Each host is referred to as a director in the documentation, and each director can be responsible for one or more VRRP instances
  • Each director will need its own copy of the configuration file, and the router_id, priority, etc. should be adjusted to reflect the nodes name and priority relative to other nodes
  • To force a specific node to master a virtual address, make sure the director's priority is higher than the other virtual routers
  • If you have multiple VRRP instances that need to failover together, you will need to add each instance to a VRRP_sync_group
  • The notification script can be used to generate custom syslog messages, or to invoke some custom logic (e.g., restart an app) when a state transition or fault occurs
  • The keepalived package comes with numerous configuration examples, which show how to configure numerous aspects of the server

Starting Keepalived


Keepalived can be executed from an RC script, or started from the command line. The following example will start keepalived using the configuration file /usr/local/etc/keepalived.conf:

$ keepalived -f /usr/local/etc/keepalived.conf 

If you need to debug keepalived issues, you can run the daemon with the "--dont-fork", "--log-console" and "--log-detail" options:

$ keepalived -f /usr/local/etc/keepalived.conf --dont-fork --log-console --log-detail 

These options will stop keepalived from fork'ing, and will provide additional logging data. Using these options is especially useful when you are testing out new configuration directives, or debugging an issue with an existing configuration file.

Locating The Router That is Managing A Virtual IP


To see which director is currently the master for a given virtual interface, you can check the output from the ip utility:

VRRP-director1$ ip addr list br0 5: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN      link/ether 00:24:8c:4e:07:f6 brd ff:ff:ff:ff:ff:ff     inet 192.168.1.6/24 brd 192.168.1.255 scope global br0     inet 192.168.1.100/32 scope global br0:100     inet 192.168.1.101/32 scope global br0:101     inet6 fe80::224:8cff:fe4e:7f6/64 scope link         valid_lft forever preferred_lft forever  VRRP-director2$ ip addr list br0 5: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN      link/ether 00:24:8c:4e:07:f6 brd ff:ff:ff:ff:ff:ff     inet 192.168.1.7/24 brd 192.168.1.255 scope global br0     inet6 fe80::224:8cff:fe4e:7f6/64 scope link         valid_lft forever preferred_lft forever 

In the output above, we can see that the virtual interfaces 192.168.1.100 and 192.168.1.101 are currently active on VRRP-director1.

Troubleshooting Keepalived And VRRP


The keepalived daemon will log to syslog by default. Log entries will range from entries that show when the keepalive daemon started, to entries that show state transitions. Here are a few sample entries that show keepalived starting up, and the node transitioning a VRRP instance to the MASTER state:

Jul  3 16:29:56 disarm Keepalived: Starting Keepalived v1.1.17 (07/03,2009) Jul  3 16:29:56 disarm Keepalived: Starting VRRP child process, pid=1889 Jul  3 16:29:56 disarm Keepalived_VRRP: Using MII-BMSR NIC polling thread... Jul  3 16:29:56 disarm Keepalived_VRRP: Registering Kernel netlink reflector Jul  3 16:29:56 disarm Keepalived_VRRP: Registering Kernel netlink command channel Jul  3 16:29:56 disarm Keepalived_VRRP: Registering gratutious ARP shared channel Jul  3 16:29:56 disarm Keepalived_VRRP: Opening file '/usr/local/etc/keepalived.conf'. Jul  3 16:29:56 disarm Keepalived_VRRP: Configuration is using : 62990 Bytes Jul  3 16:29:57 disarm Keepalived_VRRP: VRRP_Instance(VRRP_ROUTER1) Transition to MASTER STATE Jul  3 16:29:58 disarm Keepalived_VRRP: VRRP_Instance(VRRP_ROUTER1) Entering MASTER STATE Jul  3 16:29:58 disarm Keepalived_VRRP: Netlink: skipping nl_cmd msg... 

If you are unable to determine the source of a problem with the system logs, you can use tcpdump to display the VRRP advertisements that are sent on the local network. Advertisements are sent to a reserved VRRP multicast address (224.0.0.18), so the following filter can be used to display all VRRP traffic that is visible on the interface passed to the "-i" option:

$ tcpdump -vvv -n -i br0 host 224.0.0.18 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on br0, link-type EN10MB (Ethernet), capture size 96 bytes  10:18:23.621512 IP (tos 0x0, ttl 255, id 102, offset 0, flags [none], proto VRRP (112), length 40) \                 192.168.1.6 > 224.0.0.18: VRRPv2, Advertisement, vrid 100, prio 100, authtype simple,                  intvl 1s, length 20, addrs: 192.168.1.100 auth "19283746"  10:18:25.621977 IP (tos 0x0, ttl 255, id 103, offset 0, flags [none], proto VRRP (112), length 40) \                 192.168.1.6 > 224.0.0.18: VRRPv2, Advertisement, vrid 100, prio 100, authtype simple,                  intvl 1s, length 20, addrs: 192.168.1.100 auth "19283746"                          ......... 

The output contains several pieces of data that be useful for debugging problems:

authtype - the type of authentication in use (authentication configuration directive) vrid - the virtual router id (virtual_router_id configuration directive) prio - the priority of the device (priority configuration directive) intvl - how often to send out advertisements (advert_int configuration directive) auth - the authentication token sent (auth_pass configuration directive) 

Conclusion


In this article I described how to set up a host to use the keepalived daemon, and provided a sample configuration file that can be used to failover virtual interfaces between servers. Keepalived has a slew of options not covered here, and I will refer you to the keepalived source code and documentation for additional details

posted @ 2015-11-01 21:06 abin 阅读(561) | 评论 (0)编辑 收藏

2015年10月12日 #

在Keepalived集群中,其实并没有严格意义上的主、备节点,虽然可以在Keepalived配置文件中设置“state”选项为“MASTER”状态,但是这并不意味着此节点一直就是Master角色。控制节点角色的是Keepalived配置文件中的“priority”值,但并它并不控制所有节点的角色,另一个能改变节点角色的是在vrrp_script模块中设置的“weight”值,这两个选项对应的都是一个整数值,其中“weight”值可以是个负整数,一个节点在集群中的角色就是通过这两个值的大小决定的。

在一个一主多备的Keepalived集群中,“priority”值最大的将成为集群中的Master节点,而其他都是Backup节点。在Master节点发生故障后,Backup节点之间将进行“民主选举”,通过对节点优先级值“priority”和““weight”的计算,选出新的Master节点接管集群服务。


在vrrp_script模块中,如果不设置“weight”选项值,那么集群优先级的选择将由Keepalived配置文件中的“priority”值决定,而在需要对集群中优先级进行灵活控制时,可以通过在vrrp_script模块中设置“weight”值来实现。下面列举一个实例来具体说明。


假定有A和B两节点组成的Keepalived集群,在A节点keepalived.conf文件中,设置“priority”值为100,而在B节点keepalived.conf文件中,设置“priority”值为80,并且A、B两个节点都使用了“vrrp_script”模块来监控mysql服务,同时都设置“weight”值为10,那么将会发生如下情况。


在两节点都启动Keepalived服务后,正常情况是A节点将成为集群中的Master节点,而B自动成为Backup节点,此时将A节点的mysql服务关闭,通过查看日志发现,并没有出现B节点接管A节点的日志,B节点仍然处于Backup状态,而A节点依旧是Master状态,在这种情况下整个HA集群将失去意义。


下面就分析一下产生这种情况的原因,这也就是Keepalived集群中主、备角色选举策略的问题。下面总结了在Keepalived中使用vrrp_script模块时整个集群角色的选举算法,由于“weight”值可以是正数也可以是负数,因此,要分两种情况进行说明。


1. “weight”值为正数时

在vrrp_script中指定的脚本如果检测成功,那么Master节点的权值将是“weight值与”priority“值之和,如果脚本检测失败,那么Master节点的权值保持为“priority”值,因此切换策略为:

Master节点“vrrp_script”脚本检测失败时,如果Master节点“priority”值小于Backup节点“weight值与”priority“值之和,将发生主、备切换。

Master节点“vrrp_script”脚本检测成功时,如果Master节点“weight”值与“priority”值之和大于Backup节点“weight”值与“priority”值之和,主节点依然为主节点,不发生切换。


2. “weight”值为负数时

在“vrrp_script”中指定的脚本如果检测成功,那么Master节点的权值仍为“priority”值,当脚本检测失败时,Master节点的权值将是“priority“值与“weight”值之差,因此切换策略为:

Master节点“vrrp_script”脚本检测失败时,如果Master节点“priority”值与“weight”值之差小于Backup节点“priority”值,将发生主、备切换。

Master节点“vrrp_script”脚本检测成功时,如果Master节点“priority”值大于Backup节点“priority”值时,主节点依然为主节点,不发生切换。


在熟悉了Keepalived主、备角色的选举策略后,再来分析一下刚才实例,由于A、B两个节点设置的“weight”值都为10,因此符合选举策略的第一种,在A节点停止Mysql服务后,A节点的脚本检测将失败,此时A节点的权值将保持为A节点上设置的“priority”值,即为100,而B节点的权值将变为“weight”值与“priority”值之和,也就是90(10+80),这样就出现了A节点权值仍然大于B节点权值的情况,因此不会发生主、备切换。


对于“weight”值的设置,有一个简单的标准,即“weight”值的绝对值要大于Master和Backup节点“priority”值之差。对于上面A、B两个节点的例子,只要设置“weight”值大于20即可保证集群正常运行和切换。由此可见,对于“weight值的设置,要非常谨慎,如果设置不好,将导致集群角色选举失败,使集群陷于瘫痪状态。

posted @ 2015-10-12 00:50 abin 阅读(438) | 评论 (0)编辑 收藏

2015年10月11日 #

如果你在读这篇文章,说明你跟大多数开发者一样对GIT感兴趣,如果你还没有机会来试一试GIT,我想现在你就要了解它了。

GIT不仅仅是个版本控制系统,它也是个内容管理系统(CMS),工作管理系统等。如果你是一个具有使用SVN背景的人,你需要做一定的思想转换,来适应GIT提供的一些概念和特征。所以,这篇文章的主要目的就是通过介绍GIT能做什么、它和SVN在深层次上究竟有什么不同来帮助你认识它。

那好,这就开始吧…

1.GIT是分布式的,SVN不是:

这是GIT和其它非分布式的版本控制系统,例如SVN,CVS等,最核心的区别。如果你能理解这个概念,那么你就已经上手一半了。需要做一点声明,GIT并不是目前第一个或唯一的分布式版本控制系统。还有一些系统,例如BitkeeperMercurial等,也是运行在分布式模式上的。但GIT在这方面做的更好,而且有更多强大的功能特征。

GIT跟SVN一样有自己的集中式版本库或服务器。但,GIT更倾向于被使用于分布式模式,也就是每个开发人员从中心版本库/服务器上chect out代码后会在自己的机器上克隆一个自己的版本库。可以这样说,如果你被困在一个不能连接网络的地方时,就像在飞机上,地下室,电梯里等,你仍然能够提 交文件,查看历史版本记录,创建项目分支,等。对一些人来说,这好像没多大用处,但当你突然遇到没有网络的环境时,这个将解决你的大麻烦。

同样,这种分布式的操作模式对于开源软件社区的开发来说也是个巨大的恩赐,你不必再像以前那样做出补丁包,通过email方式发送出去,你只需要创建一个分支,向项目团队发送一个推请求。这能让你的代码保持最新,而且不会在传输过程中丢失。GitHub.com就是一个这样的优秀案例。

有些谣言传出来说subversion将来的版本也会基于分布式模式。但至少目前还看不出来。

2.GIT把内容按元数据方式存储,而SVN是按文件:

所有的资源控制系统都是把文件的元信息隐藏在一个类似.svn,.cvs等的文件夹里。如果你把.git目录的 体积大小跟.svn比较,你会发现它们差距很大。因为,.git目录是处于你的机器上的一个克隆版的版本库,它拥有中心版本库上所有的东西,例如标签,分 支,版本记录等。

3.GIT分支和SVN的分支不同:

分支在SVN中一点不特别,就是版本库中的另外的一个目录。如果你想知道是否合并了一个分支,你需要手工运行像这样的命令svn propget svn:mergeinfo,来确认代码是否被合并。感谢Ben同学指出这个特征。所以,经常会发生有些分支被遗漏的情况。

然而,处理GIT的分支却是相当的简单和有趣。你可以从同一个工作目录下快速的在几个分支间切换。你很容易发现未被合并的分支,你能简单而快捷的合并这些文件。

4.GIT没有一个全局的版本号,而SVN有:

目前为止这是跟SVN相比GIT缺少的最大的一个特征。你也知道,SVN的版本号实际是任何一个相应时间的源代 码快照。我认为它是从CVS进化到SVN的最大的一个突破。因为GIT和SVN从概念上就不同,我不知道GIT里是什么特征与之对应。如果你有任何的线 索,请在评论里奉献出来与大家共享。

更新:有些读者指出,我们可以使用GIT的SHA-1来唯一的标识一个代码快照。这个并不能完全的代替SVN里容易阅读的数字版本号。但,用途应该是相同的。

5.GIT的内容完整性要优于SVN:

GIT的内容存储使用的是SHA-1哈希算法。这能确保代码内容的完整性,确保在遇到磁盘故障和网络问题时降低对版本库的破坏。这里有一个很好的关于GIT内容完整性的讨论 –http://stackoverflow.com/questions/964331/git-file-integrity

GIT和SVN之间只有这五处不同吗?当然不是。我想这5个只是“最基本的”“最吸引人”的,我只想到这5点。如果你发现有比这5点更有趣的,请共享出来,欢迎。




posted @ 2015-10-11 22:41 abin 阅读(376) | 评论 (0)编辑 收藏