歪歪之数据库

石建同学 — Sun, 07 Nov 2010 09:53:00 GMT

题记：主要记录同学分享的关于数据库设计方面的内容，思考过一点，记录下来。

一、从需求开始，考虑数据库的设计，且结合具体数据库特性

  一个论坛，要求显示帖子的总条数，对于mysql、innodb引擎；上线前期完全没有问题，当人气积累，帖子达到千万级别时，此时性能的问题就会显现出来。好的设计，是需求阶段就要考虑的。对于我，这是一个思想概念上的转变。
  但引擎如果换成myisam，就算数据达到亿的级别，“统计总数”这样的问题还会存在吗？不会，myisam自身就会维护总数信息。因此设计，必须结合具体的数据库的特性来做，不能以一概全。

二、应该遵循原则（未验证）

  1.小结果集驱动大结果集。理由：mysql的连接查询原理？
  2.尽可能在索引中完成排序。理由：索引本身就是有序的
  3.只取自己需要的column ？
  4.避免复杂的连接查询和子查询。
  5.适当的数据冗余.理由：帖子表&用户表，如果帖子表拥有username，则每次帖子的显示是不需要连接查询获取username。
  6.应用层的cahce机制？

三、概念

  1.垂直拆分：按列进行分割，即把一条记录分开多个地方保存，每个子表的行数相同。帖子表，id、userid、username、content、xxx ...前面4个字段很常用，但是后面xxx等很多字段，不常用，数据量很大。进行垂直拆分，table1字段包含“id、userid、username、content”,table2包含“xxx、...”等不常用字段。优点：减少io的操作。缺点：如果需要不常用字段信息，需要连表查询。
  2.水平拆分：按记录进分分割，不同的记录分开保存，每个子表的列数相同。比如：淘宝的用户交易数据，根据用户id取模，确定具体的数据存放在那个数据库。水平拆分会给应用带来复杂性。
  3.集群：提高系统的可用性。分库的节点引入多台机器，每台机器保存的数据是一样，负载均衡在多台机器。如何均衡、探测机器的可用性，是新的问题？
  4.主备：一般的互联网应用中，经过一些数据调查得出结论，读/写的比例大概在 10：1左右。为什么要读写分离：写操作涉及到锁的问题，不管是行锁还是表锁还是块锁，在大并发的情况下，效率很低。写操作集中在一个节点上，而读操作其其他的N个节点上进行。读写分离引入的新问题：比如我的Master上的数据怎样和集群中其它Slave机器保持数据的同步和一致呢?

石建同学 2010-11-07 17:53 发表评论

sql连接查询

石建同学 — Wed, 03 Nov 2010 07:36:00 GMT

请参考：http://en.wikipedia.org/wiki/Join_(SQL)#Sample_tables

inner JOINS

  An inner join is the most common join operation used in applications and can be regarded as the default join-type. Inner join creates a new result table by combining column values of two tables (A and B) based upon the join-predicate. The query compares each row of A with each row of B to find all pairs of rows which satisfy the join-predicate. When the join-predicate is satisfied, column values for each matched pair of rows of A and B are combined into a result row. The result of the join can be defined as the outcome of first taking the Cartesian product (or cross-join) of all records in the tables (combining every record in table A with every record in table B)—then return all records which satisfy the join predicate. Actual SQL implementations normally use other approaches like a Hash join or a Sort-merge join where possible, since computing the Cartesian product is very inefficient.

  注意：innner查询（默认的连接查询方式），是先查询“Cartesian”生成中间表，再根据where条件筛选结果；但此方法非常低效，SQL具体的实现可能是 Hash join or a Sort-merge join 。
   One can further classify inner joins as equi-joins, as natural joins, or as cross-joins.

SELECT *
FROM employee INNER JOIN department
ON employee.DepartmentID = department.DepartmentID;
The following example shows a query which is equivalent to the one from the previous example.

SELECT *
FROM   employee, department
WHERE  employee.DepartmentID = department.DepartmentID;

Outer joins

An outer join does not require each record in the two joined tables to have a matching record. The joined table retains each record—even if no other matching record exists. Outer joins subdivide further into left outer joins, right outer joins, and full outer joins, depending on which table(s) one retains the rows from (left, right, or both).

Example of a left outer join, with the additional result row italicized:

SELECT *
FROM   employee  LEFT OUTER JOIN department
ON employee.DepartmentID = department.DepartmentID;

Employee.LastName	Employee.DepartmentID	Department.DepartmentName	Department.DepartmentID
Jones	33	Engineering	33
Rafferty	31	Sales	31
Robinson	34	Clerical	34
Smith	34	Clerical	34
John	NULL	NULL	NULL
Steinberg	33	Engineering	33

Example right outer join, with the additional result row italicized:

SELECT *
FROM   employee RIGHT OUTER JOIN department
ON employee.DepartmentID = department.DepartmentID;

Employee.LastName	Employee.DepartmentID	Department.DepartmentName	Department.DepartmentID
Smith	34	Clerical	34
Jones	33	Engineering	33
Robinson	34	Clerical	34
Steinberg	33	Engineering	33
Rafferty	31	Sales	31
NULL	NULL	Marketing	35

Example full outer join: （mysql is not support）

SELECT *
FROM   employee
FULL OUTER JOIN department
ON employee.DepartmentID = department.DepartmentID;

Employee.LastName	Employee.DepartmentID	Department.DepartmentName	Department.DepartmentID
Smith	34	Clerical	34
Jones	33	Engineering	33
Robinson	34	Clerical	34
John	NULL	NULL	NULL
Steinberg	33	Engineering	33
Rafferty	31	Sales	31
NULL	NULL	Marketing	35

Self-join

A query to find all pairings of two employees in the same country is desired.

An example solution query could be as follows:

SELECT F.EmployeeID, F.LastName, S.EmployeeID, S.LastName, F.Country
FROM Employee F, Employee S
WHERE F.Country = S.Country
AND F.EmployeeID < S.EmployeeID
ORDER BY F.EmployeeID, S.EmployeeID;

Which results in the following table being generated.

Employee Table after Self-join by Country
EmployeeID	LastName	EmployeeID	LastName	Country
123	Rafferty	124	Jones	Australia
123	Rafferty	145	Steinberg	Australia
124	Jones	145	Steinberg	Australia
305	Smith	306	John	Germany

Join algorithms

Three fundamental algorithms exist for performing a join operation: Nested loop join, Sort-merge join and Hash join.