1 基本场景

比如你有 N 个 cache 服务器（后面简称 cache ），那么如何将一个对象 object 映射到 N 个 cache 上呢，你很可能会采用类似下面的通用方法计算 object 的 hash 值，然后均匀的映射到到 N 个 cache ；

hash(object)%N

一切都运行正常，再考虑如下的两种情况；

1 一个 cache 服务器 m down 掉了（在实际应用中必须要考虑这种情况），这样所有映射到 cache m 的对象都会失效，怎么办，需要把 cache m 从 cache 中移除，这时候 cache 是 N-1 台，映射公式变成了 hash(object)%(N-1) ；

2 由于访问加重，需要添加 cache ，这时候 cache 是 N+1 台，映射公式变成了 hash(object)%(N+1) ；

1 和 2 意味着什么？这意味着突然之间几乎所有的 cache 都失效了。对于服务器而言，这是一场灾难，洪水般的访问都会直接冲向后台服务器；

再来考虑第三个问题，由于硬件能力越来越强，你可能想让后面添加的节点多做点活，显然上面的 hash 算法也做不到。

  有什么方法可以改变这个状况呢，这就是 consistent hashing...

2 hash 算法和单调性

　　 Hash 算法的一个衡量指标是单调性（ Monotonicity ），定义如下：

　　单调性是指如果已经有一些内容通过哈希分派到了相应的缓冲中，又有新的缓冲加入到系统中。哈希的结果应能够保证原有已分配的内容可以被映射到新的缓冲中去，而不会被映射到旧的缓冲集合中的其他缓冲区。

容易看到，上面的简单 hash 算法 hash(object)%N 难以满足单调性要求。

3 consistent hashing 算法的原理

consistent hashing 是一种 hash 算法，简单的说，在移除 / 添加一个 cache 时，它能够尽可能小的改变已存在 key 映射关系，尽可能的满足单调性的要求。

下面就来按照 5 个步骤简单讲讲 consistent hashing 算法的基本原理。

3.1 环形hash 空间

考虑通常的 hash 算法都是将 value 映射到一个 32 为的 key 值，也即是 0~2^32-1 次方的数值空间；我们可以将这个空间想象成一个首（ 0 ）尾（ 2^32-1 ）相接的圆环，如下面图 1 所示的那样。

图 1 环形 hash 空间

3.2 把对象映射到hash 空间

接下来考虑 4 个对象 object1~object4 ，通过 hash 函数计算出的 hash 值 key 在环上的分布如图 2 所示。

hash(object1) = key1;

… …

hash(object4) = key4;

图 2 4 个对象的 key 值分布

3.3 把cache 映射到hash 空间

Consistent hashing 的基本思想就是将对象和 cache 都映射到同一个 hash 数值空间中，并且使用相同的hash 算法。

假设当前有 A,B 和 C 共 3 台 cache ，那么其映射结果将如图 3 所示，他们在 hash 空间中，以对应的 hash值排列。

hash(cache A) = key A;

… …

hash(cache C) = key C;

图 3 cache 和对象的 key 值分布

说到这里，顺便提一下 cache 的 hash 计算，一般的方法可以使用 cache 机器的 IP 地址或者机器名作为hash 输入。

3.4 把对象映射到cache

现在 cache 和对象都已经通过同一个 hash 算法映射到 hash 数值空间中了，接下来要考虑的就是如何将对象映射到 cache 上面了。

在这个环形空间中，如果沿着顺时针方向从对象的 key 值出发，直到遇见一个 cache ，那么就将该对象存储在这个 cache 上，因为对象和 cache 的 hash 值是固定的，因此这个 cache 必然是唯一和确定的。这样不就找到了对象和 cache 的映射方法了吗？！

依然继续上面的例子（参见图 3 ），那么根据上面的方法，对象 object1 将被存储到 cache A 上； object2和 object3 对应到 cache C ； object4 对应到 cache B ；

3.5 考察cache 的变动

前面讲过，通过 hash 然后求余的方法带来的最大问题就在于不能满足单调性，当 cache 有所变动时，cache 会失效，进而对后台服务器造成巨大的冲击，现在就来分析分析 consistent hashing 算法。

3.5.1 移除 cache

考虑假设 cache B 挂掉了，根据上面讲到的映射方法，这时受影响的将仅是那些沿 cache B 逆时针遍历直到下一个 cache （ cache C ）之间的对象，也即是本来映射到 cache B 上的那些对象。

因此这里仅需要变动对象 object4 ，将其重新映射到 cache C 上即可；参见图 4 。

图 4 Cache B 被移除后的 cache 映射

3.5.2 添加 cache

再考虑添加一台新的 cache D 的情况，假设在这个环形 hash 空间中， cache D 被映射在对象 object2 和object3 之间。这时受影响的将仅是那些沿 cache D 逆时针遍历直到下一个 cache （ cache B ）之间的对象（它们是也本来映射到 cache C 上对象的一部分），将这些对象重新映射到 cache D 上即可。

因此这里仅需要变动对象 object2 ，将其重新映射到 cache D 上；参见图 5 。

图 5 添加 cache D 后的映射关系

4 虚拟节点

考量 Hash 算法的另一个指标是平衡性 (Balance) ，定义如下：

平衡性

　　平衡性是指哈希的结果能够尽可能分布到所有的缓冲中去，这样可以使得所有的缓冲空间都得到利用。

hash 算法并不是保证绝对的平衡，如果 cache 较少的话，对象并不能被均匀的映射到 cache 上，比如在上面的例子中，仅部署 cache A 和 cache C 的情况下，在 4 个对象中， cache A 仅存储了 object1 ，而 cache C 则存储了 object2 、 object3 和 object4 ；分布是很不均衡的。

为了解决这种情况， consistent hashing 引入了“虚拟节点”的概念，它可以如下定义：

“虚拟节点”（ virtual node ）是实际节点在 hash 空间的复制品（ replica ），一实际个节点对应了若干个“虚拟节点”，这个对应个数也成为“复制个数”，“虚拟节点”在 hash 空间中以 hash 值排列。

仍以仅部署 cache A 和 cache C 的情况为例，在图 4 中我们已经看到， cache 分布并不均匀。现在我们引入虚拟节点，并设置“复制个数”为 2 ，这就意味着一共会存在 4 个“虚拟节点”， cache A1, cache A2 代表了 cache A ； cache C1, cache C2 代表了 cache C ；假设一种比较理想的情况，参见图 6 。

图 6 引入“虚拟节点”后的映射关系

此时，对象到“虚拟节点”的映射关系为：

objec1->cache A2 ； objec2->cache A1 ； objec3->cache C1 ； objec4->cache C2 ；

因此对象 object1 和 object2 都被映射到了 cache A 上，而 object3 和 object4 映射到了 cache C 上；平衡性有了很大提高。

引入“虚拟节点”后，映射关系就从 { 对象 -> 节点 } 转换到了 { 对象 -> 虚拟节点 } 。查询物体所在 cache时的映射关系如图 7 所示。

图 7 查询对象所在 cache

“虚拟节点”的 hash 计算可以采用对应节点的 IP 地址加数字后缀的方式。例如假设 cache A 的 IP 地址为202.168.14.241 。

引入“虚拟节点”前，计算 cache A 的 hash 值：

Hash(“202.168.14.241”);

引入“虚拟节点”后，计算“虚拟节”点 cache A1 和 cache A2 的 hash 值：

Hash(“202.168.14.241#1”);  // cache A1

Hash(“202.168.14.241#2”);  // cache A2

5 小结

Consistent hashing 的基本原理就是这些，具体的分布性等理论分析应该是很复杂的，不过一般也用不到。

http://weblogs.java.net/blog/2007/11/27/consistent-hashing 上面有一个 java 版本的例子，可以参考。

http://blog.csdn.net/mayongzhan/archive/2009/06/25/4298834.aspx 转载了一个 PHP 版的实现代码。

http://www.codeproject.com/KB/recipes/lib-conhash.aspx C语言版本

一些参考资料地址：

http://portal.acm.org/citation.cfm?id=258660

http://en.wikipedia.org/wiki/Consistent_hashing

http://www.spiteful.com/2008/03/17/programmers-toolbox-part-3-consistent-hashing/

 http://weblogs.java.net/blog/2007/11/27/consistent-hashing

http://tech.idv2.com/2008/07/24/memcached-004/

http://blog.csdn.net/mayongzhan/archive/2009/06/25/4298834.aspx

每个帖子前面有一个向上的三角形，如果你觉得这个内容很好，就点击一下，投上一票。根据得票数，系统自动统计出热门文章排行榜。但是，并非得票最多的文章排在第一位，还要考虑时间因素，新文章应该比旧文章更容易得到好的排名。

Hacker News使用Paul Graham开发的Arc语言编写，源码可以从arclanguage.org下载。它的排名算法是这样实现的：

将上面的代码还原为数学公式：

其中，

　　P表示帖子的得票数，减去1是为了忽略发帖人的投票。

　　T表示距离发帖的时间（单位为小时），加上2是为了防止最新的帖子导致分母过小（之所以选择2，可能是因为从原始文章出现在其他网站，到转贴至Hacker News，平均需要两个小时）。

　　G表示"重力因子"（gravityth power），即将帖子排名往下拉的力量，默认值为1.8，后文会详细讨论这个值。

从这个公式来看，决定帖子排名有三个因素：

第一个因素是得票数P。

在其他条件不变的情况下，得票越多，排名越高。

从上图可以看到，有三个同时发表的帖子，得票分别为200票、60票和30票（减1后为199、59和29），分别以黄色、紫色和蓝色表示。在任一个时间点上，都是黄色曲线在最上方，蓝色曲线在最下方。

如果你不想让"高票帖子"与"低票帖子"的差距过大，可以在得票数上加一个小于1的指数，比如(P-1)^0.8。

第二个因素是距离发帖的时间T。

在其他条件不变的情况下，越是新发表的帖子，排名越高。或者说，一个帖子的排名，会随着时间不断下降。

从前一张图可以看到，经过24小时之后，所有帖子的得分基本上都小于1，这意味着它们都将跌到排行榜的末尾，保证了排名前列的都将是较新的内容。

第三个因素是重力因子G。

它的数值大小决定了排名随时间下降的速度。

从上图可以看到，三根曲线的其他参数都一样，G的值分别为1.5、1.8和2.0。G值越大，曲线越陡峭，排名下降得越快，意味着排行榜的更新速度越快。

知道了算法的构成，就可以调整参数的值，以适用你自己的应用程序。

 說明

解法

1. 如果圖中W所在的位置為白色，則W+1，表示未處理的部份移至至白色群組。
2. 如果W部份為藍色，則B與W的元素對調，而B與W必須各+1，表示兩個群組都多了一個元素。
3. 如果W所在的位置是紅色，則將W與R交換，但R要減1，表示未處理的部份減1。

　　写这篇博还是有初衷的：

　　之前学数据结构的时候自己看书、也上网上查了很多资料，资料都比较散、而且描述的不是很清楚，对于当时刚刚

接触算法的我，要完全理解还是有一定难度。今天刚好有时间就整理了下思路、重写分析了一下之前的疑惑的地方、

没有透彻的地方便都豁然开朗了。所以迫不及待把我的想法记录下来，和大家分享。

　　如果你也是和之前的我一样对hanoi tower没能完全消化，或者刚刚接触汉诺塔，那希望我的这种理解方式能给你些

许帮助，如果你觉得已经完全掌握的比较牢靠了，那也可以看看，有好的idea可以一起分享；毕竟交流讨论也是一种很好的

学习方式。

　　好了，废话不多说，切入正题。

关于汉诺塔起源啊、传说啊神马的就不啰嗦了，我们直接切入正题：
问题描述：

　　有一个梵塔，塔内有三个座A、B、C，A座上有诺干个盘子，盘子大小不等，大的在下，小的在上（如图）。

把这些个盘子从A座移到C座，中间可以借用B座但每次只能允许移动一个盘子，并且在移动过程中，3个座上的盘

子始终保持大盘在下，小盘在上。

描述简化：把A柱上的n个盘子移动到C柱，其中可以借用B柱。

　　我们直接假设有n个盘子：

　　先把盘子从小到大标记为1、2、3......n

　　先看原问题三个柱子的状态：
状态0　　A：按顺序堆放的n个盘子。B:空的。C：空的。

　　目标是要把A上的n个盘子移动到C。因为必须大的在下小的在上，所以最终结果C盘上最下面的应该是标号为n的盘子，试想：

要取得A上的第n个盘子，就要把它上面的n-1个盘子拿开吧？拿开放在哪里呢？共有三个柱子：A显然不是、如果放在C上

了，那么最大的盘子就没地方放，问题还是没得到解决。所以选择B柱。当然，B上面也是按照大在下小在上的原则堆放的

（记住：先不要管具体如何移动，可以看成用一个函数完成移动，现在不用去考虑函数如何实现。这点很重要）。

　　很明显：上一步完成后三个塔的状态：

状态1： 　　A：只有最大的一个盘子。B：有按规则堆放的n-1个盘子。C空的。

　　上面的很好理解吧，好，其实到这里就已经完成一半了。（如果前面的没懂，请重看一遍。point：不要管如何移动！）

我们继续：

　　这时候，可以直接把A上的最大盘移动到C盘，移动后的状态：

中间状态：　　A：空的。B：n-1个盘子。C：有一个最大盘（第n个盘子）

　　要注意的一点是：这时候的C柱其实可以看做是空的。因为剩下的所有盘子都比它小，它们中的任何一个都可以放在上面，也就是　　　　　　    　　　　　　　　  C柱上。

　　所以现在三个柱子的状态：

中间状态：　　A：空的。B：n-1个盘子。C：空的

　　想一想，现在的问题和原问题有些相似之处了吧？。。如何更相似呢？。显然，只要吧B上的n-1个盘子移动到A，待解决的问题和原问题就相比就只是规模变小了

　　现在考虑如何把B上的n-1个盘子移动到A上，其实移动方法和上文中的把n-1个盘从A移动到B是一样的，只是柱子的名称换了下而已。。（如果写成函数，只是参数调用顺序改变而已）。　

　　假设你已经完成上一步了（同样的，不要考虑如何去移动，只要想着用一个函数实现就好），请看现在的状态：

状态2：　A：有按顺序堆放的n-1个盘子。B：空的。C：按顺序堆放的第n盘子(可看为空柱)

就在刚才，我们完美的完成了一次递归。如果没看懂请从新看一遍，可以用笔画出三个状态、静下心来慢慢推理。

我一再强调的：当要把最大盘子上面的所有盘子移动到另一个空柱上时，不要关心具体如何移动，只用把它看做一个函数可以完成即可，不用关心函数的具体实现。如果你的思路纠结在这里，就很难继续深入了。

到这里，其实 基本思路已经理清了。状态2和状态0，除了规模变小 ，其它方面没有任何区别了。然后只要用相同的思维方式，就能往下深入。。。

好了，看看如何用算法实现吧：

定义函数Hanoi（a,b,c,n）表示把a上的n个盘子移动到c上，其中可以用到b。

定义函数move(m,n)表示把m上的盘子移动到n上

我们需要解决的问题正是　　Hanoi (a,b,c,n)   　　//上文中的状态0

1、把A上的n-1个移动到B：  　　Hanoi (a,c,b,n-1);       // 操作结束为状态1

2、把A上的大盘子移动到C         move(a,c)　　　　

3、把B上的n-1移动到A　　　　　Hanoi (b,c,a,n-1);　　//操作结束位状态2(和状态1相比只是规模变小)

如果现在还不能理解、请回过头再看一遍、毕竟如果是初学者不是很容易就能理解的。可以用笔记下几个关键的状态，并且看看你有没有真正的投入去看，独立去思考了。

以上、如果有不对的地方、还希望您能指出。

我对递归的一点理解：

解决实际问题时、不能太去关心实现的细节（因为递归的过程恰恰是我们实现的方法）就像这个问题，如在第一步就过多的纠结于如何把n-1个盘子移动到B上、那么你的思路就很难继续深入。只要看做是用函数实现就好，如果你能看出不管怎么移动，其实本质都一样的时候，那么就能较快的得到结果了。就像这个案例，要注意到我们做的关键几步都只是移动的顺序有改变，其中的规则没有改变，如

如果用函数表示的话，就只是参数调用的顺序有所不同了。在递归的运用中、不用关心每一步的具体实现 ，只要看做用一个函数表示就好。分析问题的时候，最好画出自己的推理过程，得到有效的状态图。

思考问题讲求思路的连贯性、力求尽快进入状态，享受完全投入到一件事中的美妙感觉

http://renaud.waldura.com/doc/java/dijkstra/ 

Dijkstra's algorithm is probably the best-known and thus most implemented shortest path algorithm. It is simple, easy to understand and implement, yet impressively efficient. By getting familiar with such a sharp tool, a developer can solve efficiently and elegantly problems that would be considered impossibly hard otherwise. Be my guest as I explore a possible implementation of Dijkstra's shortest path algorithm in Java.

What It Does

Dijkstra's algorithm, when applied to a graph, quickly finds the shortest path from a chosen source to a given destination. (The question "how quickly" is answered later in this article.) In fact, the algorithm is so powerful that it finds all shortest paths from the source to all destinations! This is known as the single-source shortest paths problem. In the process of finding all shortest paths to all destinations, Dijkstra's algorithm will also compute, as a side-effect if you will, a spanning tree for the graph. While an interesting result in itself, the spanning tree for a graph can be found using lighter (more efficient) methods than Dijkstra's.

How It Works

First let's start by defining the entities we use. The graph is made of vertices (or nodes, I'll use both words interchangeably), and edges which link vertices together. Edges are directed and have an associated distance, sometimes called the weight or the cost. The distance between the vertex u and the vertex v is noted [u, v] and is always positive.

Dijkstra's algorithm partitions vertices in two distinct sets, the set of unsettled vertices and the set of settled vertices. Initially all vertices are unsettled, and the algorithm ends once all vertices are in the settled set. A vertex is considered settled, and moved from the unsettled set to the settled set, once its shortest distance from the source has been found.

We all know that algorithm + data structures = programs, in the famous words of Niklaus Wirth. The following data structures are used for this algorithm:

d	stores the best estimate of the shortest distance from the source to each vertex
π	stores the predecessor of each vertex on the shortest path from the source
S	the set of settled vertices, the vertices whose shortest distances from the source have been found
Q	the set of unsettled vertices

With those definitions in place, a high-level description of the algorithm is deceptively simple. With s as the source vertex:

Dead simple isn't it? The two procedures called from the main loop are defined below:

An Example

So far I've listed the instructions that make up the algorithm. But to really understand it, let's follow the algorithm on an example. We shall run Dikjstra's shortest path algorithm on the following graph, starting at the source vertex a.

We start off by adding our source vertex a to the set Q. Q isn't empty, we extract its minimum, a again. We add a to S, then relax its neighbors. (I recommend you follow the algorithm in parallel with this explanation.)

Those neighbors, vertices adjacent to a, are b and c (in green above). We first compute the best distance estimate from a to b. d(b) was initialized to infinity, therefore we do:

d(b) = d(a) + [a,b] = 0 + 4 = 4

The second time around, Q contains b and c. As seen above, c is the vertex with the current shortest distance of 2. It is extracted from the queue and added to S, the set of settled nodes. We then relax the neighbors of c, which are b, d and a.

a is ignored because it is found in the settled set. But it gets interesting: the first pass of the algorithm had concluded that the shortest path from a to b was direct. Looking at c's neighbor b, we realize that:

d(b) = 4 > d(c) + [c,b] = 2 + 1 = 3

The unsettled vertex with the shortest distance is extracted from the queue, it is now b. We add it to the settled set and relax its neighbors c and d.

We skip c, it has already been settled. But a shorter path is found for d:

d(d) = 7 > d(b) + [b,d] = 3 + 1 = 4

At this point the only vertex left in the unsettled set is d, and all its neighbors are settled. The algorithm ends. The final results are displayed in red below:

• π - the shortest path, in predecessor fashion
• d - the shortest distance from the source for each vertex

This completes our description of Dijkstra's shortest path algorithm. Other shortest path algorithms exist (see the References section at the end of this article), but Dijkstra's is one of the simplest, while still offering good performance in most cases.

Implementing It in Java

The Java implementation is quite close to the high-level description we just walked through. For the purpose of this article, my Java implementation of Dijkstra's shortest path finds shortest routes between cities on a map. The RoutesMap object defines a weighted, oriented graph as defined in the introduction section of this article.

Data Structures

We've listed above the data structures used by the algorithm, let's now decide how we are going to implement them in Java.

S, the settled nodes set

This one is quite straightforward. The Java Collections feature the Set interface, and more precisely, the HashSet implementation which offers constant-time performance on the contains operation, the only one we need. This defines our first data structure.

Notice how my data structure is declared as an abstract type (Set) instead of a concrete type (HashSet). Doing so is a good software engineering practice, as it allows to change the actual type of the collection without any modification to the code that uses it.

d, the shortest distances list

As we've seen, one output of Dijkstra's algorithm is a list of shortest distances from the source node to all the other nodes in the graph. A straightforward way to implement this in Java is with a Map, used to keep the shortest distance value for every node. We also define two accessors for readability, and to encapsulate the default infinite distance.

You may notice I declare this field final. This is a Java idiom used to flag aggregation relationships between objects. By marking a field final, I am able to convey that it is part of a aggregation relationship, enforced by the properties of final—the encapsulating class cannot exist without this field.

π, the predecessors tree

Another output of the algorithm is the predecessors tree, a tree spanning the graph which yields the actual shortest paths. Because this is the predecessors tree, the shortest paths are actually stored in reverse order, from destination to source. Reversing a given path is easy with Collections.reverse().

The predecessors tree stores a relationship between two nodes, namely a given node's predecessor in the spanning tree. Since this relationship is one-to-one, it is akin to amapping between nodes. Therefore it can be implemented with, again, a Map. We also define a pair of accessors for readability.

Again I declare my data structure to be of the abstract type Map, instead of the concrete type HashMap. And tag it final as well.

Q, the unsettled nodes set

As seen in the previous section, a data structure central to Dijkstra's algorithm is the set of unsettled vertices Q. In Java programming terms, we need a structure able to store the nodes of our example graph, i.e. City objects. That structure is then looked up for the city with the current shortest distance given by d().

We could do this by using another Set of cities, and sort it according to d() to find the city with shortest distance every time we perform this operation. This isn't complicated, and we could leverage Collections.min() using a custom Comparator to compare the elements according to d().

But because we do this at every iteration, a smarter way would be to keep the set ordered at all times. That way all we need to do to get the city with the lowest distance is to get the first element in the set. New elements would need to be inserted in the right place, so that the set is always kept ordered.

A quick search through the Java collections API yields the PriorityQueue: it can sort elements according to a custom comparator, and provides constant-time access to the smallest element. This is precisely what we need, and we'll write a comparator to order cities (the set elements) according to the current shortest distance. Such a comparator is included below, along with the PriorityQueue definition. Also listed is the small method that extracts the node with the shortest distance.

One important note about the comparator: it is used by the PriorityQueue to determine both object ordering and identity. If the comparator returns that two elements are equal, the queue infers they are the same, and it stores only one instance of the element. To prevent losing nodes with equal shortest distances, we must compare the elements themselves (third block in the if statement above).

Having powerful, flexible data structures at our disposal is what makes Java such an enjoyable language (that, and garbage collection of course).

Putting It All Together

We have defined our data structures, we understand the algorithm, all that remains to do is implement it. As I mentioned earlier, my implementation is close to the high-level description given above. Note that when the only shortest path between two specific nodes is asked, the algorithm can be interrupted as soon as the destination node is reached.

The DijkstraEngine class implements this algorithm and brings it all together. See "Implementation Notes" below to download the source code.

A Word About Performance

The complexity of Dijkstra's algorithm depends heavily on the complexity of the priority queue Q. If this queue is implemented naively as I first introduced it (i.e. it is re-ordered at every iteration to find the mininum node), the algorithm performs in O(n²), where n is the number of nodes in the graph.

With a real priority queue kept ordered at all times, as we implemented it, the complexity averages O(n log m). The logarithm function stems from the collectionsPriorityQueue class, a heap implementation which performs in log(m).

Implementation Notes

The Java source code discussed in this article is available for download as a ZIP file. Extensive unit tests are provided and validate the correctness of the implementation. Some minimal Javadoc is also provided. As the code makes use of the assert facility and generics, it must be compiled with "javac -source 1.5"; the tests require junit.jar.I warmly recommend Eclipse for all Java development.

I've received a fair amount of e-mail about this article, which has become quite popular. I'm unfortunately unable to answer all your questions, and for this I apologize. Keep in mind this article (and the code) is meant as a starting point: the implementation discussed here is hopefully simple, correct, and relatively easy to understand, but is probably not suited to your specific problem. You must tailor it to your own domain.

My goal in writing this article was to share and teach a useful tool, striving for 1- simplicity and 2- correctness. I purposefully shied away from turning this exercise into a full-blown generic Java implementation. Readers after full-featured, industrial-strength Java implementations of Dijkstra's shortest path algorithm should look at the "Resources" section below.

APIs.

• Generics. An essential characteristic of collection ADTs is that we should be able to use them for any type of data. A specific Java mechanism known as generics enables this capability. The notation <Item> after the class name in each of our APIs defines the name Item as a type parameter, a symbolic placeholder for some concrete type to be used by the client. You can read Stack<Item> as "stack of items." For example, you can write code such as

  Stack<String> stack = new Stack<String>(); stack.push("Test"); ... String next = stack.pop();

  to use a stack for String objects.

• Autoboxing. Type parameters have to be instantiated as reference types, so Java automatically converts between a primitive type and its corresponding wrapper type in assignments, method arguments, and arithmetic/logic expressions. This conversion enables us to use generics with primitive types, as in the following code:

  Stack<Integer> stack = new Stack<Integer>(); stack.push(17); // auto-boxing (int -> Integer) int i = stack.pop(); // auto-unboxing (Integer -> int)

  Automatically casting a primitive type to a wrapper type is known as autoboxing, and automatically casting a wrapper type to a primitive type is known as auto-unboxing.

• Iterable collections. For many applications, the client's requirement is just to process each of the items in some way, or to iterate through the items in the collection. Java's foreach statement supports this paradigm. For example, suppose that collection is a Queue<Transaction>. Then, if the collection is iterable, the client can print a transaction list with a single statement:

  for (Transaction t : collection) StdOut.println(t);

• Bags. A bag is a collection where removing items is not supported—its purpose is to provide clients with the ability to collect items and then to iterate through the collected items. Stats.java is a bag client that reads a sequence of real numbers from standard input and prints out their mean and standard deviation.

• FIFO queues. A FIFO queue is a collection that is based on the first-in-first-out (FIFO) policy. The policy of doing tasks in the same order that they arrive server is one that we encounter frequently in everyday life: from people waiting in line at a theater, to cars waiting in line at a toll booth, to tasks waiting to be serviced by an application on your computer.

• Pushdown stack. A pushdown stack is a collection that is based on the last-in-first-out (LIFO) policy. When you click a hyperlink, your browser displays the new page (and pushes onto a stack). You can keep clicking on hyperlinks to visit new pages, but you can always revisit the previous page by clicking the back button (popping it from the stack). Reverse.java is a stack client that reads a sequence of integers from standard input and prints them in reverse order.

• Arithmetic expression evaluation. Evaluate.java is a stack client that evaluates fully parenthesized arithmetic expressions. It uses Dijkstra's 2-stack algorithm:
  • Push operands onto the operand stack.
  • Push operators onto the operator stack.
  • Ignore left parentheses.
  • On encountering a right parenthesis, pop an operator, pop the requisite number of operands, and push onto the operand stack the result of applying that operator to those operands.

  This code is a simple example of an interpreter.

Array and resizing array implementations of collections.

• Fixed-capacity stack of strings. FixedCapacityStackOfString.java implements a fixed-capacity stack of strings using an array.

• Fixed-capacity generic stack. FixedCapacityStack.java implements a generic fixed-capacity stack.

• Array resizing stack. ResizingArrayStack.java implements a generic stack using a resizing array. With a resizing array, we dynamically adjust the size of the array so that it is both sufficiently large to hold all of the items and not so large as to waste an excessive amount of space. Wedouble the size of the array in push() if it is full; we halve the size of the array in pop() if it is less than one-quarter full.

• Array resizing queue. ResizingArrayQueue.java implements the queue API with a resizing array.

Linked lists.

private class Node { Item item; Node next; }

• Building a linked list. To build a linked list that contains the items to, be, and or, we create a Node for each item, set the item field in each of th

• Insert at the beginning. The easiest place to insert a new node in a linked list is at the beginning.

• Remove from the beginning. Removing the first node in a linked list is also easy.
  

• Insert at the end. To insert a node at the end of a linked list, we maintain a link to the last node in the list.

• Traversal. The following is the idiom for traversing the nodes in a linked list.

  for (Node x = first; x != null; x = x.next) { // process x.item }

Linked-list implementations of collections.

• Linked list implementation of a stack. Stack.java implements a generic stack using a linked list. It maintains the stack as a linked list, with the top of the stack at the beginning, referenced by an instance variable first. To push() an item, we add it to the beginning of the list; to pop()an item, we remove it from the beginning of the list.

• Linked list implementation of a queue. Program Queue.java implements a generic FIFO queue using a linked list. It maintains the queue as a linked list in order from least recently to most recently added items, with the beginning of the queue referenced by an instance variable firstand the end of the queue referenced by an instance variable last. To enqueue() an item, we add it to the end of the list; to dequeue() an item, we remove it from the beginning of the list.

• Linked list implementation of a bag. Program Bag.java implements a generic bag using a linked list. The implementation is the same as Stack.javaexcept for changing the name of push() to add() and removing pop().

Iteration.

Stack<String> collection = new Stack<String>(); ... for (String s : collection) StdOut.println(s);    ...

Iterator<String> i = collection.iterator(); while (i.hasNext()) { String s = i.next(); StdOut.println(s); }

• Include the following import statement so that our code can refer to Java's java.util.Iterator interface:

  import java.util.Iterator;

• Add the following to the class declaration, a promise to provide an iterator() method, as specified in the java.lang.Iterable interface:

  implements Iterable<Item>

• Implement a method iterator() that returns an object from a class that implements the Iterator interface:

  public Iterator<Item> iterator() { return new ListIterator(); }

• Implement a nested class that implements the Iterator interface by including the methods hasNext(), next(), and remove(). We always use an empty method for the optional remove() method because interleaving iteration with operations that modify the data structure is best avoided.

  • The nested class ListIterator in Bag.java illustrates how to implement a class that implements the Iterator interface when the underlying data structure is a linked list.

  • The nested class ArrayIterator in ArrayResizingBag.java does the same when the underlying data structure is an array.

Autoboxing Q + A

Q. How does auto-boxing handle the following code fragment?

Integer a = null; int b = a;

A. It results in a run-time error. Primitive type can store every value of their corresponding wrapper type except null.

Q. Why does the first group of statements print true, but the second false?

Integer a1 = 100; Integer a2 = 100; System.out.println(a1 == a2); // true Integer b1 = new Integer(100); Integer b2 = new Integer(100); System.out.println(b1 == b2); // false Integer c1 = 150; Integer c2 = 150; System.out.println(c1 == c2); // false

A. The second prints false because b1 and b2 are references to different Integer objects. The first and third code fragments rely on autoboxing. Surprisingly the first prints true because values between -128 and 127 appear to refer to the same immutable Integer objects (Java's implementation of valueOf() retrieves a cached values if the integer is between -128 and 127), while Java constructs new objects for each integer outside this range.

Here is another Autoboxing.java anomaly.

Generics Q + A

Q. Are generics solely for auto-casting?

A. No, but we will use them only for "concrete parameterized types", where each data type is parameterized by a single type. The primary benefit is to discover type mismatch errors at compile-time instead of run-time. There are other more general (and more complicated) uses of generics, including wildcards. This generality is useful for handling subtypes and inheritance. For more information, see this Generics FAQ and this generics tutorial.

Q. Can concrete parameterized types be used in the same way as normal types?

A. Yes, with a few exceptions (array creation, exception handling, with instanceof, and in a class literal).

Q. Why do I get a "can't create an array of generics" error when I try to create an array of generics?

public class ResizingArrayStack<Item> { Item[] a = new Item[1]; }

A. Unfortunately, creating arrays of generics is not possible in Java 1.5. The underlying cause is that arrays in Java are covariant, but generics are not. In other words, String[] is a subtype of Object[], but Stack<String> is not a subtype of Stack<Object>. To get around this defect, you need to perform an unchecked cast as in ResizingArrayStack.java.

Q. So, why are arrays covariant?

A. Many programmers (and programming language theorists) consider covariant arrays to be a serious defect in Java's type system: they incur unnecessary run-time performance overhead (for example, see ArrayStoreException) and can lead to subtle bugs. Covariant arrays were introduced in Java to circumvent the problem that Java didn't originally include generics in its design, e.g., to implement Arrays.sort(Comparable[]) and have it be callable with an input array of type String[].

Q. Can I create and return a new array of a parameterized type, e.g., to implement a toArray() method for a generic queue?

A. Not easily. You can do it using reflection provided that the client passes an object of the desired concrete type to toArray() This is the (awkward) approach taken by Java's Collection Framework.