配置文件，application.yaml

java配置，HttpSessionConfiguration.java：

MongoDB Profiler is a db profiling system that can help identify inefficient

or slow queries and operations.

Levels of profiles available are:

We can enable it by setting the Profile level value using the following
command in mongo shell :

"db.setProfilingLevel(1)"

By default, mongod records slow queries to its log, as defined by slowOpThresholdMs.

NOTE

Enabling database profiler puts negative impact on MongoDB’s performance.

It’s better to enable it for specific intervals & minimal on Production Servers.

We can enable profiling on a mongod basis but This setting will not propagate
across a replica set and sharded cluster.

We can view the output in the system.profile collection in mongo shell using show profile command, or using following:

db.system.profile.find( { millis : { $gt : 200 } } )

Command returns operations that took longer than 200 ms. Similarly we
can change the values as per our need.

Enabling profile for an entire mongod instance.

For the purpose of development in testing, we can enable database profiling/settings for an 
entire mongod instance. The profiling level will be applied to all databases.

NOTE:

We can't enable the profiling settings on a mongos instance. To enable the profiling in

shard clusters, we have to enable/start profiling for each mongod instance in cluster.

Query for the recent 10 entries

db.system.profile.find().limit(10).sort( { ts : 1 } ).pretty()

Collection with the slowest queries(No. Of queries)

db.system.profile.group({key: {ns: true}, initial: {count: 0}, reduce: function(obj,prev){ prev.count++;}})

Collection with the slowest queries(No. Of millis spent)

db.system.profile.group({key: {ns: true}, initial: {millis: 0}, reduce: function(obj, prev){ prev.millis += obj.millis;}})

Most recent slow query

db.system.profile.find().sort({$natural: -1}).limit(1)

Single slowest query(Right now)

db.system.profile.find().sort({millis: -1}).limit(1)

• 一般主从：一主多从，主作读写数据，从作从主备份数据用，如果主宕机，则整个MONGODB无法工作。

• 复制式主从：一动态主多从，主由选举产生，当中一个主宕机，其他的从会选出一个主。

Expire Data from Collections by Setting TTL¶

This document provides an introduction to MongoDB’s “time to live” or TTL collection feature. TTL collections make it possible to store data in MongoDB and have the mongod automatically remove data after a specified number of seconds or at a specific clock time.

Data expiration is useful for some classes of information, including machine generated event data, logs, and session information that only need to persist for a limited period of time.

A special TTL index property supports the implementation of TTL collections. The TTL feature relies on a background thread in mongod that reads the date-typed values in the index and removes expired documentsfrom the collection.

db.log_events.createIndex( { "createdAt": 1 }, { expireAfterSeconds: 3600 } ) 

db.log_events.insert( {    "createdAt": new Date(),    "logEvent": 2,    "logMessage": "Success!" } ) 

db.log_events.createIndex( { "expireAt": 1 }, { expireAfterSeconds: 0 } ) 

db.log_events.insert( {    "expireAt": new Date('July 22, 2013 14:00:00'),    "logEvent": 2,    "logMessage": "Success!" } ) 

One approach would be use the remove command with a condition on the created_at field (the collection already have an index on this field):

db.my_collection.remove({created_at: {$lte: new Date("11/01/2012")}});

But this approach will be very slow, instead of that, a better way to do it is rename the current collection (for instance to “old_collection”) using renameCollection. Then performing a query-and-insert from the “old_collection” into “my_collection”:

db.my_collection.renameCollection("old_collection");  
db.createCollection("my_collection");  
db.my_collection.createIndex(...); // recreate the indexes for the collection  
// copy docs from old collection into the new collection 
db.old_collection.find(     
{created_at: {$gte: new Date("11/01/2012")}}   ).sort({_id: -1}).forEach(     
function(row) { db.my_collection.insert(row); }   );  // drop old collection db.old_collection.drop();  

This approach is typically faster than running a bunch of removes on your data

创建固定集合

我们通过createCollection来创建一个固定集合，且capped选项设置为true：

>db.createCollection("cappedLogCollection",{capped:true,size:10000})

还可以指定文档个数,加上max:1000属性：

>db.createCollection("cappedLogCollection",{capped:true,size:10000,max:1000})

判断集合是否为固定集合:

>db.cappedLogCollection.isCapped()

如果需要将已存在的集合转换为固定集合可以使用以下命令：

>db.runCommand({"convertToCapped":"posts",size:10000})

以上代码将我们已存在的 posts 集合转换为固定集合。

固定集合查询

固定集合文档按照插入顺序储存的,默认情况下查询就是按照插入顺序返回的,也可以使用$natural调整返回顺序。

>db.cappedLogCollection.find().sort({$natural:-1})

固定集合的功能特点

可以插入及更新,但更新不能超出collection的大小,否则更新失败,不允许删除,但是可以调用drop()删除集合中的所有行,但是drop后需要显式地重建集合。

在32位机子上一个cappped collection的最大值约为482.5M,64位上只受系统文件大小的限制。

固定集合属性及用法

属性

• 属性1:对固定集合进行插入速度极快
• 属性2:按照插入顺序的查询输出速度极快
• 属性3:能够在插入最新数据时,淘汰最早的数据

用法

• 用法1:储存日志信息
• 用法2:缓存一些少量的文档

aggregate() 方法

MongoDB中聚合的方法使用aggregate()。

语法

aggregate() 方法的基本语法格式如下所示：

>db.COLLECTION_NAME.aggregate(AGGREGATE_OPERATION)

实例

集合中的数据如下：

{    _id: ObjectId(7df78ad8902c)    title: 'MongoDB Overview',     description: 'MongoDB is no sql database',    by_user: 'w3cschool.cc',    url: 'http://www.w3cschool.cc',    tags: ['mongodb', 'database', 'NoSQL'],    likes: 100 }, {    _id: ObjectId(7df78ad8902d)    title: 'NoSQL Overview',     description: 'No sql database is very fast',    by_user: 'w3cschool.cc',    url: 'http://www.w3cschool.cc',    tags: ['mongodb', 'database', 'NoSQL'],    likes: 10 }, {    _id: ObjectId(7df78ad8902e)    title: 'Neo4j Overview',     description: 'Neo4j is no sql database',    by_user: 'Neo4j',    url: 'http://www.neo4j.com',    tags: ['neo4j', 'database', 'NoSQL'],    likes: 750 },

现在我们通过以上集合计算每个作者所写的文章数，使用aggregate()计算结果如下：

> db.mycol.aggregate([{$group : {_id : "$by_user", num_tutorial : {$sum : 1}}}]) { "result" : [ { "_id" : "w3cschool.cc", "num_tutorial" : 2 }, { "_id" : "Neo4j", "num_tutorial" : 1 } ], "ok" : 1 } >

以上实例类似sql语句： select by_user, count(*) from mycol group by by_user

在上面的例子中，我们通过字段by_user字段对数据进行分组，并计算by_user字段相同值的总和。

下表展示了一些聚合的表达式:

管道的概念

管道在Unix和Linux中一般用于将当前命令的输出结果作为下一个命令的参数。

MongoDB的聚合管道将MongoDB文档在一个管道处理完毕后将结果传递给下一个管道处理。管道操作是可以重复的。

表达式：处理输入文档并输出。表达式是无状态的，只能用于计算当前聚合管道的文档，不能处理其它的文档。

这里我们介绍一下聚合框架中常用的几个操作：

• $project：修改输入文档的结构。可以用来重命名、增加或删除域，也可以用于创建计算结果以及嵌套文档。
• $match：用于过滤数据，只输出符合条件的文档。$match使用MongoDB的标准查询操作。
• $limit：用来限制MongoDB聚合管道返回的文档数。
• $skip：在聚合管道中跳过指定数量的文档，并返回余下的文档。
• $unwind：将文档中的某一个数组类型字段拆分成多条，每条包含数组中的一个值。
• $group：将集合中的文档分组，可用于统计结果。
• $sort：将输入文档排序后输出。
• $geoNear：输出接近某一地理位置的有序文档。

管道操作符实例

1、$project实例

db.article.aggregate( { $project : {         title : 1 ,         author : 1 , }} );

这样的话结果中就只还有_id,tilte和author三个字段了，默认情况下_id字段是被包含的，如果要想不包含_id话可以这样:

db.article.aggregate( { $project : {         _id : 0 ,         title : 1 ,         author : 1 }});

2.$match实例

db.articles.aggregate( [ { $match : { score : { $gt : 70, $lte : 90 } } }, { $group: { _id: null, count: { $sum: 1 } } } ] );

$match用于获取分数大于70小于或等于90记录，然后将符合条件的记录送到下一阶段$group管道操作符进行处理。

3.$skip实例

db.article.aggregate( { $skip : 5 }); 

经过$skip管道操作符处理后，前五个文档被"过滤"掉。

• autoConnectRetry simply means the driver will automatically attempt to reconnect to the server(s) after unexpected disconnects. In production environments you usually want this set to true.

• connectionsPerHost are the amount of physical connections a single Mongo instance (it's singleton so you usually have one per application) can establish to a mongod/mongos process. At time of writing the java driver will establish this amount of connections eventually even if the actual query throughput is low (in order words you will see the "conn" statistic in mongostat rise until it hits this number per app server).

  There is no need to set this higher than 100 in most cases but this setting is one of those "test it and see" things. Do note that you will have to make sure you set this low enough so that the total amount of connections to your server do not exceed

  db.serverStatus().connections.available

  In production we currently have this at 40.

• connectTimeout. As the name suggest number of milliseconds the driver will wait before a connection attempt is aborted. Set timeout to something long (15-30 seconds) unless there's a realistic, expected chance this will be in the way of otherwise succesful connection attempts. Normally if a connection attempt takes longer than a couple of seconds your network infrastructure isn't capable of high throughput.

• maxWaitTime. Number of ms a thread will wait for a connection to become available on the connection pool, and raises an exception if this does not happen in time. Keep default.

• socketTimeout. Standard socket timeout value. Set to 60 seconds (60000).

• threadsAllowedToBlockForConnectionMultiplier. Multiplier for connectionsPerHost that denotes the number of threads that are allowed to wait for connections to become available if the pool is currently exhausted. This is the setting that will cause the "com.mongodb.DBPortPool$SemaphoresOut: Out of semaphores to get db connection" exception. It will throw this exception once this thread queue exceeds the threadsAllowedToBlockForConnectionMultiplier value. For example, if the connectionsPerHost is 10 and this value is 5 up to 50 threads can block before the aforementioned exception is thrown.

  If you expect big peaks in throughput that could cause large queues temporarily increase this value. We have it at 1500 at the moment for exactly that reason. If your query load consistently outpaces the server you should just improve your hardware/scaling situation accordingly.

• readPreference. (UPDATED, 2.8+) Used to determine the default read preference and replaces "slaveOk". Set up a ReadPreference through one of the class factory method. A full description of the most common settings can be found at the end of this post

• w. (UPDATED, 2.6+) This value determines the "safety" of the write. When this value is -1 the write will not report any errors regardless of network or database errors. WriteConcern.NONE is the appropriate predefined WriteConcern for this. If w is 0 then network errors will make the write fail but mongo errors will not. This is typically referred to as "fire and forget" writes and should be used when performance is more important than consistency and durability. Use WriteConcern.NORMAL for this mode.

  If you set w to 1 or higher the write is considered safe. Safe writes perform the write and follow it up by a request to the server to make sure the write succeeded or retrieve an error value if it did not (in other words, it sends a getLastError() command after you write). Note that until this getLastError() command is completed the connection is reserved. As a result of that and the additional command the throughput will be signficantly lower than writes with w <= 0. With a w value of exactly 1 MongoDB guarantees the write succeeded (or verifiably failed) on the instance you sent the write to.

  In the case of replica sets you can use higher values for w whcih tell MongoDB to send the write to at least "w" members of the replica set before returning (or more accurately, wait for the replication of your write to "w" members). You can also set w to the string "majority" which tells MongoDB to perform the write to the majority of replica set members (WriteConcern.MAJORITY). Typicall you should set this to 1 unless you need raw performance (-1 or 0) or replicated writes (>1). Values higher than 1 have a considerable impact on write throughput.

• fsync. Durability option that forces mongo to flush to disk after each write when enabled. I've never had any durability issues related to a write backlog so we have this on false (the default) in production.

• j *(NEW 2.7+)*. Boolean that when set to true forces MongoDB to wait for a successful journaling group commit before returning. If you have journaling enabled you can enable this for additional durability. Refer to http://www.mongodb.org/display/DOCS/Journaling to see what journaling gets you (and thus why you might want to enable this flag).

ReadPreference The ReadPreference class allows you to configure to what mongod instances queries are routed if you are working with replica sets. The following options are available :

• ReadPreference.primary() : All reads go to the repset primary member only. Use this if you require all queries to return consistent (the most recently written) data. This is the default.

• ReadPreference.primaryPreferred() : All reads go to the repset primary member if possible but may query secondary members if the primary node is not available. As such if the primary becomes unavailable reads become eventually consistent, but only if the primary is unavailable.

• ReadPreference.secondary() : All reads go to secondary repset members and the primary member is used for writes only. Use this only if you can live with eventually consistent reads. Additional repset members can be used to scale up read performance although there are limits to the amount of (voting) members a repset can have.

• ReadPreference.secondaryPreferred() : All reads go to secondary repset members if any of them are available. The primary member is used exclusively for writes unless all secondary members become unavailable. Other than the fallback to the primary member for reads this is the same as ReadPreference.secondary().

• ReadPreference.nearest() : Reads go to the nearest repset member available to the database client. Use only if eventually consistent reads are acceptable. The nearest member is the member with the lowest latency between the client and the various repset members. Since busy members will eventually have higher latencies this should also automatically balance read load although in my experience secondary(Preferred) seems to do so better if member latencies are relatively consistent.

Note : All of the above have tag enabled versions of the same method which return TaggableReadPreference instances instead. A full description of replica set tags can be found here :Replica Set Tags