paulwong

#

Deploy artifacts into Maven Repository in Jenkins

https://huongdanjava.com/deploy-artifacts-into-maven-repository-in-jenkins.html

posted @ 2020-04-06 14:13 paulwong 阅读(325) | 评论 (0)编辑 收藏

MAVEN私服-Nexus Repository Manager


Nexus Repository Manager is a tool that allows us to store and use libraries we need in projects such as Maven project…

Nexus Repository ManagerIn this tutorial, I summarize the tutorials of Huong Dan Java on the Nexus Repository Manager for your reference.


Installation

In this tutorial, I will guide you how to install Nexus Repository Manager.


Configuration

In order to create a new Maven Repository in the Nexus Repository Manager, you can refer to this tutorial.

We need to define a Role to define User rights in the Nexus Repository Manager.

To be able to do anything in the Nexus Repository Manager, you need to create and use the User.


Manipulation

Nexus Repository Manager supports us UI to upload any artifact to the Repository.

In addition to the UI, we can also use the RESTful API to upload an artifact.

posted @ 2020-04-06 14:08 paulwong 阅读(299) | 评论 (0)编辑 收藏

List sessions / active connections on MariaDB server

Using a command

Option 1

show status where variable_name = 'threads_connected'; 

Columns

  • Variable_name - Name of the variable shown
  • Value - Number of active connections

Rows

  • One row: Only one row is displayed

Sample results

Option 2

show processlist; 

Columns

  • Id - The connection identifier
  • User - The MariaDB user who issued the statement
  • Host - Host name and client port of the client issuing the statement
  • db - The default database (schema), if one is selected, otherwise NULL
  • Command - The type of command the thread is executing
  • Time - The time in seconds that the thread has been in its current state
  • State - An action, event, or state that indicates what the thread is doing
  • Info - The statement the thread is executing, or NULL if it is not executing any statement
  • Progress - The total progress of the process (0-100%)

Rows

  • One row: represents one active connection
  • Scope of rows: total of active connections

Sample results

Using a query

Option 3

select id, user, host, db, command, time, state, 
info, progress from information_schema.processlist;

Columns

  • Id - The connection identifier
  • User - The MariaDB user who issued the statement
  • Host - Host name and client port of the client issuing the statement
  • db - The default database (schema), if one is selected, otherwise NULL
  • Command - The type of command the thread is executing
  • Time - The time in seconds that the thread has been in its current state
  • State - An action, event, or state that indicates what the thread is doing
  • Info - The statement the thread is executing, or NULL if it is not executing any statement
  • Progress - The total progress of the process (0-100%)
  • memory_used - Amount of memory used by the active connection

Rows

  • One row: represents one active connection
  • Scope of rows: total of active connections

Sample results

Using the GUI

Option 4

Click on the Client Connections option of the Management tab (left navigation pane)

This action will show the Client Connections screen containing the current active connections

posted @ 2020-04-02 15:38 paulwong 阅读(287) | 评论 (0)编辑 收藏

Finding slow queries in MongoDB

Database Profiling

MongoDB Profiler is a db profiling system that can help identify inefficient

or slow queries and operations.

Levels of profiles available are:

Level

Setting

0

Off. & No profiling

1

On & only includes slow operations

2

On & Includes all operations


We can enable it by setting the Profile level value using the following
command in mongo shell :

"db.setProfilingLevel(1)"

By default, mongod records slow queries to its log, as defined by slowOpThresholdMs.

NOTE

Enabling database profiler puts negative impact on MongoDB’s performance.

It’s better to enable it for specific intervals & minimal on Production Servers.

We can enable profiling on a mongod basis but This setting will not propagate
across a replica set and sharded cluster.

We can view the output in the system.profile collection in mongo shell using show profile command, or using following:

db.system.profile.find( { millis : { $gt : 200 } } )

Command returns operations that took longer than 200 ms. Similarly we
can change the values as per our need.

Enabling profile for an entire mongod instance.

For the purpose of development in testing, we can enable database profiling/settings for an 
entire mongod instance. The profiling level will be applied to all databases.

 

NOTE:

We can't enable the profiling settings on a mongos instance. To enable the profiling in

shard clusters, we have to enable/start profiling for each mongod instance in cluster.

 

Query for the recent 10 entries

db.system.profile.find().limit(10).sort( { ts : 1 } ).pretty()

 

Collection with the slowest queries(No. Of queries)

db.system.profile.group({key: {ns: true}, initial: {count: 0}, reduce: function(obj,prev){ prev.count++;}})

 

Collection with the slowest queries(No. Of millis spent)

db.system.profile.group({key: {ns: true}, initial: {millis: 0}, reduce: function(obj, prev){ prev.millis += obj.millis;}})

 

Most recent slow query

db.system.profile.find().sort({$natural: -1}).limit(1)

 

Single slowest query(Right now)

db.system.profile.find().sort({millis: -1}).limit(1)

posted @ 2020-03-27 23:35 paulwong 阅读(308) | 评论 (0)编辑 收藏

基于LINUX的分布式文件系统GlusterFS + NFS-Ganesha

基于LINUX的,也就是用yum install就可以使用。

GlusterFS是分布文件存储系统, 也即一个文件有三个备份,每个备份可以放到不同的节点(IP)上,这样某个节点CRASH后,会从其他节点取文件。

NFS-Ganesha则是用户层面非KERNAL层面的实现了NFS SERVER的功能,但双加了扩展,对外提供基于NFS协议的文件存取服务。

资源:
GlusterFS and NFS-Ganesha integration
https://staged-gluster-docs.readthedocs.io/en/release3.7.0beta1/Features/glusterfs_nfs-ganesha_integration/

Exporting and Unexporting Volumes through nfs-ganesha
https://access.redhat.com/documentation/en-US/Red_Hat_Storage/2.1/html/Administration_Guide/Starting_and_Stopping_nfs-ganesha.html

https://www.snia.org/sites/default/files/Poornima_NFS_GaneshaForClusteredNAS.pdf





posted @ 2020-03-22 11:46 paulwong 阅读(1077) | 评论 (0)编辑 收藏

开机nfs自动挂载

1.echo "mount -t nfs -o nolock ${IP}:${remote_dir} ${local_dir}" >>  /etc/rc.local

2.echo "${IP}:/home/logs /home/logs nfs defaults 0 0" >> /etc/fstab

关于/etc/rc.local


rc.local也是我经常使用的一个脚本。该脚本是在系统初始化级别脚本运行之后再执行的,因此可以安全地在里面添加你想在系统启动之后执行的脚本。常见的情况是你可以再里面添加nfs挂载/mount脚本。此外,你也可以在里面添加一些调试用的脚本命令。例如,我就碰到过这种情况:samba服务总是无法正常运行,而检查发现,samba是在系统启动过程中就该启动执行的,也就是说,samba守护程序配置保证了这种功能本应该正确执行。碰到这种类似情况,一般我也懒得花大量时间去查为什么,我只需要简单的在/etc/rc.local脚本里加上这么一行:

/etc/init.d/samba start

这样就成功的解决了samba服务异常的问题。

posted @ 2020-03-21 19:44 paulwong 阅读(836) | 评论 (0)编辑 收藏

利用 Chef 在 Red Hat Enterprise Linux 上自动化部署 Mariadb Galera Cluster

https://www.ibm.com/developerworks/cn/linux/1611_chensz_mgc/index.html

posted @ 2020-03-21 10:55 paulwong 阅读(299) | 评论 (0)编辑 收藏

在SPRING BOOT中使用多JMS CONNECTION

使用自定义CONNECTION FACTORY,这样会覆盖SPRING 的AUTO CONFIGURATION。

ActiveMQConnectionFactoryFactory.java
import java.lang.reflect.InvocationTargetException;
import java.util.Collections;
import java.util.List;

import org.apache.activemq.ActiveMQConnectionFactory;
import org.springframework.boot.autoconfigure.jms.activemq.ActiveMQConnectionFactoryCustomizer;
import org.springframework.boot.autoconfigure.jms.activemq.ActiveMQProperties;
import org.springframework.boot.autoconfigure.jms.activemq.ActiveMQProperties.Packages;
import org.springframework.util.Assert;
import org.springframework.util.StringUtils;


/**
 * Factory to create a {
@link ActiveMQConnectionFactory} instance from properties defined
 * in {
@link SecondaryActiveMQProperties}.
 *
 * 
@author Phillip Webb
 * 
@author Venil Noronha
 
*/
class ActiveMQConnectionFactoryFactory {

    private static final String DEFAULT_EMBEDDED_BROKER_URL = "vm://localhost?broker.persistent=false";

    private static final String DEFAULT_NETWORK_BROKER_URL = "tcp://localhost:61616";

    private final ActiveMQProperties properties;

    private final List<ActiveMQConnectionFactoryCustomizer> factoryCustomizers;

    ActiveMQConnectionFactoryFactory(ActiveMQProperties properties,
            List<ActiveMQConnectionFactoryCustomizer> factoryCustomizers) {
        Assert.notNull(properties, "Properties must not be null");
        this.properties = properties;
        this.factoryCustomizers = (factoryCustomizers != null) ? factoryCustomizers : Collections.emptyList();
    }

    public <T extends ActiveMQConnectionFactory> T createConnectionFactory(Class<T> factoryClass) {
        try {
            return doCreateConnectionFactory(factoryClass);
        }
        catch (Exception ex) {
            throw new IllegalStateException("Unable to create " + "ActiveMQConnectionFactory", ex);
        }
    }

    private <T extends ActiveMQConnectionFactory> T doCreateConnectionFactory(Class<T> factoryClass) throws Exception {
        T factory = createConnectionFactoryInstance(factoryClass);
        if (this.properties.getCloseTimeout() != null) {
            factory.setCloseTimeout((intthis.properties.getCloseTimeout().toMillis());
        }
        factory.setNonBlockingRedelivery(this.properties.isNonBlockingRedelivery());
        if (this.properties.getSendTimeout() != null) {
            factory.setSendTimeout((intthis.properties.getSendTimeout().toMillis());
        }
        Packages packages = this.properties.getPackages();
        if (packages.getTrustAll() != null) {
            factory.setTrustAllPackages(packages.getTrustAll());
        }
        if (!packages.getTrusted().isEmpty()) {
            factory.setTrustedPackages(packages.getTrusted());
        }
        customize(factory);
        return factory;
    }

    private <T extends ActiveMQConnectionFactory> T createConnectionFactoryInstance(Class<T> factoryClass)
            throws InstantiationException, IllegalAccessException, InvocationTargetException, NoSuchMethodException {
        String brokerUrl = determineBrokerUrl();
        String user = this.properties.getUser();
        String password = this.properties.getPassword();
        if (StringUtils.hasLength(user) && StringUtils.hasLength(password)) {
            return factoryClass.getConstructor(String.class, String.class, String.class).newInstance(user, password,
                    brokerUrl);
        }
        return factoryClass.getConstructor(String.class).newInstance(brokerUrl);
    }

    private void customize(ActiveMQConnectionFactory connectionFactory) {
        for (ActiveMQConnectionFactoryCustomizer factoryCustomizer : this.factoryCustomizers) {
            factoryCustomizer.customize(connectionFactory);
        }
    }

    String determineBrokerUrl() {
        if (this.properties.getBrokerUrl() != null) {
            return this.properties.getBrokerUrl();
        }
        if (this.properties.isInMemory()) {
            return DEFAULT_EMBEDDED_BROKER_URL;
        }
        return DEFAULT_NETWORK_BROKER_URL;
    }
}

TwinJmsConnectionFactoryConfiguration.java
import java.util.stream.Collectors;

import org.apache.activemq.ActiveMQConnectionFactory;
import org.messaginghub.pooled.jms.JmsPoolConnectionFactory;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.autoconfigure.jms.JmsPoolConnectionFactoryFactory;
import org.springframework.boot.autoconfigure.jms.activemq.ActiveMQConnectionFactoryCustomizer;
import org.springframework.boot.autoconfigure.jms.activemq.ActiveMQProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Profile;


@Configuration
@Profile({"local"})
public class TwinJmsConnectionFactoryConfiguration {

    @Bean
    @ConfigurationProperties(prefix = "spring.activemq.primary")
    public ActiveMQProperties primaryActiveMQProperties() {
        return new ActiveMQProperties();
    }
    
    @Bean(destroyMethod = "stop")
    @Primary
    @ConditionalOnProperty(prefix = "spring.activemq.pool", name = "enabled", havingValue = "true")
    public JmsPoolConnectionFactory connectionFactory(ActiveMQProperties primaryActiveMQProperties,
            ObjectProvider<ActiveMQConnectionFactoryCustomizer> factoryCustomizers) {
        ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactoryFactory(primaryActiveMQProperties,
                factoryCustomizers.orderedStream().collect(Collectors.toList()))
                .createConnectionFactory(ActiveMQConnectionFactory.class);
        return new JmsPoolConnectionFactoryFactory(primaryActiveMQProperties.getPool())
                .createPooledConnectionFactory(connectionFactory);
    }
    
    ////////////////////////////////////////////////////////////////////////////////
    @Bean
    @ConfigurationProperties(prefix = "spring.activemq.sescond")
    public ActiveMQProperties sescondActiveMQProperties() {
        return new ActiveMQProperties();
    }
    
    @Bean(destroyMethod = "stop")
    @ConditionalOnProperty(prefix = "spring.activemq.pool", name = "enabled", havingValue = "true")
    public JmsPoolConnectionFactory sescondPooledJmsConnectionFactory(ActiveMQProperties sescondActiveMQProperties,
            ObjectProvider<ActiveMQConnectionFactoryCustomizer> factoryCustomizers) {
        ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactoryFactory(sescondActiveMQProperties,
                factoryCustomizers.orderedStream().collect(Collectors.toList()))
                .createConnectionFactory(ActiveMQConnectionFactory.class);
        return new JmsPoolConnectionFactoryFactory(sescondActiveMQProperties.getPool())
                .createPooledConnectionFactory(connectionFactory);
    }
    
}


posted @ 2020-03-19 09:45 paulwong 阅读(631) | 评论 (0)编辑 收藏

Which is better: PooledConnectionFactory or CachingConnectionFactory?

From here:

The difference between the PooledConnectionFactory and the CachingConnectionFactory is a difference in implementation. Below are some of the characteristics that differ between them:

  • Although both the PooledConnectionFactory and the CachingConnectionFactory state that they each pool connections, sessions and producers, the PooledConnectionFactory does not actually create a cache of multiple producers. It simply uses a singleton pattern to hand out a single cached producer when one is requested. Whereas the CachingConnectionFactory actually creates a cache containing multiple producers and hands out one producer from the cache when one is requested.

  • The PooledConnectionFactory is built on top of the Apache Commons Pool project for pooling JMS sessions. This allows some additional control over the pool because there are features in Commons Pool that are not being used by the PooledConnectionFactory. These additional features include growing the pool size instead of blocking, throwing an exception when the pool is exhausted, etc. You can utilize these features by creating your own Commons Pool GenericObjectPool using your own customized settings and then handing that object to the PooledConnectionFactory via the setPoolFactory method. See the following for additional info: http://commons.apache.org/pool/api-1.4/org/apache/commons/pool/impl/GenericObjectPoolFactory.html

  • The CachingConnectionFactory has the ability to also cache consumers. Just need to take care when using this feature so that you know the consumers are cached according to the rules noted in the blog post.

  • But most importantly, the CachingConnectionFactory will work with any JMS compliant MOM. It only requires a JMS connection factory. This is important if you are using more than one MOM vendor which is very common in enterprise organizations (this is mainly due to legacy and existing projects). The important point is that the CachingConnectionFactory works very well with many different MOM implementations, not only ActiveMQ.

From here:

  • If you have clustered ActiveMQs, and use failover transport it has been reported that CachingConnectionFactory is not a right choice.

  • The problem I’m having is that if one box goes down, we should start sending messages on the other, but it seems to still be using the old connection (every send times out). If I restart the program, it’ll connect again and everything works. Source: Autoreconnect problem with ActiveMQ and CachingConnectionFactory

  • The problem is that cached connections to the failed ActiveMQ was still in use and that created the problem for the user. Now, the choice for this scenario is PooledConnectionFactory.

  • If you’re using ActiveMQ today, and chances are that you may switch to some other broker (JBoss MQ, WebSphere MQ) in future, do not use PooledConnectionFactory, as it tightly couples your code to ActiveMQ.

posted @ 2020-03-19 09:37 paulwong 阅读(425) | 评论 (0)编辑 收藏

Spring Boot Data Mongodb Starter自动配置那些坑

正好做Mongodb主从复制尝试使用Spring Boot Data Mongodb Starter插件链接访问Mongodb数据库集群。

遇到的坑:

  • spring.data.mongodb.host和spring.data.mongodb.port形式不适合集群配置,会报host无法识别异常
  • spring.data.mongodb.uri中经常抛出authentication failed异常


解决办法:

  1.  对于第一个坑,请使用spring.data.mongodb.uri。如果使用了uri,则其余的host/username/password/db/auth-db这些全部无效。
  2.  对于第二个坑,请在spring.data.mongodb.uri中指定replicaSet和authsource,另外记得把所有集群节点服务器地址都列全。
    如果auth-db和db是同一个,则无需加authsource,如果不同,则加authsource=admin


我没有把authsource指定,所以一直报authentication failed异常。然后只好一点点去发掘问题点,最后查到在com.mongodb.ConnectionString类中的createCredentials中

private MongoCredential createCredentials(final Map<String, List<String>> optionsMap, final String userName,
                                              final char[] password) {
        AuthenticationMechanism mechanism = null;
        String authSource = (database == null) ? "admin" : database;
        String gssapiServiceName = null;
        String authMechanismProperties = null;

        for (final String key : AUTH_KEYS) {
            String value = getLastValue(optionsMap, key);

            if (value == null) {
                continue;
            }

            if (key.equals("authmechanism")) {
                mechanism = AuthenticationMechanism.fromMechanismName(value);
            } else if (key.equals("authsource")) {
                authSource = value;
            } else if (key.equals("gssapiservicename")) {
                gssapiServiceName = value;
            } else if (key.equals("authmechanismproperties")) {
                authMechanismProperties = value;
            }
        }


        MongoCredential credential = null;
        if (mechanism != null) {
            switch (mechanism) {
                case GSSAPI:
                    credential = MongoCredential.createGSSAPICredential(userName);
                    if (gssapiServiceName != null) {
                        credential = credential.withMechanismProperty("SERVICE_NAME", gssapiServiceName);
                    }
                    break;
                case PLAIN:
                    credential = MongoCredential.createPlainCredential(userName, authSource, password);
                    break;
                case MONGODB_CR:
                    credential = MongoCredential.createMongoCRCredential(userName, authSource, password);
                    break;
                case MONGODB_X509:
                    credential = MongoCredential.createMongoX509Credential(userName);
                    break;
                case SCRAM_SHA_1:
                    credential = MongoCredential.createScramSha1Credential(userName, authSource, password);
                    break;
                default:
                    throw new UnsupportedOperationException(format("The connection string contains an invalid authentication mechanism'. "
                                                                           + "'%s' is not a supported authentication mechanism",
                            mechanism));
            }
        } else if (userName != null) {
            credential = MongoCredential.createCredential(userName, authSource, password);
        }

        if (credential != null && authMechanismProperties != null) {
            for (String part : authMechanismProperties.split(",")) {
                String[] mechanismPropertyKeyValue = part.split(":");
                if (mechanismPropertyKeyValue.length != 2) {
                    throw new IllegalArgumentException(format("The connection string contains invalid authentication properties. "
                            + "'%s' is not a key value pair", part));
                }
                String key = mechanismPropertyKeyValue[0].trim().toLowerCase();
                String value = mechanismPropertyKeyValue[1].trim();
                if (key.equals("canonicalize_host_name")) {
                    credential = credential.withMechanismProperty(key, Boolean.valueOf(value));
                } else {
                    credential = credential.withMechanismProperty(key, value);
                }
            }
        }
        return credential;
    }


authSource默认会指向我们目标数据的数据库。然而在身份验证机制中我们通常需要指向admin。(非常想报粗口,代码作者在这里脑袋被men挤了么)。所以需要强制指定authSource中指定。具体指定方式如下:

 

 

 

 

mongodb://{用户名}:{密码}@{host1}:27017,{host2}:27017,{host3}:27017/{目标数据库}?replicaSet={复制集名称}&write=1&readPreference=primary&authsource={授权数据库}

posted @ 2020-03-17 09:39 paulwong 阅读(1955) | 评论 (0)编辑 收藏

仅列出标题
共115页: First 上一页 18 19 20 21 22 23 24 25 26 下一页 Last