paulwong

2020年7月15日 #

Spring Cloud异步场景分布式事务怎样做?试试RocketMQ

例如有如下场景,新增一订单,同时为此订单的用户增加积分。场景对应场景表,积分对应积分表,如果要防止订单增加成功而积分增加不成功,则要将此两种操作放在一个事务下。

分布式的场景下,订单服务在一个JVM下,积分服务在另一个JVM下,两者要如何才能达到数据一致(原子)性?


https://zlt2000.gitee.io/2019-09-23-rocketmq-transaction/

posted @ 2020-07-15 10:33 paulwong 阅读(12) | 评论 (0)编辑 收藏

!!zlt-microservices-platform

1. 总体架构图

mark


2. 功能介绍

mark




https://github.com/zlt2000/microservices-platform 

https://zlt2000.gitee.io/

产品文档(付费,微信登录):
https://www.kancloud.cn/zlt2000/microservices-platform


posted @ 2020-07-15 10:15 paulwong 阅读(7) | 评论 (0)编辑 收藏

2020年7月6日 #

LINUX SHELL

!!
https://tecadmin.net/tutorial/bash-scripting/

Check existence of input argument in a Bash shell script
https://stackoverflow.com/questions/6482377/check-existence-of-input-argument-in-a-bash-shell-script

How to Check if a File or Directory Exists in Bash
https://linuxize.com/post/bash-check-if-file-exists/

bash string compare to multiple correct values
https://stackoverflow.com/questions/21157435/bash-string-compare-to-multiple-correct-values

Bash – Check If Two Strings are Equal
https://tecadmin.net/tutorial/bash/examples/check-if-two-strings-are-equal/





posted @ 2020-07-06 09:23 paulwong 阅读(20) | 评论 (0)编辑 收藏

2020年6月23日 #

如何优雅地停止SPRING BATCH中的REMOTE CHUNKING JOB

SPRING BATCH中的REMOTE CHUNKING JOB,由于是基于MASTER/SLAVE的架构,其中某个STEP是会在远程机器中执行,如果要停止这个JOB,需要考虑两个问题:
1、什么时候发出停止指令
2、如何等待远程STEP的完成

一般停止JOB,可用JobOperator.stop(long executionId)来停止,但这个无法确定什么时候发出停止指令,如果是在CHUNK的处理中途发出,则会出现回滚的现象。
BATCH_STEP_EXECUTION thead tr {background-color: ActiveCaption; color: CaptionText;} th, td {vertical-align: top; font-family: "Tahoma", Arial, Helvetica, sans-serif; font-size: 8pt; padding: 4px; } table, td {border: 1px solid silver;} table {border-collapse: collapse;} thead .col0 {width: 173px;} .col0 {text-align: right;} thead .col1 {width: 82px;} .col1 {text-align: right;} thead .col2 {width: 282px;} thead .col3 {width: 164px;} .col3 {text-align: right;} thead .col4 {width: 161px;} thead .col5 {width: 161px;} thead .col6 {width: 109px;} thead .col7 {width: 127px;} .col7 {text-align: right;} thead .col8 {width: 109px;} .col8 {text-align: right;} thead .col9 {width: 118px;} .col9 {text-align: right;} thead .col10 {width: 117px;} .col10 {text-align: right;} thead .col11 {width: 142px;} .col11 {text-align: right;} thead .col12 {width: 150px;} .col12 {text-align: right;} thead .col13 {width: 166px;} .col13 {text-align: right;} thead .col14 {width: 137px;} .col14 {text-align: right;} thead .col15 {width: 109px;} thead .col16 {width: 156px;} thead .col17 {width: 161px;}
STEP_EXECUTION_ID VERSION STEP_NAME JOB_EXECUTION_ID START_TIME END_TIME STATUS COMMIT_COUNT READ_COUNT FILTER_COUNT WRITE_COUNT READ_SKIP_COUNT WRITE_SKIP_COUNT PROCESS_SKIP_COUNT ROLLBACK_COUNT EXIT_CODE EXIT_MESSAGE LAST_UPDATED
2304 169 step2HandleXXX 434 2020-06-22 16:27:54 2020-06-22 16:32:46 STOPPED 167 5010 0 4831 0 155 0 161 STOPPED org.springframework.batch.core.JobInterruptedException 2020-06-22 16:32:46


另外SPRING BATCH也不会等远程STEP执行完成,就将JOB的状态设为Complete。

发出停止的指令应通过ChunkListener达成:

public class ItemMasterChunkListener extends ChunkListenerSupport{
    
    private static final Logger log = LoggerFactory.getLogger(ItemMasterChunkListener.class);
    
    
    @Override
    public void beforeChunk(ChunkContext context) {
        log.info("ItemMasterProcessor.beforeChunk");
    }


    @Override
    public void afterChunk(ChunkContext context) {
        log.info("ItemMasterProcessor.afterChunk");
        if(XXXX.isStoppingOrPausing()) {
            log.info("context.getStepContext().getStepExecution().setTerminateOnly()");
            context.getStepContext().getStepExecution().setTerminateOnly();
        }
    }


    @Override
    public void afterChunkError(ChunkContext context) {
        log.info("ItemMasterProcessor.afterChunkError");
    }


}


配置BEAN:

@Bean
@StepScope
public ItemMasterChunkListener novaXItemMasterChunkListener() {
     return new ItemMasterChunkListener();
}
    
this.masterStepBuilderFactory
                    .<X, X>get("step2Handle")
                    .listener(itemMasterChunkListener())
                    .build();


由于是在CHUNK完成的时候发出停止指令,就不会出现ROLLBACK的情况。

等待远程STEP完成,通过读取MQ上的MESSAGE是否被消费完成,PENDDING的MESSAGE为0的条件即可。

public class JobExecutionListenerSupport implements JobExecutionListener {

    /* (non-Javadoc)
     * @see org.springframework.batch.core.domain.JobListener#afterJob()
     
*/
    @Override
    public void afterJob(JobExecution jobExecution) {
        Integer totalPendingMessages = 0;
        String queueName = "";
        
        
        String messageSelector = "JOB_EXECUTION_ID=" + jobExecution.getJobInstance().getInstanceId();
        do{
            totalPendingMessages = 
                    this.jmsTemplate.browseSelected(queueName, messageSelector, 
                                (session, browser) -> 
                                    Collections.list(browser.getEnumeration()).size()
                            );
            
            String brokerURL = null;
            if(jmsTemplate.getConnectionFactory() instanceof JmsPoolConnectionFactory) {
                JmsPoolConnectionFactory connectionFactory =
                        (JmsPoolConnectionFactory)jmsTemplate.getConnectionFactory();
                ActiveMQConnectionFactory activeMQConnectionFactory =
                        (ActiveMQConnectionFactory)connectionFactory.getConnectionFactory();
                brokerURL = activeMQConnectionFactory.getBrokerURL();
            } else if(jmsTemplate.getConnectionFactory() instanceof CachingConnectionFactory) {
                CachingConnectionFactory connectionFactory =
                        (CachingConnectionFactory)jmsTemplate.getConnectionFactory();
                ActiveMQConnectionFactory activeMQConnectionFactory =
                        (ActiveMQConnectionFactory)connectionFactory.getTargetConnectionFactory();
                brokerURL = activeMQConnectionFactory.getBrokerURL();
            }
            
            LOGGER.info("queueName = {}, {}, totalPendingMessages = {}, url={}", 
                    queueName, messageSelector, totalPendingMessages, brokerURL);
            Assert.notNull(totalPendingMessages, "totalPendingMessages must not be null.");
            try {
                Thread.sleep(5_000);
            } catch (InterruptedException e) {
                LOGGER.error(e.getMessage(), e);
            }
        } while(totalPendingMessages.intValue() > 0);
        
    }

    /* (non-Javadoc)
     * @see org.springframework.batch.core.domain.JobListener#beforeJob(org.springframework.batch.core.domain.JobExecution)
     
*/
    @Override
    public void beforeJob(JobExecution jobExecution) {
    }

}


这样整个JOB就能无异常地停止,且会等待远程STEP完成。

Reference:
https://docs.spring.io/spring-batch/docs/4.1.3.RELEASE/reference/html/common-patterns.html#stoppingAJobManuallyForBusinessReasons

https://stackoverflow.com/questions/13603949/count-number-of-messages-in-a-jms-queue

https://stackoverflow.com/questions/55499965/spring-batch-stop-job-execution-from-external-class

https://stackoverflow.com/questions/34621885/spring-batch-pollable-channel-with-replies-contains-chunkresponses-even-if-job


posted @ 2020-06-23 11:00 paulwong 阅读(130) | 评论 (0)编辑 收藏

2020年6月16日 #

为啥文件的CHECKSUM中SHA512比MD5高级?

https://stackoverflow.com/questions/2117732/reasons-why-sha512-is-superior-to-md5

posted @ 2020-06-16 10:21 paulwong 阅读(23) | 评论 (0)编辑 收藏

2020年6月4日 #

GIT资源


http://jartto.wang/tags/git/

posted @ 2020-06-04 10:38 paulwong 阅读(25) | 评论 (0)编辑 收藏

彻底搞懂 Git-Rebase

根据分支1新建了功能分支1,并在此上开发一段时间,后来分支1被别人提交了代码,因此分支1比功能分支1要新,这时,可以将功能分支1与分支1进行合并,但会多出很多COMMIT,这时就出现了rebase,
GIT会将功能分支1上的所有COMMIT另存一个文件,回退到分支1原始状态,再更新至当前分支1的状态,再把另存文件的COMMIT执行一遍,就成了已经合并的新的功能分支1。

http://jartto.wang/2018/12/11/git-rebase/

posted @ 2020-06-04 10:37 paulwong 阅读(19) | 评论 (0)编辑 收藏

2020年5月11日 #

How To Run Java Jar Application with Systemd on Linux

https://computingforgeeks.com/how-to-run-java-jar-application-with-systemd-on-linux/

systemd自启动java程序
https://www.cnblogs.com/yoyotl/p/8178363.html
------------------------------------------------------------

[Unit]
Description=TestJava
After=network.target

[Service]
Type=forking
ExecStart=/home/test/startTest.sh
ExecStop=/home/test/stopTest.sh

[Install]
WantedBy=multi-user.target

-------------------------------------------------------------
How to Autorun application at the start up in Linux
https://developer.toradex.com/knowledge-base/how-to-autorun-application-at-the-start-up-in-linux

How to automatically run program on Linux startup
https://www.simplified.guide/linux/automatically-run-program-on-startup


Systemd 入门教程:实战篇
https://www.ruanyifeng.com/blog/2016/03/systemd-tutorial-part-two.html

Systemd 入门教程:命令篇

http://www.ruanyifeng.com/blog/2016/03/systemd-tutorial-commands.html

posted @ 2020-05-11 16:16 paulwong 阅读(53) | 评论 (0)编辑 收藏

2020年5月9日 #

MariaDB Galera Cluster

What is MariaDB Galera Cluster?
https://mariadb.com/kb/en/what-is-mariadb-galera-cluster/

Prepare yum install repository:
https://downloads.mariadb.org/mariadb/repositories/#distro=CentOS&distro_release=centos7-amd64--centos7&mirror=coreix&version=10.4

MariaDB Galera Cluster部署实战
https://jeremyxu2010.github.io/2018/02/mariadb-galera-cluster%E9%83%A8%E7%BD%B2%E5%AE%9E%E6%88%98/

9 Tips for Going in Production with Galera Cluster for MySQL
https://severalnines.com/blog/9-tips-going-production-galera-cluster-mysql

HA for MySQL and MariaDB - Comparing Master-Master Replication to Galera Cluster
https://severalnines.com/database-blog/ha-mysql-and-mariadb-comparing-master-master-replication-galera-cluster

Galera Cluster for MySQL - Tutorial
https://severalnines.com/resources/tutorials/galera-cluster-mysql-tutorial










posted @ 2020-05-09 11:08 paulwong 阅读(31) | 评论 (0)编辑 收藏

2020年5月6日 #

How to disable IPv6 on CentOS / RHEL 7


https://www.thegeekdiary.com/centos-rhel-7-how-to-disable-ipv6/

https://linuxconfig.org/redhat-8-enable-disable-ipv6


posted @ 2020-05-06 12:42 paulwong 阅读(33) | 评论 (0)编辑 收藏

仅列出标题  下一页