THURSDAY, FEBRUARY 9, 2012
Multicast Vs. Unicast with WebLogic Clustering
WebLogic Clusters manage membership via messaging within its members. The members join/leave the cluster as well as update other members via messages to the entire cluster. There are two ways for cluster messaging in WLS - Multicast or Unicast. This post goes provides a brief overview of the cluster messaging and some general guidelines around using the different messaging options.
WebLogic clustering provides a homogeneous model for managing a set of server instances while providing scalability, load balancing and high availability to the services running within the member instances. For the end client, it appears like one single and uniform service with all the RASP (reliability, availability, scalability, performance) capabilities. Each cluster member is aware of other members and advertise each other services in addition to their own. Refer tohttp://docs.oracle.com/cd/E11035_01/wls100/cluster/failover.html for more details on replication and failover of clustered services.
Unicast & Multicast
Multicast is easier to explain over Unicast. Multicast is a broadcast UDP option for sending a packet/announcement over to a group that is listening on a specific multicast address and port over which the announcement is sent.There is a defined range for valid Multicast address (220.127.116.11 to 18.104.22.168). Everyone listening on the given address hears the announcement just like following a Twitter post. Some limitations with Multicast is the TTL (time to live) across machines/subnets/routers needs to be adjusted and the routers configured to retransmit the multicast packet across subnets. More details on the weblogic specific multicast configurations can be seen in my Exalogic and Multicast blog post.
Unicast is more of a point to point UDP option to send the packet to a specific member and not everyone. That way, unicast is more of a private conversation between two individuals while multicast is more of a shout to a group or room. Both are UDP based, so there can be losses unlike TCP that handles retransmissions on message loss. But Unicast can span across routers and does not have to worry about TTL without the everyone hearing the announcement. So, Network Admins in general prefer to go with Unicast over Multicast for these reasons.
WLS Cluster Configuration
When a cluster is created within a WLS Domain (either via wlst or config wizard or copy/update of existing domain), it is configured as using Unicast or Multicast messaging. As part of complete Cluster configuration, the multicast listen address and port should be specified if going with Multicast option. Also the managed servers should be targeted/added to the Cluster instance. There can be multiple clusters and any given managed server instance can only belong to atmost one cluster (or none). This will let the clustered managed servers (Admin Server should not be part of any cluster nor can cluster span domains) go with Unicast or Multicast when they are started. It should be noted that Multicast used to be the only option in WebLogic prior to 10.0 version while either Unicast or Multicast can be used from version 10 onwards.
How does each member join the cluster and see or recognize others? At bootstrap time, the managed servers get their configuration via JMX MBeans from the Admin server (or from cached configurations if the Admin server was down) and recognize that they belong to a cluster and they have to go with Unicast or Multicast.
WLS Multicast Messaging
So, in a multicast messaging cluster, the managed servers start listening to the specified multicast address once it checks its configuration and knows it belongs to so and so cluster and has to use specified multicast address and port. Once it listens on the specified multicast address, it sends an announcement about its arrival to others via multicast - more like a shout-out. Other running clustered members of the same cluster who are already listening will respond back and add the new member to their list of known cluster members. The new member will also update its cluster list with other members. This process continues as new members get added. The membership gets renewed based on each member sending periodic announcements to other members proclaiming its liveliness. If a member goes down (shutdown or killed or not able to respond), then it wont be able to send its broadcasts and other members will drop it from the cluster list and re-add it when it comes back online. If a member is directly talking to another cluster member, just the direct socket connection is enough in establishing its membership with its connected member. Since multicast is a broadcast, just a single announcement is retain a instance's membership and let others continue to maintain it in their cluster list till its time to renew its membership. Its a mesh where every member can see every other member in the cluster.
WLS Unicast Messaging
With Unicast, WLS divides the cluster into a multiple groups, each having a max of 10 members. The division happens as the servers come up. Each of the server listens on their specified listen address and the other members can send/receive unicast packets over that listen address as every clustered member knows the configurations of other cluster members.
The oldest within the group is designated the group leader. The group leader communicates over Unicast to other members (over the specified member's listen address) within its group and adds/drops them based on them renewing their membership or not responding. The group leaders communicate amongst themselves. So, if there are 4 group leaders, each group having 10 servers, a membership announcement from server1 in group1 will be picked by GroupLeader1 of group1 and retransmitted to all other members within the group1 as well as to other three group leaders. The remaining group leaders will in-turn retransmit the membership information within their groups. This way, every change is picked up and retransmitted by the group leaders to within their group and to other groups via group leader to group leader communication. The group leaders remain the hub for each group while they themselves form a mesh with other group leaders.
Comparison between Unicast & Multicast
Only option in pre-10.0 versions of WLS, continues to exist in version 10+
Available from version 10 onwards
Requires configurations to Routers, TTL
No configuration required
Requires configuring the Multicast Listen Address and port
Just specify the listen address (can be Default Channel or use a Custom Network Channel for Cluster communication)
One announcement to join/maintain membership
one transmission to group leader has to be retransmitted to other group members (N) + to other group leaders (M) who then again retransmit to their group members resulting in (NxM) packets
Everyone sees everyone
Group Leaders have to do real heavy lifting of retransmitting every thing across its group and other group leaders and can get bogged down in just retransmitting
Can lead to big broadcasts through the entire subnet/LAN if there are frequent joins/drops of members or change in services (JNDI updates of bound services, frequent app deployments or members going out of sync)
Not a broadcast throughout the subnet/lan, but still more packets to be sent across as Group Leaders have to retransmit everything. Can consume bigger bandwidth
As listed in the table above, these would be my guidelines:
1. If the cluster is small and simple (under 20 members), go with Unicast. No configurations required and the group leaders wont be stretched retransmitting data.
2. If the Network configurations strictly prohibit multicast and members have to reside in different subnets and cluster sizes are still under or mid-20s, go with Unicast.
3. If its a real large cluster (over 20 members) and members can reside in the same subnet or even if they are not on same subnet but network router configurations can allow multicast, go with Multicast.
4. If over 50 members, change to network configurations to allow multicast and stick with Multicast.
5. Over 100 members, try to break up the domain into multiple separate domains and individual clusters for better administration and management even though Multicast option can handle such large domains. Or use custom scripts/wlst to manage and monitor individual members instead of relying on a single application (like console) to manage/monitor/handle all servers at the domain level.
The reason for recommending Multicast over Unicast for large clusters is due to the work load on the group leaders and retransmissions. The group leaders have to retransmit every member's packet within their group as well as to other group leaders (who again send to their group members) which can just lead to more and more work as the cluster grows bigger.
Another big reason is that WLS Multicast based messaging is quite mature and stable compared to Unicast which got only introduced with version 10. Also, with unicast, there can still be retransmits (as its not auto error correcting) and it can consume more bandwidth due to repeat retransmits by group leaders to others compared to one transmit for multicast.
I hope this article helps clarify the internal working of WLS Cluster membership around Unicast vs Multicast messaging while providing some guidance on the option to use based on requirements and constraints.
Posted by Sabha Parameswaran at 12:33 PM