Multiple NPEs encountered when running under high load/latency in cluster mode (via hazelcast plugin)
Description
When a cluster experiences high load and/or latency, cluster task execution timeouts may begin to occur. This has the effect of returning null values for various remote objects, especially remote client sessions which are accessed frequently.
Here is an example:
java.lang.NullPointerException at org.jivesoftware.openfire.spi.RoutingTableImpl$2.compare(RoutingTableImpl.java:608) at org.jivesoftware.openfire.spi.RoutingTableImpl$2.compare(RoutingTableImpl.java:606) at java.util.TimSort.countRunAndMakeAscending(TimSort.java:324) at java.util.TimSort.sort(TimSort.java:189) at java.util.TimSort.sort(TimSort.java:173) at java.util.Arrays.sort(Arrays.java:659) at java.util.Collections.sort(Collections.java:217) at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToBareJID(RoutingTableImpl.java:606) at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToLocalDomain(RoutingTableImpl.java:293) at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.java:238) at org.jivesoftware.openfire.MessageRouter.route(MessageRouter.java:138) at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:80) at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:114) at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:72) at org.jivesoftware.openfire.http.HttpSession.sendPendingPackets(HttpSession.java:650) at org.jivesoftware.openfire.http.HttpSessionManager$HttpPacketSender.run(HttpSessionManager.java:425) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)