Optimize algorithm used for sending packets to remote servers when new connections are required
We have a queue of packets that need to be sent to remote servers for which a connection does not exist. There is a pool of threads that take packets from the pool, try to establish a connection to the remote server and then send the packet. The problem is that when many packets need to be sent to the same remote server then many threads from the pool will be used (one for each packet) and only one will be actually trying to establish the connection while the other threads are blocked. This algorithm does not scale well.
Use the same thread to send all queued packets whose target is the same remote server. This will ensure a correct delivery order and will also leave the thread pool with threads available for sending packets to other servers.
This fix also fixes the packets out of order problem when sending packets to remote servers. However, the packet out of order is not fully fixed yet. There is also a problem while reading and processing the incoming traffic from remote servers. For each received packet (from the same server to the same target local entity) many threads will run in parallel thus breaking the order of the received packets.