connection_pool: fix excessive idle CPU usage while waiting for requests

[ThCompiler]

While using ConnectionPool, it was found that it uses 100% cpu for its threads when at rest. After examining its operation, it was realized that the _request_process_loop runs in an infinite loop, without blocking on waiting for the next request. To reduce the load, I suggest using already implemented wait for the next element in the queue's from package queue .

To check the solution, I implemented a simple test that calculates the n-th element of the Fibonacci sequence using a ConnectionPool running in the background:

import queue
import time

from tarantool import Connection
from tarantool.connection_pool import ConnectionPool as OriginalConnectionPool

def fib(n):
    a = 0
    b = 1
    for _ in range(n):
        a, b = b, a + b
    return a


class ConnectionPool(OriginalConnectionPool):
    # Override of the standard pool loop, which loaded the processor at 100% 
    # with an infinite constant loop
    def _request_process_loop( # noqa: max-complexity=8
        self, key, unit, last_refresh,
    ):
        """
        Request process background loop for a pool server. Started in
        a separate thread, one thread per server.

        :param key: Result of
            :meth:`~tarantool.connection_pool._make_key`.
        :type key: :obj:`str`

        :param unit: Server metainfo.
        :type unit: :class:`~tarantool.connection_pool.PoolUnit`

        :param last_refresh: Time of last metainfo refresh.
        :type last_refresh: :obj:`float`
        """

        while unit.request_processing_enabled:
            try:
                task = unit.input_queue.get(timeout=self.refresh_delay)
            except queue.Empty:
                task = None

            if task:
                method = getattr(Connection, task.method_name)
                try:
                    resp = method(unit.conn, *task.args, **task.kwargs)
                except Exception as exc:
                    unit.output_queue.put(exc)
                else:
                    unit.output_queue.put(resp)

            now = time.time()
            if now - last_refresh > self.refresh_delay:
                self._refresh_state(key)
                last_refresh = time.time()

def test_bench_updated_pool(benchmark):
    pool = ConnectionPool(
        addrs=[
            {
                'host': '127.0.0.1',
                'port': 3301,
            },
            {
                'host': '127.0.0.1',
                'port': 3302,
            },
            {
                'host': '127.0.0.1',
                'port': 3303,
            },
        ],
        user='guest',
    )

    benchmark(fib, 1000000)

    pool.close()

def test_bench_original_pool(benchmark):
    pool = OriginalConnectionPool(
        addrs=[
            {
                'host': '127.0.0.1',
                'port': 3301,
            },
            {
                'host': '127.0.0.1',
                'port': 3302,
            },
            {
                'host': '127.0.0.1',
                'port': 3303,
            },
        ],
        user='guest',
    )

    benchmark(fib, 1000000)

    pool.close()

I ran the test with the changes and the old ConnectionPool behavior. So i got the following test results:

I also looked at the distribution of processor activity using py-spy for each of the tests:

• for current behavior:

• for new behavior:

As you can see from the py-spy snapshots, after the change, the processor is almost fully used for calculating Fibonacci sequence, while with the current behavior, most of the time is spent on _request_process_loop.

[bigbes]

lgtm

[DifferentialOrange]

Seems like a nice fix, let's add a CHANGELOG entry about its effect.

[ThCompiler]

Added entry about changes to CHANGELOG.

[DifferentialOrange]

Commit message codestyle should be something like

connection_pool: fix cpu overload if stale

[ThCompiler]

Updated commit message in accordance with code style.

[oleg-jukovec]

Please, update the commit message (see above) and add a new entry to the CHANGELOG.md.

[ThCompiler]

I had to update the build os version for the plugin to 24.04, as it stopped supporting 20.04. 24.04 was selected based on a similar version update in #330.

[ThCompiler]

I checked failed jobs on CI.

Windows system jobs fail because the default version of setuptools is 65.5.0 in the CI image. When dependencies are installed in a step, setuptools updates to the latest version 82.0.1 which meets the requirements of >=75.3.2. The pkg_resources has been removed in this version of setuptools. https://setuptools.pypa.io/en/latest/. As a quick solution, I can set the upper bound for the setuptools version to <82.0 in the testing requirements.

In Linux, a jobs that use the master Tarantool branch fail because the maximum decimal precision has been increased to 76 in Tarantool 3.5. tarantool/tarantool#11497. I can try to fix this in the implementation of the decimal type for msgpack.

Should I make these changes in this pull request?