Magento 2 optimize the queue consumers

If you're experiencing issues with Magento 2 cron jobs getting stuck or running for a long time, there are ways to optimize how Magento handles the queue consumers.

By default, consumers will keep polling for messages as long as the number of processed messages is less than the max_messages value, which can be specified in the env.php file. However, this approach is typically recommended for larger merchants where a constant flow of messages is expected.

For smaller merchants, it's suggested that the consumers close the TCP connection and terminate without waiting for additional messages to enter the queue. To change this default behavior, you can add or modify your env.php file with the following instructions.

'queue' => [
    'consumers_wait_for_messages' => 0,
]

Moreover, if your shop is powered by Magento 2.4.1 or a newer version, you can further decrease CPU and memory usage by including the "only_spawn_when_message_available" parameter in the queue configuration array. Setting this parameter to true or 1 will ensure that the consumer only runs when there is a message available in the queue, and it will terminate automatically when there are no more messages to process. To configure this, you can follow the example below in your env.php file:

'queue' => [
    'consumers_wait_for_messages' => 0,
    'only_spawn_when_message_available' => 1
]