Make consumer_timeout configurable via TLS Parameters

[chrisjsewell]

#2990, it's fair to say, is not a universally loved change; for myself, it kind of breaks our whole program 😢.

One compromise I believe makes sense, is to allow consumer_timeout to be set by the consumer, as part of the connection URI (preferably allowing for false), since it seems reasonable the client should have a say in how long it may take to complete a task.

See: https://www.rabbitmq.com/uri-query-parameters.html#tls

Originally posted in #2990 (comment)

[chrisjsewell]

No experience with erlang; but happy to try a PR if what I suggest seems amenable 😬

[michaelklishin]

I will convert this issue to a GitHub discussion. Currently GitHub will automatically close and lock the issue even though your question will be transferred and responded to elsewhere. This is to let you know that we do not intend to ignore this but this is how the current GitHub conversion mechanism makes it seem for the users :(

[michaelklishin]

I'm not sure how TLS is relevant here. I'll assume you mean "connection parameters".

You can "allow for false" by setting the value to something like 24 hours. After a certain threshold, a high enough timeout becomes irrelevant.

AMQP 0-9-1 and AMQP 1.0 clients can specify additional consumer settings at the protocol level. If the goal is to disable the limit, though, why would it matter where the timeout value is fetched from?

[chrisjsewell]

Heya,

I'm not sure how TLS is relevant here. I'll assume you mean "connection parameters".

Well I just mean parameters that can be set via: https://www.rabbitmq.com/uri-query-parameters.html#tls, so yes mainly "connection parameters"

If the goal is to disable the limit, though, why would it matter where the timeout value is fetched from?

well in essence we do not have access to the server; we cannot influence how that is configured (so yeh I guess what you mention below), the only control we have is what is passed in the connection URI which, up until now, has been sufficient

[michaelklishin]

One scenario I can think of where client-side control would be really necessary is hosted RabbitMQ that is entirely outside of application developer's control. @kjnilsson WDYT?

[michaelklishin]

Apparently this setting is at the channel level and there can be multiple consumers per channel, which significantly complicates any possible implementation: delivery tags are not associated with specific consumer state.

consumer_timeout is 30m by default, so it does not affect most people. Those who need multi-hour long processing can acknowledge earlier, even before starting any actual work. Then making such consumer idempotent becomes even more important but timeouts won't be a factor.

[chrisjsewell]

Thanks for the quick response; errr, don't know if this helps our use case, will think on it

[pboekelheide]

Those who need multi-hour long processing can acknowledge earlier, even before starting any actual work. Then making such consumer idempotent becomes even more important but timeouts won't be a factor.

Can I ask how this would be handled in situations where you can lose the consumer, either due to the machine shutting down or the consumer crashing? If you acknowledge earlier you would lose that message, no?

[bensimian]

Hi, I am trying to set the consumer_timeout in /etc/rabbitmq/rabbitmq.conf but It does not seem to take affect. I set it to 30 seconds so that my QE team is able to test a particular scenario but when having the consumer service timeout for 40 seconds we are still able to ack without any error. To confirm after setting the value I stopped the server (stop_app) , reset with rabbitmqctl. To confirm that the changes of conf file take affect, we changed the username and password and can confirm our consumer services were unable to connect. Are we missing anything with regards to the consumer_timeout? thanks you

[sharshenov]

Some managed RabbitMQ hosting providers such as AWS AmazonMQ do not provide ability to set custom configuration. URI params configuration for consumer_timeout could be very convenient in this case.

[michaelklishin]

Those environments should probably bump the limit up to a few hours or disable it entirely. We have explained that it would be very difficult and potentially unsafe to make this limit configurable per-channel. My guess is that per-connection would entail similar risks.