[NEW] Need blockable notification at end of multi/exec. #2770
Description
The problem/use-case that the feature addresses
A module can determine that a multi/exec has started by checking for the multi/exec flag on each mutative keyspace notification. However, there is no direct way to determine that a multi/exec has completed.
The search index update machinery is multi/exec aware. When mutations for a multi/exec are received, they are queued until the end of the multi/exec and then the index is updated atomically, preventing any query operation from observing the index in an illegal state (illegal in the sense of seeing a partially updated index).
Today, the search module infers the end of a multi/exec in a lazy fashion, i.e., when the next non-mutative keyspace notification is received OR the next search command is received it knows that a previous multi/exec has completed and the index update can be scheduled.
This behavior has negative consequences for two reasons.
- It fails to optimally use available resources. In particular, the period of time between the actual end of the multi/exec and the detected end of the multi/exec (i.e., the start of the next search operation) cannot be used to update the index.
- The update latency of the multi/exec mutations is transferred to a later, and likely unrelated, command. For example, if an application submits a multi/exec then the next, possibly totally unrelated, query command will see the increased latency of the update operation.
Description of the feature
Search wants to receive explicit notification of the end of a multi/exec operation. This removes the need for search to lazy-detect the end. This solves negative consequence number 1 (see above) in that it allows the scheduling of the mutation to begin as soon as possible.
Allowing the client to be blocked upon receipt of this notification would solve the other negative consequence. The latency of the index update of the multi/exec operation would properly be incurred by the client executing that command, not the next -- possibly unrelated -- search operation.
It is proposed that a new keyspace or server event notification category be created that is fired at the end of a multi/exec. It should be possible to block the client out of that notification, such that when the client is unblocked, it proceeds to the next command.
Alternatives you've considered
What we have implemented today.
Additional information