-
Notifications
You must be signed in to change notification settings - Fork 11
Description
One requested feature that we cannot currently provide is the total number of messages in a stream (or "user entries"). The reason that simply taking the first offset in a stream and subtracting this from the last offset is that there are non-user offsets used for internal purposes such as offset and writer tracking. To accurately calculate the number of user entries we'd need to know the number of non-user entries in a stream.
The chunk header has 4 bytes of reserved data which could be used for a "non-user offset" counter. If we had this information we could calculate the number of messages in a stream by doing: (LastOffset - FirstOffset) - (LastNonUserCount - FirstNonUserCount)
- which only requires us to read the chunk header of the first and last chunks of the stream.
Potential issues:
- for existing streams the count would not be correct until we'd flushed out all segments that do not have this information
- The non-user entry counter is limited to 32 bits, we'd have to consider if it is likely to ever reach this value and what to do if it does.