-
Notifications
You must be signed in to change notification settings - Fork 822
Description
Describe the bug
A clear and concise description of what the bug is.
a lot of unexpected EOF error when reading from partition cause here.
Lines 1421 to 1433 in 2af3101
default: | |
var kafkaError Error | |
if errors.As(err, &kafkaError) { | |
r.sendError(ctx, err) | |
} else { | |
r.withErrorLogger(func(log Logger) { | |
log.Printf("the kafka reader got an unknown error reading partition %d of %s at offset %d: %s", r.partition, r.topic, toHumanOffset(offset), err) | |
}) | |
r.stats.errors.observe(1) | |
conn.Close() | |
break readLoop | |
} | |
} |
I was try to log the detail found that when
highWaterMark + 1 == offset
and remain is 0. newMessageSetReader will return io.EOF(because remain is 0, read head first offset with io.EOF error)Lines 859 to 866 in 2af3101
var msgs *messageSetReader | |
if err == nil { | |
if highWaterMark == offset { | |
msgs = &messageSetReader{empty: true} | |
} else { | |
msgs, err = newMessageSetReader(&c.rbuf, remain) | |
} | |
} |
It's cause a lot of error log and conn.Close a lot of recreate connection and TIME_WAIT.
Kafka Version
- What version(s) of Kafka are you testing against?
v2.2.0- What version of kafka-go are you using?
v0.4.47
To Reproduce
Resources to reproduce the behavior:
---
# docker-compose.yaml
#
# Adding a docker-compose file will help the maintainers setup the environment
# to reproduce the issue.
#
# If one the docker-compose files available in the repository may be used,
# mentioning it is also a useful alternative.
...
package main
import (
"github.com/segmentio/kafka-go"
)
func main() {
// Adding a fully reproducible example will help maintainers provide
// assistance to debug the issues.
...
}
Expected Behavior
A clear and concise description of what you expected to happen.
Don't return error when highWaterMark + 1 == offset
and remain is 0.
Shall we just add remain == 0
condition ?
Line 861 in ebca72e
if highWaterMark == offset { |
if highWaterMark == offset || remain == 0 {
IMO, when remain is 0 return empty is reasonable.
Observed Behavior
A clear and concise description of the behavior you observed.
Often times, pasting the logging output from a kafka.Reader or kafka.Writer will
provide useful details to help maintainers investigate the issue and provide a
fix. If possible, providing stack traces or CPU/memory profiles may also contain
valuable information to understand the conditions that triggered the issue.
Additional Context
Add any other context about the problem here.