Fix GCS not-found detection to unwrap ErrObjectNotExist (#358)#359
Merged
Conversation
cloud.google.com/go/storage v1.60.0 (bumped from v1.30.1 in a452c77) now wraps storage.ErrObjectNotExist via formatObjectErr instead of returning the bare sentinel. The direct equality checks (err == storage.ErrObjectNotExist) in GetChunk and HasChunk therefore no longer match, so expected cache misses are misclassified as fatal errors. This made "desync make" abort with "storage: object doesn't exist: googleapi: Error 404: No such object" when storing a new chunk. Use errors.Is, which unwraps, in all three locations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
desync makefails when it encounters a chunk that doesn't yet exist in a Google Cloud Storage bucket, aborting with:The missing object is expected — it's a new chunk awaiting upload — and should be treated as a cache miss, not a fatal error.
Root cause
gcs.godetected "object not found" using direct equality (err == storage.ErrObjectNotExist) in three places (GetChunkafterNewReaderand afterio.ReadAll, andHasChunkafterAttrs).The dependency bump in a452c77 (
cloud.google.com/go/storagev1.30.1 → v1.60.0) changed the library to wrap the sentinel rather than return it bare.formatObjectErrnow does:so the returned error is no longer
== storage.ErrObjectNotExist. BothAttrsandNewReaderroute throughformatObjectErr, producing exactly the"storage: object doesn't exist: googleapi: Error 404: No such object"message above.In
ChunkStorage.StoreChunk, the dedup checkif hasChunk, err := s.ws.HasChunk(...); err != nil || hasChunkthen propagates this misclassified error and aborts the wholemake. The same flaw also turns legitimate read misses inGetChunkinto hard errors instead ofChunkMissing, breaking cache/failover self-repair. (S3 is unaffected; it uses status-code-based detection.)Fix
Use
errors.Is, which unwraps the chain, in all three locations. No import change is needed —github.com/pkg/errors(already imported) passesIsstraight through to the stdlib.Fixes #358