Conversation
Rename functions and methods within the gcsService to improve clarity and align with Go conventions.
Moved the gcs artifact service implementation to a separate gcs package, and the test suite to artifactservice_test.
The mockable_gcs interface is only relevant for testing and its use can be limited to the gcs package.
…Add parallel execution on delete all versions.
|
|
||
| nextVersion := int64(1) | ||
|
|
||
| // TODO race condition, could use mutex but it's a remote resource so the issue would still occurs |
There was a problem hiding this comment.
@dpasiukevich comment
Curious, how python/java handled this for their GCS implementation?
Do they have it as a known documented limitation or a TODO?
As an idea: version is an int64. What about if we fit a timestamp in there? e.g. https://pkg.go.dev/time#Time.UnixNano?
This gives us:
Monotonously increasing.
Nanosecond granularity -> minimum chance of collisions.
We'd have a capacity till 2262. After that our descendants I'm sure will solve it.
Author
@baptmont baptmont comment
The adk documentation doesn't address this issue/limitation.
The idea of using a timestamp works, but we would differ from the other adk implementations.
A system using both adk python and adk go agents would have conflicting behaviors.
| } | ||
| // TODO agent can create files with multiple segments for example file a/b.txt | ||
| // This a/b.txt file will show as b.txt when listed and trying to load it will fail. | ||
| filename := segments[len(segments)-2] // appName/userId/sessionId/filename/version or appName/userId/user/filename/version |
There was a problem hiding this comment.
@dpasiukevich comment
Overall question -- can we use GCS version features (asking out of curiosity) instead of handling versions ourselves?
https://pkg.go.dev/cloud.google.com/go/storage#Query
http://cloud/storage/docs/using-versioned-objects#storage-list-object-versions-go
@dpasiukevich comment
following up: One major issue it may solve -- fetching the last version of the artifact in one call if it's possible. O(N) -> O(1) for artifact creation request.
Is it possible to fetch latest artifact in one call?
Author
@baptmont comment
When using the GCS versioning the "noncurrent versions only appear in requests that explicitly call for them to be included" as such only the latest version would be retrieved, making it O(1).
This is something that should be explored for all versions of adk, since for now we want for a system using both adk python and adk go agents to have the same behavior.
* Add GCSArtifactService. * Fix dependecies diff * Add io.Close err validation * Standardize function and method naming for gcsService. Rename functions and methods within the gcsService to improve clarity and align with Go conventions. * Add artifactservice/gcs package, add request detailed validation Moved the gcs artifact service implementation to a separate gcs package, and the test suite to artifactservice_test. * Make mockable_gcs interfaces unexported The mockable_gcs interface is only relevant for testing and its use can be limited to the gcs package. * Add ValidatorTestCases interface to reduce duplicated code on test execution * Move mockable_storage to test file * Improve gcs service error handling, add support for parts with text. Add parallel execution on delete all versions. * Add err validation to setAttrSelection * Change test suite to include NewPartFromText example * Add Client.Options to gcs artifact service constructor for configuring the client * Add err context for request validation in artifact service
2 participants
Moved from #91
Changed to track forked gcs branch instead of main.