-
Notifications
You must be signed in to change notification settings - Fork 703
chore(deps): update object_store requirement from 0.12.3 to 0.13.0 in /integrations/object_store #7118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
chore(deps): update object_store requirement from 0.12.3 to 0.13.0 in /integrations/object_store #7118
Conversation
Updates the requirements on [object_store](https://github.com/apache/arrow-rs-object-store) to permit the latest version. - [Changelog](https://github.com/apache/arrow-rs-object-store/blob/main/CHANGELOG.md) - [Commits](apache/arrow-rs-object-store@v0.12.3...v0.13.0) --- updated-dependencies: - dependency-name: object_store dependency-version: 0.13.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
9cfbbe3 to
b5c695a
Compare
…bject_store-0.13.0
2c6b1c4 to
fbf0c71
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| Ok(PutResult { e_tag, version }) | ||
| } | ||
|
|
||
| async fn put_multipart( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is now internally implemented by ObjectStoreExt.
| }) | ||
| } | ||
|
|
||
| async fn delete(&self, location: &Path) -> object_store::Result<()> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replaced with delete_stream.
| from: &Path, | ||
| to: &Path, | ||
| // TODO: if we need to support rename options in the future | ||
| _options: RenameOptions, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems OpenDAL doesn't support different rename mode now.
| async fn copy_if_not_exists(&self, from: &Path, to: &Path) -> object_store::Result<()> { | ||
| self.copy_request(from, to, true).await | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved to ObjectStoreExt.
| locations | ||
| .then(move |location| { | ||
| let client = client.clone(); | ||
| async move { | ||
| let location = location?; | ||
| let decoded_location = percent_decode_path(location.as_ref()); | ||
| client | ||
| .delete(&decoded_location) | ||
| .into_send() | ||
| .await | ||
| .map_err(|err| format_object_store_error(err, location.as_ref()))?; | ||
| Ok(location) | ||
| } | ||
| }) | ||
| .boxed() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
object_store uses multiple batching mechanisms:
fn delete_stream(
&self,
locations: BoxStream<'static, Result<Path>>,
) -> BoxStream<'static, Result<Path>> {
let client = Arc::clone(&self.client);
locations
.try_chunks(1_000)
.map(move |locations| {
let client = Arc::clone(&client);
async move {
// Early return the error. We ignore the paths that have already been
// collected into the chunk.
let locations = locations.map_err(|e| e.1)?;
client
.bulk_delete_request(locations)
.await
.map(futures::stream::iter)
}
})
.buffered(20)
.try_flatten()
.boxed()
}40d4ad0 to
11e578c
Compare
|
DataFusion example failed to compile because datafusion is yet to upgrade to object_store 0.13. |
Updates the requirements on object_store to permit the latest version.
Changelog
Sourced from object_store's changelog.
... (truncated)
Commits
eedbf3dPrepare for 0.13.0 release (#581)44e258cAdd A "upgrading to 0.13.0" guide in docs (#584)24e5fbcUpdate release schedule on README (#583)378bfe7correctly expose HttpError through RetryError::source (#580)fa40170Pathimprovements (#546)493a5d0fix: Add more details on what isn't implemented (#575)7b09e4eDocumentation for backend support of bulk delete (#571)65ad88bRemove dev dependency on openssl (#577)de4a30eAllow explicitly specifying GCS base URL (#567)92b1378refactor!: movedeletetoObjectStoreExt(#549)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot mergewill merge this PR after your CI passes on it@dependabot squash and mergewill squash and merge this PR after your CI passes on it@dependabot cancel mergewill cancel a previously requested merge and block automerging@dependabot reopenwill reopen this PR if it is closed@dependabot closewill close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually@dependabot show <dependency name> ignore conditionswill show all of the ignore conditions of the specified dependency@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)