Skip to content

MAST: Enable cloud dataset by default#3534

Open
snbianco wants to merge 3 commits intoastropy:mainfrom
snbianco:enable-cloud-dataset
Open

MAST: Enable cloud dataset by default#3534
snbianco wants to merge 3 commits intoastropy:mainfrom
snbianco:enable-cloud-dataset

Conversation

@snbianco
Copy link
Contributor

@snbianco snbianco commented Feb 16, 2026

Current Behavior

  • Access to cloud dataset is not enabled by default.
  • Users have to explicitly run the enable_cloud_dataset function before they use any cloud-related functions.

Proposed Behavior

  • The cloud dataset is enabled by default, so users do not have to explicitly call enable_cloud_dataset.
  • To disable the cloud dataset, users can alter a config value or call the disable_cloud_dataset function.

Advantages

  • Faster downloads (in some cases)
  • Removes a hidden step that users may not even know is available to them.
  • More in line with the direction that MAST is heading re. Roman and other large datasets
  • More convenient for users in cloud platforms
  • Fewer download failures
  • Reduce operational load on MAST servers

What Changes for Users?

  • No longer have to call enable_cloud_dataset explicitly
  • Cloud downloads will be preferred automatically.
  • Files in the cloud will be pulled from there instead of MAST servers.
  • This is just default behavior and can be changed.
  • Cloud dataset will NOT be enabled if the user does not have the prerequisite packages (boto3, botocore)
  • Users can change whether the cloud dataset is automatically enabled with a configuration variable

Other Things to Note

  • Cloud access is instantiated lazily only when a relevant method is called (download_file, download_products, get_cloud_uri(s))
  • Fixes a bug introduced in Get cloud missions dynamically and fix cloud download workflow #3488 where botocore, an optional package, is imported without a guard. Now, both boto3 and botocore are imported with try blocks and ImportError is properly handled.
  • Observations internally keeps track of whether the cloud dataset is explicitly enabled or disabled. If not explicitly enabled by the user (they call enable_cloud_dataset), warning messages are not logged when a product cannot be found in the cloud and the download falls back to on-prem. This is to avoid clogging up the console and introducing warning messages where there were none before.

Close #3546

@codecov
Copy link

codecov bot commented Feb 16, 2026

Codecov Report

❌ Patch coverage is 83.92857% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.71%. Comparing base (de342ea) to head (62832b0).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
astroquery/mast/cloud.py 69.23% 4 Missing ⚠️
astroquery/exceptions.py 0.00% 3 Missing ⚠️
astroquery/mast/observations.py 94.87% 2 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3534   +/-   ##
=======================================
  Coverage   72.71%   72.71%           
=======================================
  Files         219      219           
  Lines       20473    20528   +55     
=======================================
+ Hits        14886    14927   +41     
- Misses       5587     5601   +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@snbianco snbianco marked this pull request as ready for review February 16, 2026 23:25
@snbianco snbianco requested a review from bsipocz February 16, 2026 23:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mast import crashes when botocore optional dependency is not installed

2 participants