KNB COG Hosting

[rellimylime]

Figure out how to submit COGs to KNB and how access will work.

@FlukeAndFeather

[hylaea-miller]

How to Submit COGs to KNB

For datasets >50 GB, their platform indicates to contact KNB Help before submission via email: knb-help@nceas.ucsb.edu
What to request:

• Guidance for large file uploads
• If appropriate, clarification on data access limitations for COGs

Required information for data upload:

• ORCID
• Title - Include topic, geographic location, dates, and scale. Write out all abbreviations.
• Dates - Collection or creation date in YYYY, YYYY-MM-DD, or YYYY-MM-DD HH:MM:SS format
• Description - Short, comprehensive geographic description of sampling sites/locations
• Bounding Box Coordinates - Two lat/lng pairs for geographic extent, or one pair for single point
• High-quality metadata - Comprehensive documentation
• Digital Object Identifier (DOI) - Can be added after initial submission

Data Access

Initial research confirmed that KNB does not support HTTP byte‑range requests, which are required for efficient COG subsetting.
Alternative Solutions Proposed by Carlo

Option 1: Tile‑Based Approach (Using KNB)

• Pre‑process the rasters into smaller regional tiles.
• Upload each tile to KNB as a separate record.
• Users download only the tiles that intersect their area of interest.

Discussion Points

• How much additional processing time will tiling require?
• What is the optimal tile size for balancing usability and performance?
• Can we pre‑define regional subsets based on common use cases (e.g., counties)?
• How will tiling influence STAC catalog complexity and the overall user experience?
• Deliverables may need to be adjusted.

Option 2: Alternative Databases that Support COG Subsetting

• Source Cooperative – Nonprofit and relatively new platform. Free for public datasets during beta, but may introduce fees based on data volume once beta ends. For long‑term or automated data submission, contact: hello@source.coop
• Hugging Face – It doesn’t appear to support COG‑specific functionality. Operates similarly to GitHub for hosting files.
• AWS – Supports COG subsetting via HTTP byte‑range requests. Requires selecting an appropriate service configuration. Costs vary with usage, and access credentials may be required.

Discussion Points

• Budget constraints
• Applicability to our use case
• Ease of navigation and learning curve