Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .Rbuildignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,15 @@
^CONTRIBUTING\.md$
^NEWS\.md$
^app\.R$
^man/figures/class-editor\.png$
^man/figures/drag-select\.png$
^man/figures/gallery-view\.png$
^man/figures/image-card-states\.png$
^man/figures/interface-overview\.png$
^man/figures/logo\.ico$
^man/figures/logo\.svg$
^man/figures/logo_icon\.png$
^man/figures/measure-tool\.png$
^man/figures/relabel-workflow\.png$
^man/figures/sample-browser\.png$
^man/figures/settings-dialog\.png$
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: ClassiPyR
Title: A Shiny App for Manual Image Classification and Validation of IFCB Data
Version: 0.0.0.9000
Version: 0.1.0
Authors@R: c(
person("Anders", "Torstensson", email = "anders.torstensson@smhi.se", role = c("aut", "cre"),
comment = c("Swedish Meteorological and Hydrological Institute", ORCID = "0000-0002-8283-656X")),
Expand Down
68 changes: 31 additions & 37 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,13 @@
# ClassiPyR (development version)
# ClassiPyR 0.1.0

## Features
Initial release of ClassiPyR, a Shiny application for manual classification and validation of Imaging FlowCytobot (IFCB) plankton images.

### SQLite Database Backend
- Annotations are now stored in a local SQLite database (`annotations.sqlite`) by default
- Works out of the box with no Python dependency - only R packages (RSQLite, DBI) are needed
- MATLAB `.mat` file export is still available as an opt-in for ifcb-analysis compatibility
- Storage format configurable in Settings: "SQLite" (default), "MAT file", or "Both"
- Existing `.mat` annotations continue to work and can be loaded as before
- `import_mat_to_db()` utility for bulk migration of existing `.mat` files to SQLite
- Sample discovery scans both `.mat` files and the SQLite database
- When loading a sample, SQLite is checked first (faster), with `.mat` fallback
## Features

### Sample Management
- Load samples from ROI files with automatic year/month filtering
- Support for validation mode (existing classifications) and annotation mode (new samples)
- Resume previous annotations from saved MAT files
- Resume previous annotations from saved files
- Navigate between samples with previous/next/random buttons
- Filter samples by classification status (all/classified/annotated/unannotated)
- Samples with both manual annotations AND auto-classifications can switch between modes
Expand All @@ -30,15 +22,6 @@
- ✎✓ = Has both (can switch between modes)
- * = Unannotated

### File Index Cache
- Disk-based file index cache for faster app startup on subsequent launches
- Avoids expensive recursive directory scans when folder contents haven't changed
- Sync button in sidebar to manually refresh the file index
- Cache age indicator shows when folders were last scanned
- `rescan_file_index()` function for headless use (e.g. cron jobs)
- Cache stored in platform-appropriate config directory alongside settings
- Auto-sync option (enabled by default) to control whether app scans on startup

### Image Gallery
- Paginated image display (50/100/200/500 images per page)
- Images grouped by class on consecutive pages for efficient review
Expand All @@ -63,39 +46,50 @@
- Export class list as .mat or .txt
- Visual warnings for classes in classifications not in class2use list

### Annotation Storage
- SQLite database backend (default) — no Python dependency required
- Optional MATLAB `.mat` file export for ifcb-analysis compatibility (requires Python/scipy)
- Configurable storage format in Settings: "SQLite", "MAT file", or "Both"
- `import_mat_to_db()` and `export_db_to_mat()` for migration between formats
- Sample discovery scans both `.mat` files and the SQLite database
- When loading a sample, SQLite is checked first (faster), with `.mat` fallback
- Separate database folder setting (defaults to output folder)

### Output
- Save annotations to SQLite database (default, no Python needed)
- Optional: save annotations as MATLAB-compatible .mat files (using iRfcb, requires Python)
- Configurable storage format: SQLite only, MAT only, or both
- Save validation statistics as CSV (in `validation_statistics/` subfolder)
- Save validation statistics as CSV
- Organize output PNGs by class folder (for CNN training)
- Auto-save when navigating between samples
- Support for non-standard folder structures via direct ADC path resolution
- Graceful handling of empty (0-byte) ADC files

### File Index Cache
- Disk-based file index cache for faster app startup on subsequent launches
- Avoids expensive recursive directory scans when folder contents haven't changed
- Sync button in sidebar to manually refresh the file index
- Cache age indicator shows when folders were last scanned
- `rescan_file_index()` function for headless use (e.g. cron jobs)
- Auto-sync option (enabled by default) to control whether app scans on startup

### Settings & Persistence
- Configurable folder paths via settings modal
- Cross-platform web-based folder browser (shinyFiles)
- Settings persisted between sessions (`.classipyr_settings.json`)
- Settings persisted between sessions
- Class list file path remembered and auto-loaded on startup
- Annotator name tracking for statistics
- Cache invalidation when folder paths change in settings

### User Interface
- Clean, modern interface using bslib (Flatly theme)
- Mode indicator showing current sample and progress/accuracy
- Validation Statistics tab shows appropriate content based on mode
- Switch between annotation/validation modes for dual-mode samples

## Pre-releases

- **v0.1.0-beta.2** (2026-02-04): File index cache, cross-platform folder browser, annotation mode sorting, and notification improvements.
- **v0.1.0-beta.1** (2026-01-29): First beta version.

## Technical Notes
- SQLite is the default annotation storage - works out of the box with RSQLite (no external dependencies)
- Python with scipy is optional - only needed for MAT file export (ifcb-analysis compatibility)
- SQLite is the default annotation storage works out of the box with RSQLite
- Python with scipy is optional only needed for MAT file export
- Uses iRfcb package for IFCB data handling
- Session cache preserves work when switching samples
- File index cache reduces startup time by avoiding redundant folder scans
- Security: Input validation, XSS prevention, path traversal protection

## Development
This application was developed through human-AI collaboration:
- **Anders Torstensson**: Project vision, requirements, testing, and guidance
- **Claude Code (Anthropic)**: Implementation, code generation, and iterative refinement
- Input validation, XSS prevention, and path traversal protection
4 changes: 2 additions & 2 deletions inst/CITATION
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ bibentry(
comment = c(ORCID = "0000-0002-8283-656X"))
),
year = "2026",
note = "R package version 0.0.0.9000",
note = "R package version 0.1.0",
url = "https://doi.org/10.5281/zenodo.18414999",
textVersion = paste(
"Torstensson, A. (2026).",
"ClassiPyR: A Shiny Application for Manual Image Classification and Validation of IFCB Data.",
"R package version 0.0.0.9000.",
"R package version 0.1.0.",
"https://doi.org/10.5281/zenodo.18414999"
)
)
8 changes: 7 additions & 1 deletion inst/app/server.R
Original file line number Diff line number Diff line change
Expand Up @@ -2285,7 +2285,13 @@ server <- function(input, output, session) {
update_current_sample_status(rv$current_sample)
}

showNotification(paste("Saved to", config$output_folder), type = "message")
save_msg <- switch(save_fmt,
sqlite = paste("Saved to database in", config$db_folder),
mat = paste("Saved to", config$output_folder),
both = paste("Saved to database and", config$output_folder),
paste("Saved to", config$output_folder)
)
showNotification(save_msg, type = "message")

}, error = function(e) {
showNotification(paste("Error saving:", e$message), type = "error")
Expand Down
Binary file modified man/figures/drag-select.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified man/figures/gallery-view.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified man/figures/image-card-states.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified man/figures/interface-overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified man/figures/logo_icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
68 changes: 50 additions & 18 deletions tests/testthat/test-database.R
Original file line number Diff line number Diff line change
Expand Up @@ -385,32 +385,52 @@ test_that("import_mat_to_db migrates data correctly", {
skip_if_not(reticulate::py_available(), "Python not available")
skip_if_not(reticulate::py_module_available("scipy"), "scipy not available")

sample_name <- "D20220522T000439_IFCB134"

# Check if there's a test annotation mat file
output_test <- testthat::test_path("test_data", "manual")
test_mat <- file.path(output_test, paste0(sample_name, ".mat"))
skip_if_not(file.exists(test_mat), "No test MAT annotation file for migration test")
sample_name <- "D20230101T120000_IFCB134"
class2use <- c("unclassified", "Diatom", "Ciliate", "Dinoflagellate")

db_dir <- tempfile("db_")
# Create a MAT file via export_db_to_mat so the test is self-contained
db_dir <- tempfile("db_export_")
dir.create(db_dir)
db_path <- get_db_path(db_dir)

result <- import_mat_to_db(test_mat, db_path, sample_name, "migrated")
classifications <- data.frame(
file_name = sprintf("%s_%05d.png", sample_name, 1:4),
class_name = c("Diatom", "Ciliate", "Diatom", "Dinoflagellate"),
stringsAsFactors = FALSE
)
save_annotations_db(db_path, sample_name, classifications, class2use, "exporter")

mat_dir <- tempfile("mat_")
dir.create(mat_dir)
result <- export_db_to_mat(db_path, sample_name, mat_dir)
expect_true(result)

test_mat <- file.path(mat_dir, paste0(sample_name, ".mat"))
expect_true(file.exists(test_mat))

# Now import the MAT file into a fresh database
db_dir2 <- tempfile("db_import_")
dir.create(db_dir2)
db_path2 <- get_db_path(db_dir2)

result <- import_mat_to_db(test_mat, db_path2, sample_name, "migrated")
expect_true(result)

# Verify data was imported
samples <- list_annotated_samples_db(db_path)
samples <- list_annotated_samples_db(db_path2)
expect_true(sample_name %in% samples)

unlink(db_dir, recursive = TRUE)
unlink(c(db_dir, db_dir2, mat_dir), recursive = TRUE)
})

test_that("import_mat_to_db returns FALSE for missing file", {
result <- import_mat_to_db(
"/nonexistent/file.mat",
tempfile(fileext = ".sqlite"),
"sample", "test"
expect_warning(
result <- import_mat_to_db(
"/nonexistent/file.mat",
tempfile(fileext = ".sqlite"),
"sample", "test"
),
"MAT file not found"
)
expect_false(result)
})
Expand Down Expand Up @@ -464,14 +484,20 @@ test_that("export_db_to_mat returns FALSE for missing sample", {
stringsAsFactors = FALSE),
c("unclassified", "Diatom"), "test")

result <- export_db_to_mat(db_path, "nonexistent_sample", db_dir)
expect_warning(
result <- export_db_to_mat(db_path, "nonexistent_sample", db_dir),
"No annotations found for sample"
)
expect_false(result)

unlink(db_dir, recursive = TRUE)
})

test_that("export_db_to_mat returns FALSE for non-existent database", {
result <- export_db_to_mat("/nonexistent/db.sqlite", "sample", tempdir())
expect_warning(
result <- export_db_to_mat("/nonexistent/db.sqlite", "sample", tempdir()),
"Database not found"
)
expect_false(result)
})

Expand Down Expand Up @@ -740,7 +766,10 @@ test_that("export_db_to_png returns FALSE for missing sample", {
"D20220522T000439_IFCB134.roi")
skip_if_not(file.exists(roi_path), "Test ROI file not found")

result <- export_db_to_png(db_path, "nonexistent_sample", roi_path, tempdir())
expect_warning(
result <- export_db_to_png(db_path, "nonexistent_sample", roi_path, tempdir()),
"No annotations found for sample"
)
expect_false(result)

unlink(db_dir, recursive = TRUE)
Expand All @@ -757,7 +786,10 @@ test_that("export_db_to_png returns FALSE for missing ROI file", {
stringsAsFactors = FALSE),
c("unclassified", "Diatom"), "test")

result <- export_db_to_png(db_path, "sample_A", "/nonexistent/file.roi", tempdir())
expect_warning(
result <- export_db_to_png(db_path, "sample_A", "/nonexistent/file.roi", tempdir()),
"ROI file not found"
)
expect_false(result)

unlink(db_dir, recursive = TRUE)
Expand Down
Loading