feat: separate SQLite database path from output folder

DESCRIPTION

@@ -24,7 +24,9 @@ Imports:
     dplyr,
     DT,
     jsonlite,
-    reticulate
+    reticulate,
+    DBI,
+    RSQLite
 Suggests:
     testthat (>= 3.0.0),
     covr,

NAMESPACE

@@ -3,29 +3,50 @@
 export(copy_images_to_class_folders)
 export(create_empty_changes_log)
 export(create_new_classifications)
+export(export_all_db_to_mat)
+export(export_all_db_to_png)
+export(export_db_to_mat)
+export(export_db_to_png)
 export(filter_to_extracted)
 export(get_config_dir)
+export(get_db_path)
+export(get_default_db_dir)
 export(get_file_index_path)
 export(get_sample_paths)
 export(get_settings_path)
+export(import_all_mat_to_db)
+export(import_mat_to_db)
 export(init_python_env)
 export(is_valid_sample_name)
+export(list_annotated_samples_db)
+export(load_annotations_db)
 export(load_class_list)
 export(load_file_index)
 export(load_from_classifier_mat)
 export(load_from_csv)
+export(load_from_db)
 export(load_from_mat)
 export(read_roi_dimensions)
 export(rescan_file_index)
 export(run_app)
 export(sanitize_string)
+export(save_annotations_db)
 export(save_file_index)
 export(save_sample_annotations)
 export(save_validation_statistics)
+export(update_annotator)
+importFrom(DBI,dbConnect)
+importFrom(DBI,dbDisconnect)
+importFrom(DBI,dbExecute)
+importFrom(DBI,dbGetQuery)
+importFrom(DBI,dbWriteTable)
 importFrom(DT,renderDT)
+importFrom(RSQLite,SQLite)
 importFrom(bslib,bs_theme)
 importFrom(dplyr,filter)
 importFrom(iRfcb,ifcb_annotate_samples)
+importFrom(iRfcb,ifcb_create_manual_file)
+importFrom(iRfcb,ifcb_extract_pngs)
 importFrom(iRfcb,ifcb_get_mat_variable)
 importFrom(jsonlite,fromJSON)
 importFrom(reticulate,py_available)

NEWS.md

@@ -2,6 +2,16 @@
 
 ## Features
 
+### SQLite Database Backend
+- Annotations are now stored in a local SQLite database (`annotations.sqlite`) by default
+- Works out of the box with no Python dependency - only R packages (RSQLite, DBI) are needed
+- MATLAB `.mat` file export is still available as an opt-in for ifcb-analysis compatibility
+- Storage format configurable in Settings: "SQLite" (default), "MAT file", or "Both"
+- Existing `.mat` annotations continue to work and can be loaded as before
+- `import_mat_to_db()` utility for bulk migration of existing `.mat` files to SQLite
+- Sample discovery scans both `.mat` files and the SQLite database
+- When loading a sample, SQLite is checked first (faster), with `.mat` fallback
+
 ### Sample Management
 - Load samples from ROI files with automatic year/month filtering
 - Support for validation mode (existing classifications) and annotation mode (new samples)
@@ -54,7 +64,9 @@
 - Visual warnings for classes in classifications not in class2use list
 
 ### Output
-- Save annotations as MATLAB-compatible .mat files (using iRfcb)
+- Save annotations to SQLite database (default, no Python needed)
+- Optional: save annotations as MATLAB-compatible .mat files (using iRfcb, requires Python)
+- Configurable storage format: SQLite only, MAT only, or both
 - Save validation statistics as CSV (in `validation_statistics/` subfolder)
 - Organize output PNGs by class folder (for CNN training)
 - Auto-save when navigating between samples
@@ -76,7 +88,8 @@
 - Switch between annotation/validation modes for dual-mode samples
 
 ## Technical Notes
-- Requires Python with scipy for MAT file writing (optional - only for ifcb-analysis compatibility)
+- SQLite is the default annotation storage - works out of the box with RSQLite (no external dependencies)
+- Python with scipy is optional - only needed for MAT file export (ifcb-analysis compatibility)
 - Uses iRfcb package for IFCB data handling
 - Session cache preserves work when switching samples
 - File index cache reduces startup time by avoiding redundant folder scans