Feat/filewatcher go example #284
Conversation
| ## Requirements | ||
|
|
||
| * Go **1.22+** | ||
| * macOS (uses `fsnotify`, compatible with FSEvents) |
There was a problem hiding this comment.
I think fsnotify according to their readme is cross platform right? Or atleast can work on linux&mac
| * PDF-only file filtering | ||
| * Detects file lifecycle events: | ||
|
|
||
| * `FILE_ADDED` |
There was a problem hiding this comment.
Nice work outlining this
I'm curious how file renames are represented. As a FILE_UPDATED? or as a FILE_REMOVED+FILE_ADDED
There was a problem hiding this comment.
We can probably ignore the built binary as well
| * Directory not found → Program exits with a descriptive error | ||
| * API failure → Logged; watcher continues running | ||
| * Permission issues → Logged as warnings | ||
| * Non-PDF files → Ignored |
There was a problem hiding this comment.
More like it ignores files not suffixed with .pdf ... they may be valid pdf files but you took the suffix detection approach (which is fine for this example tbh)
Alternatively we could have used a detection approach where we try and check the first 5 bytes are %PDF- and that would potentially catch more valid pdfs. However, because we are leaving actual PDF parsing for some upstream component I'd say the suffix approach is fine but we should properly document
| cfg := &Config{} | ||
|
|
||
| flag.StringVar(&cfg.Dir, "dir", "", "Directory to watch") | ||
| flag.StringVar(&cfg.APIURL, "api-url", "", "API endpoint") |
There was a problem hiding this comment.
HTTP API is not a bad choice tbh but given that we expect the watcher to run on the same system as the future components (atleast for this example), how about a unix domain socket path?
There was a problem hiding this comment.
I think this is deprecated now given the structure below
|
|
||
| --- | ||
|
|
||
| ## Project Structure |
There was a problem hiding this comment.
The file watcher is one piece out of the project... perhaps the entire project should be named something like pdfsearch and the filewatcher is one binary under pdfsearch that pushes events to the indexer
There was a problem hiding this comment.
Either that or filewatcher, indexer and searcher are things you spin up at once as such
// instantiate some connecting channel between watcher and indexer
go fileWatcher.Watch(connChann)
go indexer.Index(connChann)
go searcher.InitAPI()2 participants
Adds a filewatcher CLI that watches a directory for PDF file changes and posts normalized file events to an HTTP API. Includes event debouncing for macOS, structured logging, and a clean, extensible architecture for future processing pipelines to examples directory for go lang