Add a transfer_complete callback to storage connectors

[RodolfoSilva]

This will allow to dispatch an action when a transfer file is completed. Like create an Oban job to convert the file uploaded or notify any other service when the transfer of a file is completed.

[camatcode]

I'll be able to take a look on the weekend - watch this space. :D. Be sure to add a test when you can.

[RodolfoSilva]

I'll be able to take a look on the weekend - watch this space. :D. Be sure to add a test when you can.

Done. I’ve added Mimic to stub transfer_complete/3 and verify that it’s called. I don’t see another way to test this behavior without stubbing the function. If you have any ideas for testing it differently, I’d really appreciate it.

[camatcode]

@RodolfoSilva The code looks fine, I'm just wondering a little about the use-case.

By adding it as a behaviour callback, we're saying this is useful for every storage connector, but the File and S3 connectors just say :ok.

I could see it being useful for custom connectors, but I'm not sure why we would need it in the behaviour.

For a custom connector, if its useful, then they would just implement a transfer complete within their implementation of get_content and create_write_func.

If we do go forward with it, I think it would be better to have maybe def on_transfer_complete(provided_func) where one could pass in a function that gets called when transfer is complete.

Let me know what you think 🎅

[RodolfoSilva]

@camatcode I like this idea, but how can we implement it? The goal of this behavior is to allow me to take some action after the file finishes uploading to the server, instead of having to constantly poll or scan for changes.

[camatcode]

Have a look @RodolfoSilva , see if that will help your case.

Add to the storage_config a module and function with arity 3 (type, path, connector_state) that will be called with STOR and RETR completes.

  storage_config: %{
    # Optional: Handler for transfer completion notifications
    # Default: Logs "Transfer complete: <type> <path>"
    on_transfer_complete: {MyApp.TransferHandler, :handle_complete}
  }

Let me know what you think.

[camatcode]

saving myself a commit