Enable (simple?) integration of external devices by means of widgets/GUI elements?

[beniroquai]

Hey, I don't know if this is the right place to ask for this "feature", but I was wondering if somebody already demanded for an easy way to integrate external hardware control using widgets/GUI snippets. In this Youtube Video by Nico, he mentioned that the Laser engine by the Ries group has a "simple" way to do that.

I was thinking if it's worth thinking about having external devices e.g. encapsulated using Labthings (@rwb27) exposed through a small widget that enables simple device control. This could for example be a pump, a small pipetting robot, or whatever is interfaced through an external device adapter. The GUI definition could be part of Labthings if I'm not mistaken (e.g. HTML)?

[nicost]

The Ries's group effort (by Joran Deschamps) is the EMU plugin: https://micro-manager.org/wiki/EMU. It seems taht Labthings is in Python, which could be used through pycromanager.

[beniroquai]

Perfect! I just found their publication too.
May I ask you if you had a look at Richard Bowman et al's Openflexure Server? It heavily makes use of REST API. This is a very nice solution to integrate IoT-based devices (e.g. Rapi, ESP32). This in combination with labthings makes integration into GUIs using HTML snippets very easy.
Do you think there is room for similar interfaces in the new MMCore? (Does this fit here, or it rather a new issue?)

[rwb27]

I've spent a while thinking about labthings/micromanager interoperability; I think some amount of intercompatibility would be simple, but some of it would take some C++ that I've yet to find time to write. While I'm sure there would be some benefit in making web communication part of the core, my current understanding is that LabThings support could be added through device adapters, albeit in a slightly complicated way if we want to be fully generic.

Currently, pycromanager would allow a LabThings server to expose functionality from hardware controlled using MMCore (and the associated huge library of device adapters). That would require someone to glue together LabThings and pycromanager, which is a manageable Python project (not that I've made any attempt at doing it yet). That would allow HTML/JavaScript GUIs, and control from whatever language you like, using a REST API.

The other way round is harder; to control LabThings devices from MMCore (and thus to make those devices available to the huge library of tools and protocols that rely on it) we'd need to write a hub device adapter that handles translating MMCore API calls into HTTP requests, and ideally also handles mDNS-sd to locate LabThings devices on the network. We'd then need a set of device adapters that can be created by the hub, to represent each labthing that it finds with appropriate MMCore class(es).

Either way, we would need to define appropriate "semantic types" for LabThings, so that the web services can declare themselves to be particular MMCore classes. Getting this right will take a bit of thought, and as and when it becomes something we're able to work on, I will be sure to bring it to the attention of the MMCore team.

If we do get it right, it potentially opens a lot of doors. The REST API approach would allow proprietary hardware vendors to expose control via their own closed-source software. While of course I'm in favour of everything open, it can't be denied that many manufacturers provide important functionality in their own software, and aren't happy to release that openly. This leads to two big problems - first, any open source driver has to re-implement (or reverse engineer) that functionality, which is effort that could be spent elsewhere. Secondly, you usually have to shut down the manufacturer's software and re-initialise the device from within MMCore, which can waste time repeating set-up tasks, and require more functionality in the device adapter.

One of the big benefits of using a REST interface is that the manufacturer can make their hardware and software controllable, at the same time as keeping it running. That means you can use the supplied software to monitor your camera and tweak the settings, but also use it in automated protocols via micromanager - without having to shut it down and restart in between.

If LabThings can provide those benefits to MMCore, that would be brilliant - not to mention massively expanding the pool of people who might make use of the WoT approach to lab automation.

[beniroquai]

I copy that! Having access to the large variety of small IoT-based devices such as ESP32 or alike to control all kinds of triggers and open-source microscopy hardware would be very very interesting. In best case, the installation of additional hardware works similar to Fijis way of dealing with plugins; i.e. people would store a "thing" definition on a github repo, that gets loaded by the "hub" or micromanager itself.
Let's see if we find some money for this. I'm not good enough in programming, but would be very keen on using it ;-)