Customer coordinate system

[Mikrodoktor]

We need to define, how experiences X/Y/Z movements when using the joystick or other ways of moving the sample and where we put the origin. This is not to be mixed up with how we internally define our origin and axes movement directions. This can be different.

251006: updated diagramm @gokugiant @ethanjli @beniroquai PLease check and change, ammend in the document. This is a proposal.
see
https://drive.google.com/file/d/1H7C-IWbQwBwSe1n6OUeoYS8VMGkopR8t/view?usp=drive_link

[beniroquai]

@Mikrodoktor I cannot access the file unfortunately. Would you mind sharing it through this repo instead? We can create a new folder DOCS or something :)

[Mikrodoktor]

Hab einen Google Drive Link geteilt. Geht es jetzt?

[beniroquai]

Nice! First question: Is there a standard / DIN / ISO? Also, we have the additional input of the joystick. I think we had this issue where we defined something internally.

openUC2/ImSwitch-ReactAPP#19

The wellplates (watched from above) have the same coordinate system as matplotlib / python. left upper corner is 0. According to Jens, we have a right handed system. Hence z is going down

Things we have to keep in mind is - we move the sample vs. moving the microscope. We look from below vs. from above.

Who is in cahrge of this issue?

[Mikrodoktor]

Step by step.

First step:

With the drawing I want us to align what the user experience is. If a joystick is used.
My experience:

• I am experiencing a microscope as if "I move" the objective. So I am expecting my inspection point to move into the direction my joystick moves (X/Y). Since the objective is stationary and the stage moves, it needs to move into opposite direction.
• Same holds true for Z. Moving up means I move the objetive up to inspect more elevated parts of the sample. Stage needs to move down.
  @ethanjli @gokugiant @beniroquai
  Is this what you expect when working with a microscope and using a joystick feature?

Second step:

• define which axes is which (X/Y/Z)
• I would go with what I have drawn in the document.
• I have checked with CHAT GPT and it told me there is no binding norm and extracted, what the main micoscope maunfacturers use.
  Here is the PDF. In essence, there is not one way of doing it.

Microscope_Coordinate_Systems.pdf

Third step

• Where is the origin of the sample the customer inspects?
• We have different sample forms. Is the origin different for each sample form?

Fourth step:

• Define +/- for each axes in line with existing conventions.

Then done.

[ethanjli]

Is this what you expect when working with a microscope and using a joystick feature?

My mental model is the "Google Maps" mental model:

• On my keyboard: When I want to see the area which is hidden by the left edge of my screen, I press the "left" arrow key. When I want to see the area which is hidden by the top edge of my screen, I press the "up" arrow key.
• On my phone: When I want to see the area which is hidden by the left edge of my screen, I drag my finger to the right. When I want to see the area which is hidden by the top edge of my screen, I drag my finger down.
• I don't know which approach (leftward/upward input corresponds to leftward/upward motion, or rightward/downward input corresponds to leftward/upward motion) would feel more intuitive to me with a joystick, but I'd like to have the option to try both and see which one I prefer. Many video games support this approach by having one default mode, and then providing an "invert axes" motion in a settings panel somewhere; typically there's also a similar "invert scrolling option" for touchpads and mouse scroll wheels in a mouse/touchpad settings panel. If I had to choose a default option, I'd probably choose the "leftward/upward input corresponds to leftward/upward motion" option—but I'm biased because when I play video games I use a keyboard, not a game controller with joysticks 🙂.

As an end-user who has never used joysticks with microscopes before, I would like to have either of these mental models without having to worry about whether the objective is moving or whether the stage is moving for any of the axes of motion (and perhaps such configurations may be different in non-FRAME openUC2-based microscopes).

[Mikrodoktor]

Agreed, the end user gets a default mental model and doesn't have to worry about anything else. And if we offer additional mental models, fine.
I referred to the joystick, since Bene has implemented a joystick in the live view page and I used it for tests. So it is not a real joystick. To me it is an equivalent to using arrow keys on the keyboard.

But we actually have a joystick => the PS4 Game controller.

So let us list the possible "inputs"

• im Switch live view page => currently programmed Stage control options:
  Option 1: Multiple Axis view

Option 2: Joystick control

• PS4 Game controller (Bluetooth connection)
• more?

[jhnnsrs]

@beniroquai asked me to give my few cents. Generally what I have seen with micromanager and is to convert absolute stage coordinates with an affine transformation to the camera space and do some sort of pixel size calibration routine to establish that in the first place. Position than would be center on optical axis.

[beniroquai]

So basically have an affine transformation between frame production and later use somehwere in e.g. ImSwitch. We could stick to the google maps convention the (?).
The problem is (so far) that the Joystick runs through the microcontroller directly (so doesn't now anything about the camera space) and all serial/explicit hardware controls run through ImSwitch (which knows about the camera obviously). We could have something that maps all axis by e.g. calibrating the affine matrix with a test script.
Pretty well done here: https://gitlab.com/openflexure/microscope-extensions/camera-stage-mapping
But too complex for my taste.

Still, my question would be: even if we have a google maps feeling, how do we move over/under the sample?

That's our current coordinate system (Would like to stick to that..)

A microscope should be able to reproduce it as if we look from above then (same goes for histslides with their marks and texts etc.

As for the Joystick we could have permanent sign switches that we can set from the serial side to toggle their direction logic.

[beniroquai]

@Mikrodoktor any update here? :)

[ethanjli]

@Mikrodoktor Thanks for providing this new diagram! Now that I see it visually, there is something slightly confusing about the +X/+Y directions versus what is displayed as +X and +Y in ImSwitch-Joystick. Specifically: in our coordinate axes, +X points to the right (i.e. in the direction of "East") and +Y points in the direction of "South". On the other hand, ImSwitch-Joystick's buttons show a yellow arrow with +X pointing to the left ("West") and a blue arrow with +Y pointing in the direction of "North". Should we modify ImSwitch-Joystick to make the yellow arrow pointing left/"West" say "-X" instead of "+X", and to make the blue arrow pointing "North" say "-X" instead of "+X"?