Sorting out the overlap between skeleton_keys and neuron_morphology

[gouwens]

There are multiple functions in skeleton_keys that are essentially modified versions of functions from neuron_morphology (kind of a semi-fork). Some of them may be more suited to being in skeleton_keys than neuron_morphology, as they may be more related to data handling/pipeline management than feature calculation. In any case, we probably don't want to have duplicate functionality hanging around for the long-term.

I've tried to go through skeleton-keys to identify the overlapping functions.

Database queries

• skeleton_keys.database_queries.query_for_cortical_surfaces & neuron_morphology.snap_polygons._from_lims.query_for_cortical_surfaces - The skeleton_keys version does some additional matching using biospecimen_id to resolve ambiguous situations. It also returns the resolution of the drawings' images.
• skeleton_keys.database_queries.query_for_layer_polygons & neuron_morphology.snap_polygons._from_lims. query_for_layer_polygons - The neuron_morphology version optionally validates the polygons, but does not return the resolution. Might make sense for validation to happen outside the query function (since drawings can come from files in skeleton_keys).

My inclination is to move LIMS queries into skeleton_keys out from neuron_morphology, but we'd have to see how integrated those functions are in neuron_morphology (there are also some query-related functions in neuron_morphology that aren't in skeleton_keys).

Depth profiles

There is no PCA of depth profiles in neuron_morphology, but there are functions for calculating depth histograms and earthmover distances in neuron_morphology.features.layer for individual neurons. There doesn't seem to be an easy way to get the histograms for a bunch of neurons together, though. There is also an overlap feature calculation in neuron_morphology.features.statistics.overlap which doesn't make use of the histograms (it uses the node coordinates, which could either be from a layer-aligned morphology or original morphology; the latter wouldn't account for curvature in the overlap calculation).

I'm not super familiar with the implementation of the depth profile histograms in neuron_morphology - it seems kind of complicated at first glance, but maybe it's worth evaluating.

Layer drawings

• skeleton_keys.drawings.snap_hand_drawn_polygons & neuron_morphology.snap_polygons.__main__.run_snap_polygons - The neuron_morphology version seems set up to be run as an executable (can have a side effect of writing images). The versions seems to have diverged a bit; probably worth comparing the performance on a set of problem drawings. The neuron_morphology version no longer uses the rasterization strategy, so some of the problems checked for by the skeleton_keys version might not come up any more.

This probably belongs in neuron_morphology, but could be rewritten as an actual API function (versus being tied to an executable).

Layer alignment

• skeleton_keys.layer_alignment.layer_aligned_y_values & neuron_morphology.layered_point_depths.__main__.run_layered_point_depths - The former is written to extract the calculation from being tied to an executable in the latter form, which takes an input SWC file and an output CSV file path as parameters rather than morphology objects and sets of coordinates, respectively. The neuron_morphology version also doesn't handle points outside of layers (i.e. in the white matter). "layered depths" is also a phrase that doesn't really make sense and should probably be changed.
• skeleton_keys.layer_alignment.nodes_fractions_within_layer & neuron_morphology.layered_point_depths.__main__.get_node_intersections - The neuron_morphology version works on a single node and returns more information than just the fraction through the layer (as it's intended for a CSV output rather than a modified Morphology object). The skeleton_keys version is vectorized for better performance.
• skeleton_keys.layer_alignment.step_all_nodes & neuron_morphology.layered_point_depths.__main__.step_from_node - the skeleton_keys version acts on all nodes in a layer at a time to improve performance. It also has a dependency on geopandas to do a vectorized intersection operation.
• skeleton_keys.layer_alignment.layer_locations & neuron_morphology.layered_point_depths.__main__.containing_layer - the skeleton_keys version uses a vectorized intersection to work on all points at once, rather than a single point as in neuron_morphology
• skeleton_keys.layer_alignment.setup_interpolator_without_nan & neuron_morphology.layered_point_depths.__main__.setup_interpolator - the skeleton_keys version extends non-nan values to the edge of the field to avoid problems when the ray tracing hits an edge

These probably belong in neuron_morphology, but refactored to not have them tied to an executable.

Slice angle tilt correction

• skeleton_keys.slice_angle.slice_angle_tilt & neuron_morphology.transforms.tilt_correction.compute_tilt_correction - the skeleton_keys version uses the ccf_streamlines package, which is a bit cleaner now.

My inclination is for this kind of operation to be in skeleton_keys.