Miscellaneous QOL

[talmo]

Laundry list of small features and tweaks:

High level APIs:

• sleap.load_video(): create a sleap.Video from filename
• sleap.load_config(): load training job config json from model folders
• sleap.load_file() now takes a match_to kwarg to align data structure instances with other labels for easy comparison and manipulation.
• sleap.versions(): print package/OS versions
• Labels, LabeledFrame, Instance, PredictedInstance, Video, Skeleton now all have readable __repr__ and __str__, and __len__ for easy inspection of contents (nodes, visible parts).
• Labels.extract() to pull out a subset of frames into new labels while retaining metadata from the base labels.
• Labels.export() to save to analysis HDF5
• Labels.numpy(), LabeledFrame.numpy() to convert main data structures to numpy
• Labels.describe() is more comprehensive overview of a dataset.
• Labels.load_deeplabcut_folder() to interactively import a maDLC dataset.

Core workflow

• Add ability to add any frame as suggestion and modify suggestion list. This makes it possible to use the suggestions as a "labeling queue". This can be used to keep track of labeled frames and frames intended to be labeled (either detected algorithmically or manually selected frames of interest by user). This list also serves as a convenient target for inference for initialization/reprediction after training:
  
• Add Labels.clear_suggestions(), Labels.unlabeled_suggestions, Labels.get_unlabeled_suggestion_inds(), Labels.remove_empty_frames(), LabelsReader.from_user_labeled_frames(), LabelsReader.from_unlabeled_suggestions() to support suggestion-driven workflow
• Remote workflow: Support for packaging output model folders as a zip file after training for easy download. Support for zip files as input to sleap.load_model().
• Remote workflow: Support for packaging training + inference data, training job configurations, and CLI launch scripts into a single zip file for easy upload to Colab or other remote node:
  
  
• File -> Save as... now uses labels.v000.slp as the default filename and scans the containing directory for labels following this pattern to increment the version number automatically. Saving the labels to a new version now takes just two keyboard strokes (Ctrl + Shift + S -> Enter by default). These two changes should direct users towards label versioning best practices.

GUI/UX

• Window state (panels/tabs locations and visibility) are now persisted across sessions and restored when GUI is opened.
• Add option to adjust node marker size to make them easier to see on high resolution displays. This and other visualization settings are now appropriately persisted across sessions.
• Adjusted keyboard shortcuts to more sensible defaults and added ability to reset to "factory defaults".
• Added progress indicators for long running operations such as exporting training packages and running inference. Inference progress is also displayed in terminal and notebooks:
  
  
• Training and inference now have a Cancel button which immediately kills the subprocess and cleans up temp files or incomplete outputs.
• Training package exporting options are moved to a submenu with clearer descriptions of what they will save.
• Moved random flip augmentation options to a simpler dropdown menu (none, horizontal, vertical) rather than two checkboxes (enable/disable + horizontal/vertical)
• Add name and description fields to TrainingJobConfig to make it easier to annotate preset profiles with information relevant for users to select between them. This will be displayed in the GUI in a future update to drive a simplified training dialog workflow.
• Move track/identity-related options to Tracks menu and add a toggle for labels propagation. When disabled, setting track identity no longer affects subsequent frames.

Training/inference

• Training CLI now doesn't override true parameters in the config with flag false defaults (e.g., config.outputs.save_visualizations works when --save_viz is not passed).
• Training CLI no longer requires the positional labels_path arg since this can be specified in the config like the validation and test splits.
• Training will now delete the visualizations subfolders if config.outputs.delete_viz_images is true (the default). This was often larger than the models themselves and usually only used during GUI-based training for live visualization.
• Training will now zip the model folder if self.config.outputs.zip_outputs is true to support core remote workflow.
• sleap.load_model() now supports loading models from zip files to support core remote workflow.
• Massive refactor of the sleap.nn.inference submodule: deleted unused classes and methods and move a lot of functionality to the Predictor base class.
• Predictors now use tf.keras.Model.predict_on_batch() in inference loops to drastically improve speed by cached traced/autographed model call every batch.
• Inference now supports progress reporting using rich or by outputting JSON-encoded updates to stdout which can be captured by any caller. FPS and ETA are now calculated using a recent rolling buffer for more accurate estimates without needing to wait for startup/autograph amortization.
• Reworked inference CLI (sleap-track) to make it much, much clearer and follow a more linear workflow. Deprecated many redundant args (still supported but hidden from sleap-track -h).
• Inference CLI now takes a video or labels file as the positional data_path argument (e.g., sleap-track -m path/to/model "labels.slp") without using the --labels flag.
• Inference CLI now uses a single LabelsReader/VideoReader provider to iterate over all inference targets. Previously, this was restarted per video, slowing down the whole process.
• Inference on suggestions now only predicts on suggestions associated with unlabeled frames.
• Inference CLI now stores much more metadata in Labels.provenance before saving, including system info, paths, sleap version, timestamps, etc.
• TrainingJobConfig now caches the sleap version and filename it was loaded from and saved to during I/O ops

Fixes

• Instance counting no longer double counts Instances/PredictedInstances. This affects the model folder auto-naming, labeled instance count in suggestions table and more.
• sleap-label will now run in CPU-only mode to prevent pre-allocating all the GPU memory if any tensorflow ops are called.
• sleap.load_model() will now default to disabling pre-allocation of all the GPU memory even if called interactively to prevent the same issue.
• Fix dataset splitting to ensure there is a minimum of one sample per train/val split. Supports training with a single(!) label now.
• Suggestions row is now selected appropriately when navigating suggestions.
• Flip augmentation now applied correctly across all model types.