MayaFlux::Kinesis::Vision Namespace Reference

Namespaces
namespace	Kernels

Classes
struct	BoundingBox
	Axis-aligned bounding box in normalised image coordinates. More...
struct	BufferOutputs
	Live device views of the GPU-resident export buffers, one named slot per exporting op, populated only when its own export flag was set. More...
struct	CannyParams
struct	ChannelBand
	Inclusive accepted range of one channel. More...
struct	ComponentResult
	Result of connected component labelling. More...
struct	ConfineParams
	Parameters for Confine. More...
struct	ConnectedComponentsParams
	Connected-component label export and color output. More...
struct	Contour
	Closed contour extracted from a binary mask. More...
struct	ContoursBufferView
	Live device view of FindContours' traced contours, when FindContoursParams::export_contours_buffer is set. More...
struct	DetectEdgesParams
	Parameters for VisionIntent::DetectEdges. More...
struct	DetectFeaturesAnalysis
	Keypoint reduction, computed on GPU by the same reduction shader as TrackObjectsAnalysis (track_reduce.comp, fed previous = position so the velocity output is always zero and simply not surfaced here): centroid and bounds over every detected keypoint. More...
struct	DetectFeaturesParams
	Parameters for VisionIntent::DetectFeatures. More...
struct	ElementShape
	Per-element shape descriptors, parallel to FindElementsAnalysis::boxes (shapes[i] describes boxes[i]). More...
struct	EstimateMotionParams
	Parameters for VisionIntent::EstimateMotion. More...
struct	ExtractPeaksParams
	Peak extraction parameters. More...
struct	FieldSample
	Per-channel mean/min/max over a sampled image region, computed on GPU (Yantra's region_sample.comp) rather than downloaded and reduced on the caller's side. More...
struct	FilterSeparableParams
struct	FindContoursParams
	Contour filtering, trace limits, and output selection. More...
struct	FindElementsAnalysis
	FindElements output: contours plus the per-component BoundingBoxes ConnectedComponents already computes on GPU as a matter of course. More...
struct	FindElementsParams
	Parameters for VisionIntent::FindElements. More...
struct	GaussianBlurParams
struct	GradientResult
	Gradient maps produced by Sobel and Scharr operators. More...
struct	HarrisParams
	Harris response parameters. More...
struct	ImageOutputs
	One image slot per VisionOp that produces an image, populated only for whichever of these ops ran in the sequence. More...
struct	KernelSpec
	Compile-time 1D separable filter kernel. More...
struct	Keypoint
	Single interest point produced by a corner or blob detector. More...
struct	MeasureAppearanceAnalysis
	MeasureAppearance output: whole-frame photometric characterisation, computed on GPU (region_sample.comp over VisionResult::images.sobel, and a 256-bin brightness histogram over VisionResult::gray). More...
struct	MeasureAppearanceParams
	Parameters for VisionIntent::MeasureAppearance. More...
struct	MorphParams
struct	NormalizeRangeParams
struct	OpticalFlowDenseParams
	Parameters for OpticalFlowDense. More...
struct	OtsuParams
	Otsu threshold. More...
struct	PyramidLayout
	Placement of every pyramid level in a single atlas. More...
struct	PyramidLevel
	Extent and position of one pyramid level inside an atlas. More...
struct	ResolvedSequence
	One VisionSequence to run, and where its image argument comes from. More...
struct	SegmentationParams
	How FindElements separates foreground from background. More...
struct	SnapshotEntry
	Image captured mid-pipeline by a Snapshot step. More...
struct	ThresholdAdaptiveParams
	Adaptive threshold. More...
struct	ThresholdParams
	Fixed threshold. More...
struct	TrackKeypointsParams
	Parameters for TrackKeypoints. More...
struct	TrackObjectsAnalysis
	Track reduction, computed on GPU (Yantra's track_reduce.comp) from the (context-filtered) subject tracks: sum reduced to mean for centroid/velocity, min/max reduced to bounds. More...
struct	TrackObjectsContext
	Caller-supplied context for VisionIntent::TrackObjects. More...
struct	TrackObjectsParams
	Parameters for VisionIntent::TrackObjects. More...
struct	TrackResult
	Result for a single tracked point. More...
struct	VisionAnalysis
	Composed analysis output: one populated field per active intent that produced a result this call. More...
struct	VisionAnalysisContext
	Composed context: caller-supplied data for whichever intents need it, consulted after a VisionQuery's resolved sequences have run. More...
class	VisionExecutor
	Stateful executor for a VisionSequence. More...
struct	VisionPass
	State threaded through one execution of a VisionSequence. More...
struct	VisionQuery
	Composed query: which VisionIntents are active, and each active one's parameters. More...
struct	VisionResult
	Result of executing a VisionSequence on one frame. More...
struct	VisionSequence
	Ordered VisionSteps and the backend chosen to execute them. More...
struct	VisionStep
	One step in a VisionSequence: an op and its parameters. More...

Typedefs
using	CpuVisionPass = VisionPass< size_t >
using	GpuVisionPass = VisionPass< std::shared_ptr< Core::VKImage > >
using	SeedAccessor = std::shared_ptr< Core::VKImage >(*)(const VisionResult &)
	Picks a VisionResult field to seed a later sequence's image argument with, instead of the query's own raw frame.
using	SegmentationMethod = std::variant< OtsuParams, ThresholdAdaptiveParams, ThresholdParams >
	Threshold applied to the converted frame.
using	StructuredOutput = std::variant< std::monostate, GradientResult, ComponentResult, std::vector< Contour >, std::vector< Keypoint >, std::vector< TrackResult > >
using	VisionParams = std::variant< std::monostate, ThresholdParams, ThresholdAdaptiveParams, OtsuParams, NormalizeRangeParams, GaussianBlurParams, FilterSeparableParams, CannyParams, MorphParams, HarrisParams, ExtractPeaksParams, TrackKeypointsParams, ConnectedComponentsParams, FindContoursParams, OpticalFlowDenseParams, ConfineParams >
	Parameter variant covering all ops that carry parameters.

Enumerations
enum class	ChannelMask : uint8_t { NONE = 0U , Ch0 = 1U << 0U , Ch1 = 1U << 1U , Ch2 = 1U << 2U }
	Selects channels of the current image for a multichannel threshold. More...
enum class	SegmentationSpace : uint8_t { Gray , Rgba , Hsv }
	Colour representation FindElements segments in, which decides the conversion its chain leads with. More...
enum class	VisionBackend : uint8_t { GPU , CPU }
	Selects where a VisionSequence executes. More...
enum class	VisionIntent : uint32_t {   NONE = 0U , FindElements = 1U << 0U , TrackObjects = 1U << 1U , DetectFeatures = 1U << 2U ,   DetectEdges = 1U << 3U , EstimateMotion = 1U << 4U , MeasureAppearance = 1U << 5U , Classify = 1U << 6U ,   EstimateDepth = 1U << 7U , DetectAnomaly = 1U << 8U , Register = 1U << 9U }
	Composable flags naming what a vision analysis should produce. More...
enum class	VisionOp : uint8_t {   RgbaToGray , RgbaToHsv , GrayToRgba , Downsample2x ,   Threshold , ThresholdAdaptive , ThresholdOtsu , NormalizeInplace ,   NormalizeRange , GaussianBlur , FilterSeparable , Sobel ,   Scharr , Canny , Erode , Dilate ,   Open , Close , MorphGradient , ConnectedComponents ,   FindContours , HarrisResponse , ExtractPeaks , TrackKeypoints ,   Snapshot , OpticalFlowDense , Confine }
	Named operations available in a VisionSequence. More...
enum class	VisionStatus : uint8_t { COMPLETE , SUSPENDED }
	Whether a run carried the sequence to its end. More...

Functions
void	apply_contour_mask (std::span< float > pixels, uint32_t w, uint32_t h, uint32_t channels, const Contour &contour, float origin_x, float origin_y, float scale_x, float scale_y)
	Zero all pixels in pixels that fall outside contour.
void	canny (std::span< const float > gray, std::span< float > dst, uint32_t w, uint32_t h, float sigma, float low_threshold, float high_threshold)
	Canny edge detector writing into a caller-supplied buffer.
std::vector< float >	canny (std::span< const float > gray, uint32_t w, uint32_t h, float sigma, float low_threshold, float high_threshold)
	Canny edge detector.
void	close (std::span< const float > mask, std::span< float > tmp, std::span< float > dst, uint32_t w, uint32_t h, uint32_t radius)
	Morphological closing writing into dst.
std::vector< float >	close (std::span< const float > mask, uint32_t w, uint32_t h, uint32_t radius)
	Morphological closing: dilate then erode.
ComponentResult	connected_components (std::span< const float > mask, uint32_t w, uint32_t h)
	Label connected foreground components in a binary mask.
void	dilate (std::span< const float > mask, std::span< float > dst, uint32_t w, uint32_t h, uint32_t radius)
	Dilation writing into dst.
std::vector< float >	dilate (std::span< const float > mask, uint32_t w, uint32_t h, uint32_t radius)
	Morphological dilation with square structuring element.
void	downsample_2x (std::span< const float > src, std::span< float > dst, uint32_t w, uint32_t h, uint32_t &new_w, uint32_t &new_h)
	2x box-filter downsample writing into a caller-supplied buffer.
void	downsample_2x (std::span< const float > src, std::span< float > dst, uint32_t w, uint32_t h, uint32_t channels, uint32_t &new_w, uint32_t &new_h)
	2x box-filter downsample for multi-channel interleaved data.
std::vector< float >	downsample_2x (std::span< const float > src, uint32_t w, uint32_t h, uint32_t &new_w, uint32_t &new_h)
	2x box-filter downsample.
void	erode (std::span< const float > mask, std::span< float > dst, uint32_t w, uint32_t h, uint32_t radius)
	Erosion writing into dst.
std::vector< float >	erode (std::span< const float > mask, uint32_t w, uint32_t h, uint32_t radius)
	Morphological erosion with square structuring element.
std::vector< Keypoint >	extract_peaks (std::span< const float > response, uint32_t w, uint32_t h, float threshold, uint32_t nms_radius)
	Extract peaks from a response map via non-maximum suppression.
void	filter_horizontal_planes (std::span< const float *const > src, std::span< float *const > dst, uint32_t w, uint32_t h, std::span< const float > kernel)
	Fused horizontal separable filter over N planes.
void	filter_separable (std::span< const float > src, std::span< float > tmp, std::span< float > dst, uint32_t w, uint32_t h, std::span< const float > kernel_x, std::span< const float > kernel_y)
	Separable 2D filter writing into caller-supplied buffers.
std::vector< float >	filter_separable (std::span< const float > src, uint32_t w, uint32_t h, std::span< const float > kernel_x, std::span< const float > kernel_y)
	Apply a separable 2D filter via two 1D passes.
void	filter_vertical_planes (std::span< const float *const > src, std::span< float *const > dst, uint32_t w, uint32_t h, std::span< const float > kernel)
	Fused vertical separable filter over N planes.
std::vector< Contour >	find_contours (const ComponentResult &components, uint32_t w, uint32_t h, float min_area=0.0F, uint32_t max_contours=0, uint32_t max_points_per_contour=0)
	Extract contours from an existing CPU component labeling.
std::vector< Contour >	find_contours (std::span< const float > mask, uint32_t w, uint32_t h, float min_area=0.0F, uint32_t max_contours=0, uint32_t max_points_per_contour=0)
	Extract outer and hole contours from a binary mask.
void	gaussian_blur (std::span< const float > src, std::span< float > tmp, std::span< float > dst, uint32_t w, uint32_t h, float sigma)
	Gaussian blur writing into caller-supplied buffers.
std::vector< float >	gaussian_blur (std::span< const float > src, uint32_t w, uint32_t h, float sigma)
	Separable Gaussian blur.
void	gray_to_rgba (std::span< const float > gray, std::span< float > dst, uint32_t w, uint32_t h)
	Global threshold writing into caller-supplied buffer.
std::vector< float >	gray_to_rgba (std::span< const float > gray, uint32_t w, uint32_t h)
	Convert single-channel gray to RGBA.
void	harris_response (std::span< const float > gray, std::span< float > dx, std::span< float > dy, std::span< float > tmp, std::span< float > ixx, std::span< float > iyy, std::span< float > ixy, std::span< float > sxx, std::span< float > syy, std::span< float > sxy, std::span< float > dst, uint32_t w, uint32_t h, float k, std::span< const float > kern)
	Compute the Harris corner response map into caller-supplied buffers.
std::vector< float >	harris_response (std::span< const float > gray, uint32_t w, uint32_t h, float k=0.04F, float sigma=1.0F)
	Compute the Harris corner response map.
void	hash_combine (size_t &seed, size_t value)
	Combine a hash into an existing seed, FNV-style.
void	hash_region (size_t &seed, const std::optional< BoundingBox > &region)
	Combine an optional region into a hash: absence and presence hash differently, presence by its four bounds.
size_t	hash_vision_step (VisionOp op, const VisionParams &params)
	Hash a VisionStep's op and parameters together.
void	morph_gradient (std::span< const float > mask, std::span< float > tmp, std::span< float > dst, uint32_t w, uint32_t h, uint32_t radius)
	Morphological gradient (dilate - erode) writing into dst.
std::vector< float >	morph_gradient (std::span< const float > mask, uint32_t w, uint32_t h, uint32_t radius)
	Morphological gradient: dilate minus erode.
void	normalize_inplace (std::span< float > data)
	Stretch values to fill [0, 1] using the span's observed min and max.
void	normalize_range_inplace (std::span< float > data, float lo, float hi)
	Clamp and remap values from [lo, hi] to [0, 1] in-place.
void	open (std::span< const float > mask, std::span< float > tmp, std::span< float > dst, uint32_t w, uint32_t h, uint32_t radius)
	Morphological opening writing into dst.
std::vector< float >	open (std::span< const float > mask, uint32_t w, uint32_t h, uint32_t radius)
	Morphological opening: erode then dilate.
PyramidLayout	pyramid_layout (uint32_t w, uint32_t h, uint32_t requested, uint32_t min_extent=16)
	Place up to requested pyramid levels for a w x h base image.
std::vector< ResolvedSequence >	resolve (const VisionQuery &query)
	Resolve a VisionQuery into the ResolvedSequences needed to satisfy every active intent.
void	rgba_to_gray (std::span< const float > rgba, std::span< float > dst, uint32_t w, uint32_t h)
	Convert RGBA to luminance gray using BT.601 coefficients.
std::vector< float >	rgba_to_gray (std::span< const float > rgba, uint32_t w, uint32_t h)
	Convert RGBA to luminance gray using BT.601 coefficients.
void	rgba_to_hsv (std::span< const float > rgba, std::span< float > dst, uint32_t w, uint32_t h)
	Convert RGBA to HSV writing into caller-supplied buffer.
std::vector< float >	rgba_to_hsv (std::span< const float > rgba, uint32_t w, uint32_t h)
	Convert RGBA to HSV.
void	scharr (std::span< const float > gray, std::span< float > dx, std::span< float > dy, std::span< float > tmp, uint32_t w, uint32_t h)
	Scharr gradient writing dx and dy into caller-supplied buffers.
GradientResult	scharr (std::span< const float > gray, uint32_t w, uint32_t h)
	Scharr gradient operator.
void	sobel (std::span< const float > gray, std::span< float > dx, std::span< float > dy, std::span< float > tmp, uint32_t w, uint32_t h)
	Sobel gradient writing dx and dy into caller-supplied buffers.
GradientResult	sobel (std::span< const float > gray, uint32_t w, uint32_t h)
	Sobel gradient operator.
std::vector< float >	threshold (std::span< const float > gray, float value)
	Global threshold: output is 1.0f where input >= value, else 0.0f.
void	threshold (std::span< const float > gray, std::span< float > dst, float value)
	Global threshold writing into caller-supplied buffer.
void	threshold_adaptive (std::span< const float > gray, std::span< float > dst, uint32_t w, uint32_t h, uint32_t block_size, float offset)
	Adaptive threshold writing into caller-supplied buffer.
std::vector< float >	threshold_adaptive (std::span< const float > gray, uint32_t w, uint32_t h, uint32_t block_size, float offset)
	Adaptive threshold using local block mean minus offset.
std::vector< float >	threshold_otsu (std::span< const float > gray)
	Otsu threshold: finds the optimal global threshold by maximising inter-class variance, then applies it.
void	threshold_otsu (std::span< const float > gray, std::span< float > dst)
	Otsu threshold writing into caller-supplied buffer.
bool	track_follows_peaks (const VisionSequence &seq)
	True when an ExtractPeaks step is immediately followed by TrackKeypoints.
std::vector< TrackResult >	track_keypoints (std::span< const float > prev_gray, std::span< const float > curr_gray, std::span< const float > grad_dx, std::span< const float > grad_dy, uint32_t w, uint32_t h, std::span< const glm::vec2 > prev_points, uint32_t window_radius=7, uint32_t max_iterations=20, float eigen_threshold=1e-4F, float error_threshold=0.3F, float forward_backward_threshold=0.0F)
	Track keypoints using a pre-computed Sobel gradient of prev_gray.
std::vector< TrackResult >	track_keypoints (std::span< const float > prev_gray, std::span< const float > curr_gray, uint32_t w, uint32_t h, std::span< const glm::vec2 > prev_points, uint32_t window_radius=7, uint32_t max_iterations=20, float eigen_threshold=1e-4F, float error_threshold=0.3F, float forward_backward_threshold=0.0F)
	Track keypoints from prev_gray to curr_gray via Lucas-Kanade.
bool	tracks_keypoints (const VisionSequence &seq)
	True when any step tracks keypoints.