Loading...
Searching...
No Matches
Public Types |
Public Member Functions |
Private Member Functions |
Private Attributes |
List of all members
MayaFlux::Yantra::VisionExtractor Class Reference
Extraction from each VisionAnalysis field. More...
#include <VisionExtractor.hpp>
Inheritance diagram for MayaFlux::Yantra::VisionExtractor: Collaboration diagram for MayaFlux::Yantra::VisionExtractor:Public Types | |
| using | Base = FunctionalOperation< std::shared_ptr< Kakshya::SignalSourceContainer >, std::vector< Kakshya::DataVariant > > |
| Public Types inherited from MayaFlux::Yantra::FunctionalOperation< std::shared_ptr< Kakshya::SignalSourceContainer >, std::vector< Kakshya::DataVariant > > | |
| using | Fn = std::function< output_type(const input_type &)> |
| using | input_type = Datum< InputType > |
| using | output_type = Datum< OutputType > |
| Public Types inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| using | input_type = Datum< InputType > |
| using | output_type = Datum< OutputType > |
Public Member Functions | |
| std::shared_ptr< Core::VKImage > | annotate (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t source_w, uint32_t source_h) |
| source with everything the analysis found drawn on it: element boxes (yellow) and contours (cyan), track arrows from previous to current position (green tracked, red lost) with their bounds and centroid (white), keypoints (magenta) with their bounds. | |
| std::shared_ptr< Core::VKImage > | crop (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t out_w, uint32_t out_h) |
| Crop of source at the focused element, or track/keypoint bounds. | |
| std::shared_ptr< Core::VKImage > | crops (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t tile_w, uint32_t tile_h) |
| Every find_elements box resampled to tile_w x tile_h, side by side in one atlas (tile_w * count, tile_h), in box order. | |
| std::shared_ptr< Core::VKImage > | edges (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t source_w, uint32_t source_h) |
| source kept where analysis.detect_edges reaches the edge threshold, transparent elsewhere. | |
| std::shared_ptr< Core::VKImage > | equalize (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t source_w, uint32_t source_h) |
| Histogram equalisation of source from the histogram measure_appearance measured. | |
| std::shared_ptr< Core::VKImage > | expose (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, float target, uint32_t source_w, uint32_t source_h) |
| source scaled so its measured mean brightness reaches target. | |
| const std::optional< ElementFocus > & | get_element_focus () const |
| size_t | get_element_index () const |
| VisionExtractMode | get_mode () const |
| std::shared_ptr< Core::VKImage > | mask (VisionGpuExecutor &executor, const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t w, uint32_t h) |
| Pixel-precise silhouette of the focused element from the executor's label buffer. | |
| std::shared_ptr< Core::VKImage > | motion_mask (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t source_w, uint32_t source_h) |
| source kept where analysis.estimate_motion moves faster than the motion limits with enough confidence. | |
| std::shared_ptr< Core::VKImage > | patches (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t patch_w, uint32_t patch_h, uint32_t source_w, uint32_t source_h) |
| One fixed-size crop per keypoint, else per track, side by side in one atlas. | |
| Kinesis::Vision::FieldSample | sample (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t source_w, uint32_t source_h) |
| Mean, min and max over the focused element or track/keypoint bounds. | |
| std::vector< glm::vec4 > | sample_points (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t radius, uint32_t source_w, uint32_t source_h) |
| Mean over a (2 * radius + 1) squared window at every keypoint, else every track, in one dispatch, in point order. | |
| std::vector< Kinesis::Vision::FieldSample > | sample_regions (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t source_w, uint32_t source_h) |
| sample() for every find_elements box in one dispatch, in box order. | |
| std::optional< size_t > | select_element (const Kinesis::Vision::VisionAnalysis &analysis, const ElementFocus &focus) |
| Index into find_elements boxes of the element ranked first by focus, chosen on the GPU from the analyzer's own area, shape and nearest-neighbour results. | |
| void | set_edge_threshold (float threshold) |
| Edge value edges() keeps a pixel at. | |
| void | set_element_focus (std::optional< ElementFocus > focus) |
| Rank elements by shape instead of using a fixed index; empty restores the index. | |
| void | set_element_index (size_t index) |
| void | set_exposure_target (float target) |
| Mean brightness expose() scales the frame to. | |
| void | set_mode (VisionExtractMode mode) |
| void | set_motion_limits (float min_speed, float min_confidence, float min_activity) |
| motion_mask() limits: minimum flow speed in pixels of the flow image, minimum solver confidence, and a minimum VisionAnalysis::motion_activity below which nothing is dispatched. | |
| void | set_output_size (uint32_t w, uint32_t h) |
| void | set_patch_size (uint32_t w, uint32_t h) |
| void | set_sample_radius (uint32_t radius) |
| Window radius sample_points() averages over. | |
| void | set_tile_size (uint32_t w, uint32_t h) |
| std::shared_ptr< Core::VKImage > | silhouette (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t tile_w, uint32_t tile_h) |
| silhouettes() for the focused element only. | |
| std::shared_ptr< Core::VKImage > | silhouettes (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::VisionAnalysis &analysis, uint32_t tile_w, uint32_t tile_h) |
| Silhouette of every element cut from its contour polygons: the outer contour and its holes, even-odd filled. | |
| VisionExtractor (VisionExtractMode mode=VisionExtractMode::Crop) | |
| Public Member Functions inherited from MayaFlux::Yantra::FunctionalOperation< std::shared_ptr< Kakshya::SignalSourceContainer >, std::vector< Kakshya::DataVariant > > | |
| FunctionalOperation (Fn fn) | |
| FunctionalOperation (std::shared_ptr< GpuExecutionContext< InputType, OutputType > > gpu_backend) | |
| Construct GPU-only: dispatches entirely via gpu_backend, no callable, no CPU fallback. | |
| std::map< std::string, std::any > | get_all_parameters () const override |
| Retrieves all parameters and their values. | |
| OperationType | get_operation_type () const override |
| Returns the category of this operation for grammar and registry discovery. | |
| std::any | get_parameter (const std::string &name) const override |
| Retrieves a parameter's current value. | |
| void | set_parameter (const std::string &name, std::any value) override |
| Sets a named parameter that configures the operation's behavior. | |
| Public Member Functions inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| void | add_dependency (std::shared_ptr< ComputeOperation > dep) |
| output_type | apply_operation (const input_type &input) |
| Public synchronous execution interface. | |
| output_type | apply_operation_with_dependencies (const input_type &input) |
| Applies the operation with dependencies resolved. | |
| OutputType | apply_to_data (const InputType &data) |
| Convenience overload that extracts just the data from result. | |
| ComputeOperation () | |
| Constructor with data type validation warnings. | |
| output_type | execute (const input_type &input) |
| OpUnit interface - operations can act as units in dependency graphs. | |
| virtual const std::shared_ptr< Kakshya::SignalSourceContainer > & | get_container_for_regions () const |
| const auto & | get_dependencies () const |
| const ExecutionContext & | get_last_execution_context () const |
| virtual std::string | get_name () const |
| Get operation name for debugging/introspection. | |
| bool | has_gpu_backend () const |
| output_type | operator() (const InputType &data) |
| Convenience overload for direct data processing (backward compatibility) | |
| virtual void | set_container_for_regions (const std::shared_ptr< Kakshya::SignalSourceContainer > &container) |
| void | set_gpu_backend (std::shared_ptr< GpuExecutionContext< InputType, OutputType > > backend) |
| Attach a GPU execution backend. | |
| void | set_last_execution_context (const ExecutionContext &ctx) |
| void | set_post_execution_hook (const OperationHookCallback &hook) |
| void | set_pre_execution_hook (const OperationHookCallback &hook) |
| void | set_reconstruction_callback (const ReconstructionCallback &callback) |
| virtual bool | validate_input (const input_type &) const |
| Validates if the input data meets the operation's requirements. | |
| virtual | ~ComputeOperation ()=default |
| Virtual destructor for proper cleanup of derived classes. | |
Private Member Functions | |
| std::shared_ptr< Core::VKImage > | fill_elements (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::FindElementsAnalysis &elements, const std::vector< size_t > &indices, uint32_t tile_w, uint32_t tile_h) |
| std::optional< size_t > | focus_box_index (const Kinesis::Vision::VisionAnalysis &analysis) |
| Box index of the element the single-element methods act on. | |
| TextureExecutionContext & | image_context () |
| The two contexts every method shares, one shader swapped in per call. | |
| std::optional< uint32_t > | label_for_analysis (const Kinesis::Vision::VisionAnalysis &analysis) |
| std::vector< glm::vec2 > | points_for_analysis (const Kinesis::Vision::VisionAnalysis &analysis) const |
| TextureExecutionContext & | reduce_context () |
| std::optional< Kinesis::Vision::BoundingBox > | region_for_analysis (const Kinesis::Vision::VisionAnalysis &analysis) |
| std::shared_ptr< Core::VKImage > | resolve_image (const std::shared_ptr< Kakshya::SignalSourceContainer > &source, size_t index) |
| output_type | run_operation (const input_type &input) |
| std::shared_ptr< Core::VKImage > | tone (const std::shared_ptr< Core::VKImage > &source, const Kinesis::Vision::MeasureAppearanceAnalysis &appearance, uint32_t mode, float gain, uint32_t source_w, uint32_t source_h) |
Private Attributes | |
| float | m_edge_threshold { 0.5F } |
| size_t | m_element_index { 0 } |
| float | m_exposure_target { 0.5F } |
| std::optional< ElementFocus > | m_focus |
| std::shared_ptr< TextureExecutionContext > | m_image_ctx |
| float | m_min_activity { 0.0F } |
| float | m_min_confidence { 0.1F } |
| float | m_min_speed { 0.5F } |
| VisionExtractMode | m_mode |
| uint32_t | m_out_h { 0 } |
| uint32_t | m_out_w { 0 } |
| uint32_t | m_patch_h { 0 } |
| uint32_t | m_patch_w { 0 } |
| std::shared_ptr< TextureExecutionContext > | m_reduce_ctx |
| uint32_t | m_sample_radius { 1 } |
| uint32_t | m_tile_h { 0 } |
| uint32_t | m_tile_w { 0 } |
| uint32_t | m_upload_h { 0 } |
| std::shared_ptr< Core::VKImage > | m_upload_image |
| uint32_t | m_upload_w { 0 } |
Additional Inherited Members | |
| Protected Member Functions inherited from MayaFlux::Yantra::FunctionalOperation< std::shared_ptr< Kakshya::SignalSourceContainer >, std::vector< Kakshya::DataVariant > > | |
| output_type | operation_function (const input_type &input) override |
| Executes the computational transformation on the input data. | |
| Protected Member Functions inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| output_type | apply_hooks (const input_type &input, const ExecutionContext &context) |
| virtual std::future< output_type > | apply_operation_async (const input_type &input) |
| Optional async implementation - default delegates to operation_function. | |
| virtual output_type | apply_operation_chained (const input_type &input, const ExecutionContext &ctx) |
| Optional chain-aware implementation - default delegates to operation_function. | |
| virtual output_type | apply_operation_internal (const input_type &input, const ExecutionContext &context) |
| Internal execution method - ComputeMatrix can access this. | |
| virtual output_type | apply_operation_parallel (const input_type &input, const ExecutionContext &ctx) |
| Optional parallel-aware implementation - default delegates to operation_function. | |
| output_type | convert_result (std::vector< std::vector< double > > &result_data, DataStructureInfo &metadata) |
| Convert processed double data back to OutputType using metadata and optional callback. | |
| Protected Attributes inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| std::shared_ptr< Kakshya::SignalSourceContainer > | m_container |
| std::shared_ptr< GpuExecutionContext< InputType, OutputType > > | m_gpu_backend |
| ExecutionContext | m_last_execution_context |
Detailed Description
Extraction from each VisionAnalysis field.
find_elements: crop() one element, crops() all, mask() by label buffer, silhouettes() by contour polygon (no executor), sample_regions(), and select_element() to rank by shape. tracks and keypoints: patches() and sample_points(). measure_appearance: equalize() and expose(). detect_edges: edges(). estimate_motion: motion_mask(). Any field: annotate() draws everything found onto the frame.
Which find_elements entry the single-element methods use comes from set_element_focus() when set (GPU-ranked by shape), otherwise from set_element_index() into contours (sorted by area when max_contours was set) or, with no contours, into boxes.
Thresholds and sizes used by run_operation are set through setters, the same convention as the other operations; the direct methods take only the data and the dimensions of what they read or write.
Image-producing methods share one output slot, so a returned image stays valid only until the next call that produces an image of the same size. Hold on to a result by using it (display, copy) before extracting again.
Definition at line 102 of file VisionExtractor.hpp.
The documentation for this class was generated from the following files:
