|
MayaFlux 1.0.0
Digital-First Multimedia Processing Framework
|
Whether each 32 ms chunk of a mono 16 kHz stream is voice activity, from a learned network. More...
#include <VadAnalyzer.hpp>
Public Types | |
| using | base_type = Analyzer<InputType, OutputType> |
| using | input_type = Datum<InputType> |
| using | output_type = Datum<OutputType> |
| using | supports = Supports<VadAnalyzerConfig> |
| Public Types inherited from MayaFlux::Yantra::Analyzer< InputType, OutputType > | |
| using | base_type = ComputeOperation<InputType, OutputType> |
| using | input_type = Datum<InputType> |
| using | output_type = Datum<OutputType> |
| Public Types inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| using | input_type = Datum<InputType> |
| using | output_type = Datum<OutputType> |
Public Member Functions | |
| VadAnalysis | analyze_speech (const input_type &data) |
| Type-safe speech probability analysis. | |
| VadAnalysis | analyze_speech (const InputType &data) |
| ConfigRefs | configs () const override |
| The base view of the config structs this operation holds. | |
| AnalysisType | get_analysis_type () const override |
| Gets the analysis type category for this analyzer. | |
| std::string | get_name () const override |
| Get operation name for debugging/introspection. | |
| VadAnalysis | get_speech_analysis () const |
| SileroVadModel::ProbabilityBuffer | hidden_buffer () |
| Where the network's recurrent output of the latest run is on the device, SileroVadModel::hidden_width values per chunk. | |
| SileroVadModel::ProbabilityBuffer | probability_buffer () |
| Where the probabilities of the latest run are on the device, for a consumer on the same recording. | |
| size_t | record (std::span< const float > samples) |
| Run every chunk the samples complete without waiting for the result. | |
| void | reset () |
| Start a new signal: forget the recurrent state, the context and any partial chunk. | |
| size_t | resident_count () const noexcept |
| Probabilities the latest run left in probability_buffer. | |
| void | share_recording (std::shared_ptr< GpuDispatchRecorder > recording) |
| Append to a recording the caller shares with other work. | |
| VadAnalyzer (const Kakshya::TensorSet &weights, VadAnalyzerConfig config={}) | |
| Construct from the network weights. | |
| Public Member Functions inherited from MayaFlux::Yantra::Analyzer< InputType, OutputType > | |
| std::vector< std::any > | analyze_batch (const std::vector< input_type > &inputs) |
| Batch analysis for multiple inputs. | |
| std::vector< std::any > | analyze_batch (const std::vector< InputType > &inputs) |
| std::any | analyze_data (const input_type &data) |
| User-facing analysis method - returns analysis results directly. | |
| std::any | analyze_data (const InputType &data) |
| std::any | get_current_analysis () const |
| Access cached analysis from last operation. | |
| OperationType | get_operation_type () const override |
| Returns the category of this operation for grammar and registry discovery. | |
| bool | has_current_analysis () const |
| Check if analysis has been performed. | |
| bool | validate_input (const input_type &input) const override |
| Validates input data meets analyzer requirements. | |
| virtual | ~Analyzer ()=default |
| Public Member Functions inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| void | add_dependency (std::shared_ptr< ComputeOperation > dep) |
| output_type | apply_operation (const input_type &input) |
| Public synchronous execution interface. | |
| output_type | apply_operation_with_dependencies (const input_type &input) |
| Applies the operation with dependencies resolved. | |
| OutputType | apply_to_data (const InputType &data) |
| Convenience overload that extracts just the data from result. | |
| ComputeOperation () | |
| Constructor with data type validation warnings. | |
| template<typename Config> requires std::derived_from<Config, OperationConfig> | |
| const Config & | config () const |
| The stored config struct of type Config, or its defaults when none was set. | |
| output_type | execute (const input_type &input) |
| OpUnit interface - operations can act as units in dependency graphs. | |
| const auto & | get_dependencies () const |
| const ExecutionContext & | get_last_execution_context () const |
| void | merge_config (const Facets &incoming) |
Adds the config structs of incoming, replacing stored ones of the same type. | |
| output_type | operator() (const InputType &data) |
| Convenience overload for direct data processing (backward compatibility). | |
| template<typename Config> requires std::derived_from<Config, OperationConfig> | |
| const Config * | resolved_config () const |
| The stored config struct of type Config, or nullptr when absent. | |
| template<typename Config> requires std::derived_from<Config, OperationConfig> | |
| void | set_config (Config config) |
| Stores a config struct, whole, under its own type. | |
| void | set_last_execution_context (const ExecutionContext &ctx) |
| void | set_post_execution_hook (const OperationHookCallback &hook) |
| void | set_pre_execution_hook (const OperationHookCallback &hook) |
| void | set_reconstruction_callback (const ReconstructionCallback &callback) |
| template<typename... Configs> | |
| ConfigRefs | stored_configs (Supports< Configs... >) const |
| The stored configs among those listed, in listed order. | |
| virtual | ~ComputeOperation ()=default |
| Virtual destructor for proper cleanup of derived classes. | |
Static Public Member Functions | |
| static std::string | diagnose (const VadAnalyzerConfig &config) |
| Why a config cannot be analysed. | |
| static float | fitness (const input_type &) |
| One while a device is up, zero otherwise. | |
Static Public Attributes | |
| static constexpr Target | target = targets::vulkan |
Protected Member Functions | |
| output_type | analyze_implementation (const input_type &input) override |
| Pure virtual analysis implementation - derived classes implement this. | |
| Protected Member Functions inherited from MayaFlux::Yantra::Analyzer< InputType, OutputType > | |
| virtual output_type | apply_granularity_formatting (const output_type &raw_output) |
| Output formatting by granularity (override for custom formatting). | |
| output_type | operation_function (const input_type &input) override |
| Core analysis implementation - must be overridden by derived classes. | |
| template<typename AnalysisResultType> | |
| void | store_current_analysis (AnalysisResultType &&result) const |
| virtual bool | validate_analysis_input (const input_type &) const |
| Input validation (override for custom validation logic). | |
| Protected Member Functions inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| output_type | apply_hooks (const input_type &input, const ExecutionContext &context) |
| virtual std::future< output_type > | apply_operation_async (const input_type &input) |
| Optional async implementation - default delegates to operation_function. | |
| virtual output_type | apply_operation_chained (const input_type &input, const ExecutionContext &ctx) |
| Optional chain-aware implementation - default delegates to operation_function. | |
| virtual output_type | apply_operation_internal (const input_type &input, const ExecutionContext &context) |
| Internal execution method - ComputeMatrix can access this. | |
| virtual output_type | apply_operation_parallel (const input_type &input, const ExecutionContext &ctx) |
| Optional parallel-aware implementation - default delegates to operation_function. | |
| output_type | convert_result (std::vector< std::vector< double > > &result_data, DataStructureInfo &metadata) |
| Convert processed double data back to OutputType using metadata and optional callback. | |
Private Member Functions | |
| SileroVadModel & | network () const |
Private Attributes | |
| size_t | m_chunks_done { 0 } |
| std::unique_ptr< SileroVadModel > | m_model |
| std::shared_ptr< GpuDispatchRecorder > | m_recording |
| std::shared_ptr< const Kakshya::TensorSet > | m_weights |
Additional Inherited Members | |
| Protected Attributes inherited from MayaFlux::Yantra::Analyzer< InputType, OutputType > | |
| std::any | m_current_analysis |
| std::optional< output_type > | m_current_output |
| Protected Attributes inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| ExecutionContext | m_last_execution_context |
Whether each 32 ms chunk of a mono 16 kHz stream is voice activity, from a learned network.
Where WaveformAnalyzer measures the shape of the waveform, this says what the sound is: a sung note and a fan hum are as periodic as a vowel, and only a learned model separates them from speech. The probabilities are the network's own, before any thresholding.
The analyzer is named for the task. Today it runs one network, Silero VAD at 16 kHz, recognised by its tensors; weights for any other network are rejected with the reason.
The analyzer is a stream. The recurrent state, the 64 samples of context before the next chunk and any partial chunk persist between calls, so successive calls continue one signal. Call reset for a new one. For a whole clip, set flush_tail so the final partial chunk is run too. One channel only.
The weights are an explicit dependency: the caller loads the network's tensors and passes them. The network runs in native float on the vulkan target and has no CPU twin. It is not registered with the default registry, because it has no selection to resolve by, and its result arrives a chunk late, so it must not run on the audio thread.
Definition at line 94 of file VadAnalyzer.hpp.