MayaFlux 1.0.0
Digital-First Multimedia Processing Framework
Loading...
Searching...
No Matches
MayaFlux::Yantra::VadAnalyzer< InputType, OutputType > Class Template Reference

Whether each 32 ms chunk of a mono 16 kHz stream is voice activity, from a learned network. More...

#include <VadAnalyzer.hpp>

Inheritance diagram for MayaFlux::Yantra::VadAnalyzer< InputType, OutputType >:
Collaboration diagram for MayaFlux::Yantra::VadAnalyzer< InputType, OutputType >:

Public Types

using base_type = Analyzer<InputType, OutputType>
using input_type = Datum<InputType>
using output_type = Datum<OutputType>
using supports = Supports<VadAnalyzerConfig>
Public Types inherited from MayaFlux::Yantra::Analyzer< InputType, OutputType >
using base_type = ComputeOperation<InputType, OutputType>
using input_type = Datum<InputType>
using output_type = Datum<OutputType>
Public Types inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType >
using input_type = Datum<InputType>
using output_type = Datum<OutputType>

Public Member Functions

VadAnalysis analyze_speech (const input_type &data)
 Type-safe speech probability analysis.
VadAnalysis analyze_speech (const InputType &data)
ConfigRefs configs () const override
 The base view of the config structs this operation holds.
AnalysisType get_analysis_type () const override
 Gets the analysis type category for this analyzer.
std::string get_name () const override
 Get operation name for debugging/introspection.
VadAnalysis get_speech_analysis () const
SileroVadModel::ProbabilityBuffer hidden_buffer ()
 Where the network's recurrent output of the latest run is on the device, SileroVadModel::hidden_width values per chunk.
SileroVadModel::ProbabilityBuffer probability_buffer ()
 Where the probabilities of the latest run are on the device, for a consumer on the same recording.
size_t record (std::span< const float > samples)
 Run every chunk the samples complete without waiting for the result.
void reset ()
 Start a new signal: forget the recurrent state, the context and any partial chunk.
size_t resident_count () const noexcept
 Probabilities the latest run left in probability_buffer.
void share_recording (std::shared_ptr< GpuDispatchRecorder > recording)
 Append to a recording the caller shares with other work.
 VadAnalyzer (const Kakshya::TensorSet &weights, VadAnalyzerConfig config={})
 Construct from the network weights.
Public Member Functions inherited from MayaFlux::Yantra::Analyzer< InputType, OutputType >
std::vector< std::any > analyze_batch (const std::vector< input_type > &inputs)
 Batch analysis for multiple inputs.
std::vector< std::any > analyze_batch (const std::vector< InputType > &inputs)
std::any analyze_data (const input_type &data)
 User-facing analysis method - returns analysis results directly.
std::any analyze_data (const InputType &data)
std::any get_current_analysis () const
 Access cached analysis from last operation.
OperationType get_operation_type () const override
 Returns the category of this operation for grammar and registry discovery.
bool has_current_analysis () const
 Check if analysis has been performed.
bool validate_input (const input_type &input) const override
 Validates input data meets analyzer requirements.
virtual ~Analyzer ()=default
Public Member Functions inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType >
void add_dependency (std::shared_ptr< ComputeOperation > dep)
output_type apply_operation (const input_type &input)
 Public synchronous execution interface.
output_type apply_operation_with_dependencies (const input_type &input)
 Applies the operation with dependencies resolved.
OutputType apply_to_data (const InputType &data)
 Convenience overload that extracts just the data from result.
 ComputeOperation ()
 Constructor with data type validation warnings.
template<typename Config>
requires std::derived_from<Config, OperationConfig>
const Config & config () const
 The stored config struct of type Config, or its defaults when none was set.
output_type execute (const input_type &input)
 OpUnit interface - operations can act as units in dependency graphs.
const auto & get_dependencies () const
const ExecutionContext & get_last_execution_context () const
void merge_config (const Facets &incoming)
 Adds the config structs of incoming, replacing stored ones of the same type.
output_type operator() (const InputType &data)
 Convenience overload for direct data processing (backward compatibility).
template<typename Config>
requires std::derived_from<Config, OperationConfig>
const Config * resolved_config () const
 The stored config struct of type Config, or nullptr when absent.
template<typename Config>
requires std::derived_from<Config, OperationConfig>
void set_config (Config config)
 Stores a config struct, whole, under its own type.
void set_last_execution_context (const ExecutionContext &ctx)
void set_post_execution_hook (const OperationHookCallback &hook)
void set_pre_execution_hook (const OperationHookCallback &hook)
void set_reconstruction_callback (const ReconstructionCallback &callback)
template<typename... Configs>
ConfigRefs stored_configs (Supports< Configs... >) const
 The stored configs among those listed, in listed order.
virtual ~ComputeOperation ()=default
 Virtual destructor for proper cleanup of derived classes.

Static Public Member Functions

static std::string diagnose (const VadAnalyzerConfig &config)
 Why a config cannot be analysed.
static float fitness (const input_type &)
 One while a device is up, zero otherwise.

Static Public Attributes

static constexpr Target target = targets::vulkan

Protected Member Functions

output_type analyze_implementation (const input_type &input) override
 Pure virtual analysis implementation - derived classes implement this.
Protected Member Functions inherited from MayaFlux::Yantra::Analyzer< InputType, OutputType >
virtual output_type apply_granularity_formatting (const output_type &raw_output)
 Output formatting by granularity (override for custom formatting).
output_type operation_function (const input_type &input) override
 Core analysis implementation - must be overridden by derived classes.
template<typename AnalysisResultType>
void store_current_analysis (AnalysisResultType &&result) const
virtual bool validate_analysis_input (const input_type &) const
 Input validation (override for custom validation logic).
Protected Member Functions inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType >
output_type apply_hooks (const input_type &input, const ExecutionContext &context)
virtual std::future< output_type > apply_operation_async (const input_type &input)
 Optional async implementation - default delegates to operation_function.
virtual output_type apply_operation_chained (const input_type &input, const ExecutionContext &ctx)
 Optional chain-aware implementation - default delegates to operation_function.
virtual output_type apply_operation_internal (const input_type &input, const ExecutionContext &context)
 Internal execution method - ComputeMatrix can access this.
virtual output_type apply_operation_parallel (const input_type &input, const ExecutionContext &ctx)
 Optional parallel-aware implementation - default delegates to operation_function.
output_type convert_result (std::vector< std::vector< double > > &result_data, DataStructureInfo &metadata)
 Convert processed double data back to OutputType using metadata and optional callback.

Private Member Functions

SileroVadModel & network () const

Private Attributes

size_t m_chunks_done { 0 }
std::unique_ptr< SileroVadModel > m_model
std::shared_ptr< GpuDispatchRecorder > m_recording
std::shared_ptr< const Kakshya::TensorSet > m_weights

Additional Inherited Members

Protected Attributes inherited from MayaFlux::Yantra::Analyzer< InputType, OutputType >
std::any m_current_analysis
std::optional< output_type > m_current_output
Protected Attributes inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType >
ExecutionContext m_last_execution_context

Detailed Description

template<ComputeData InputType = std::vector<Kakshya::DataVariant>, ComputeData OutputType = Eigen::MatrixXd>
class MayaFlux::Yantra::VadAnalyzer< InputType, OutputType >

Whether each 32 ms chunk of a mono 16 kHz stream is voice activity, from a learned network.

Where WaveformAnalyzer measures the shape of the waveform, this says what the sound is: a sung note and a fan hum are as periodic as a vowel, and only a learned model separates them from speech. The probabilities are the network's own, before any thresholding.

The analyzer is named for the task. Today it runs one network, Silero VAD at 16 kHz, recognised by its tensors; weights for any other network are rejected with the reason.

The analyzer is a stream. The recurrent state, the 64 samples of context before the next chunk and any partial chunk persist between calls, so successive calls continue one signal. Call reset for a new one. For a whole clip, set flush_tail so the final partial chunk is run too. One channel only.

The weights are an explicit dependency: the caller loads the network's tensors and passes them. The network runs in native float on the vulkan target and has no CPU twin. It is not registered with the default registry, because it has no selection to resolve by, and its result arrives a chunk late, so it must not run on the audio thread.

Definition at line 94 of file VadAnalyzer.hpp.


The documentation for this class was generated from the following file: