|
MayaFlux 1.0.0
Digital-First Multimedia Processing Framework
|
The speech intervals of a whole mono 16 kHz recording. More...
#include <SpeechSegmenter.hpp>
Public Types | |
| using | base_type = Extractor<InputType, OutputType> |
| using | input_type = Datum<InputType> |
| using | output_type = Datum<OutputType> |
| using | supports = Supports<SpeechSegmenterConfig> |
| Public Types inherited from MayaFlux::Yantra::Extractor< InputType, OutputType > | |
| using | base_type = ComputeOperation<InputType, OutputType> |
| using | input_type = Datum<InputType> |
| using | output_type = Datum<OutputType> |
| Public Types inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| using | input_type = Datum<InputType> |
| using | output_type = Datum<OutputType> |
Public Member Functions | |
| ConfigRefs | configs () const override |
| The base view of the config structs this operation holds. | |
| SpeechSegmentation | extract_segments (const input_type &data) |
| Type-safe segmentation. | |
| SpeechSegmentation | extract_segments (const InputType &data) |
| ExtractionType | get_extraction_type () const override |
| Gets the extraction type category for this extractor. | |
| SpeechSegmentation | segment (std::span< const double > samples, const SpeechSegmenterConfig &config) |
| Segment samples under the given settings, without the Datum wrapper. | |
| SpeechSegmenter (const Kakshya::TensorSet &vad_weights, SpeechSegmenterConfig config={}) | |
| Construct from the weights of the speech network. | |
| Public Member Functions inherited from MayaFlux::Yantra::Extractor< InputType, OutputType > | |
| std::vector< OutputType > | extract_batch (const std::vector< input_type > &inputs) |
| Batch extraction for multiple inputs. | |
| std::vector< OutputType > | extract_batch (const std::vector< InputType > &inputs) |
| OutputType | extract_data (const input_type &data) |
| Type-safe extraction method. | |
| OutputType | extract_data (const InputType &data) |
| OutputType | extract_with_scope (const input_type &data, ExtractionScope scope) |
| Extract with specific scope, without changing the stored settings. | |
| OutputType | extract_with_scope (const InputType &data, ExtractionScope scope) |
| std::string | get_name () const override |
| Gets human-readable name for this extractor. | |
| OperationType | get_operation_type () const override |
| Returns the category of this operation for grammar and registry discovery. | |
| virtual | ~Extractor ()=default |
| Public Member Functions inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| void | add_dependency (std::shared_ptr< ComputeOperation > dep) |
| output_type | apply_operation (const input_type &input) |
| Public synchronous execution interface. | |
| output_type | apply_operation_with_dependencies (const input_type &input) |
| Applies the operation with dependencies resolved. | |
| OutputType | apply_to_data (const InputType &data) |
| Convenience overload that extracts just the data from result. | |
| ComputeOperation () | |
| Constructor with data type validation warnings. | |
| template<typename Config> requires std::derived_from<Config, OperationConfig> | |
| const Config & | config () const |
| The stored config struct of type Config, or its defaults when none was set. | |
| output_type | execute (const input_type &input) |
| OpUnit interface - operations can act as units in dependency graphs. | |
| const auto & | get_dependencies () const |
| const ExecutionContext & | get_last_execution_context () const |
| void | merge_config (const Facets &incoming) |
Adds the config structs of incoming, replacing stored ones of the same type. | |
| output_type | operator() (const InputType &data) |
| Convenience overload for direct data processing (backward compatibility). | |
| template<typename Config> requires std::derived_from<Config, OperationConfig> | |
| const Config * | resolved_config () const |
| The stored config struct of type Config, or nullptr when absent. | |
| template<typename Config> requires std::derived_from<Config, OperationConfig> | |
| void | set_config (Config config) |
| Stores a config struct, whole, under its own type. | |
| void | set_last_execution_context (const ExecutionContext &ctx) |
| void | set_post_execution_hook (const OperationHookCallback &hook) |
| void | set_pre_execution_hook (const OperationHookCallback &hook) |
| void | set_reconstruction_callback (const ReconstructionCallback &callback) |
| template<typename... Configs> | |
| ConfigRefs | stored_configs (Supports< Configs... >) const |
| The stored configs among those listed, in listed order. | |
| virtual bool | validate_input (const input_type &) const |
| Validates if the input data meets the operation's requirements. | |
| virtual | ~ComputeOperation ()=default |
| Virtual destructor for proper cleanup of derived classes. | |
Static Public Member Functions | |
| static std::string | diagnose (const SpeechSegmenterConfig &config) |
| Why a config cannot be segmented with. | |
| static float | fitness (const input_type &) |
| One while a device is up, zero otherwise. | |
Static Public Attributes | |
| static constexpr Target | target = targets::vulkan |
Protected Member Functions | |
| output_type | extract_implementation (const input_type &input) override |
| Pure virtual extraction implementation - derived classes implement this. | |
| std::string | get_extractor_name () const override |
| Get extractor-specific name (derived classes override this). | |
| Protected Member Functions inherited from MayaFlux::Yantra::Extractor< InputType, OutputType > | |
| virtual output_type | apply_content_filtering (const output_type &raw_output) |
| Apply content-based filtering (override for custom filtering). | |
| virtual output_type | apply_data_sampling (const output_type &raw_output) |
| Apply data sampling (override for custom sampling). | |
| virtual output_type | apply_scope_filtering (const output_type &raw_output, ExtractionScope scope) |
| Apply scope filtering to results. | |
| virtual ExtractionScope | extraction_scope () const |
| The scope the stored config selects; FULL_DATA unless a family overrides it. | |
| virtual output_type | filter_to_target_regions (const output_type &raw_output) |
| Filter results to target regions (override for custom filtering). | |
| output_type | operation_function (const input_type &input) override |
| Core operation implementation - called by ComputeOperation interface. | |
| virtual bool | validate_extraction_input (const input_type &) const |
| Input validation (override for custom validation logic). | |
| Protected Member Functions inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| output_type | apply_hooks (const input_type &input, const ExecutionContext &context) |
| virtual std::future< output_type > | apply_operation_async (const input_type &input) |
| Optional async implementation - default delegates to operation_function. | |
| virtual output_type | apply_operation_chained (const input_type &input, const ExecutionContext &ctx) |
| Optional chain-aware implementation - default delegates to operation_function. | |
| virtual output_type | apply_operation_internal (const input_type &input, const ExecutionContext &context) |
| Internal execution method - ComputeMatrix can access this. | |
| virtual output_type | apply_operation_parallel (const input_type &input, const ExecutionContext &ctx) |
| Optional parallel-aware implementation - default delegates to operation_function. | |
| output_type | convert_result (std::vector< std::vector< double > > &result_data, DataStructureInfo &metadata) |
| Convert processed double data back to OutputType using metadata and optional callback. | |
Private Member Functions | |
| output_type | build_output (const SpeechSegmentation &record, std::span< const double > samples, DataStructureInfo &info) |
| void | find_speech (std::span< const double > samples, const SpeechSegmenterConfig &cfg, SpeechSegmentation &record) |
| void | measure_speech (std::span< const double > samples, const SpeechSegmenterConfig &cfg, SpeechSegmentation &record) |
Static Private Member Functions | |
| static VadAnalyzerConfig | vad_settings (const SpeechSegmenterConfig &config) |
Private Attributes | |
| MatrixIntervalExtractor | m_intervals |
| StandardVadAnalyzer | m_vad |
| std::optional< StandardWaveformAnalyzer > | m_waveform |
Additional Inherited Members | |
| Protected Attributes inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| ExecutionContext | m_last_execution_context |
The speech intervals of a whole mono 16 kHz recording.
Runs Silero VAD over the recording and thresholds its probabilities into speech intervals. With SpeechSettings::measure it also runs a WaveformAnalyzer once over the recording and reports, per interval, the share of voiced windows and their median frequency. Without it no waveform analysis is made.
Voiced means periodic in the 70 to 400 Hz band, which a hum or a sung note also is; the speech intervals are what say the sound is speech.
OutputType selects the output: Kakshya::RegionGroup (one region per speech interval, the default) or std::vector<Kakshya::DataVariant> (the speech audio, concatenated). The facet SpeechSegmentation is always set. Blocking, one channel, not in the default registry. It must not run on the audio thread.
Definition at line 232 of file SpeechSegmenter.hpp.