|
MayaFlux 1.0.0
Digital-First Multimedia Processing Framework
|
The phonemes of a whole mono 16 kHz recording, with the pauses between speech taken out. More...
#include <PhonemeExtractor.hpp>
Classes | |
| struct | Piece |
Public Types | |
| using | base_type = Extractor<InputType, OutputType> |
| using | input_type = Datum<InputType> |
| using | output_type = Datum<OutputType> |
| using | supports = Supports<PhonemeExtractorConfig> |
| Public Types inherited from MayaFlux::Yantra::Extractor< InputType, OutputType > | |
| using | base_type = ComputeOperation<InputType, OutputType> |
| using | input_type = Datum<InputType> |
| using | output_type = Datum<OutputType> |
| Public Types inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| using | input_type = Datum<InputType> |
| using | output_type = Datum<OutputType> |
Public Member Functions | |
| ConfigRefs | configs () const override |
| The base view of the config structs this operation holds. | |
| PhonemeExtraction | extract_phonemes (const input_type &data) |
| Type-safe phoneme extraction. | |
| PhonemeExtraction | extract_phonemes (const InputType &data) |
| ExtractionType | get_extraction_type () const override |
| Gets the extraction type category for this extractor. | |
| PhonemeExtractor (const Kakshya::TensorSet &vad_weights, const Kakshya::TensorSet &phoneme_weights, PhonemeExtractorConfig config={}) | |
| Construct from the weights of both networks. | |
| Public Member Functions inherited from MayaFlux::Yantra::Extractor< InputType, OutputType > | |
| std::vector< OutputType > | extract_batch (const std::vector< input_type > &inputs) |
| Batch extraction for multiple inputs. | |
| std::vector< OutputType > | extract_batch (const std::vector< InputType > &inputs) |
| OutputType | extract_data (const input_type &data) |
| Type-safe extraction method. | |
| OutputType | extract_data (const InputType &data) |
| OutputType | extract_with_scope (const input_type &data, ExtractionScope scope) |
| Extract with specific scope, without changing the stored settings. | |
| OutputType | extract_with_scope (const InputType &data, ExtractionScope scope) |
| std::string | get_name () const override |
| Gets human-readable name for this extractor. | |
| OperationType | get_operation_type () const override |
| Returns the category of this operation for grammar and registry discovery. | |
| virtual | ~Extractor ()=default |
| Public Member Functions inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| void | add_dependency (std::shared_ptr< ComputeOperation > dep) |
| output_type | apply_operation (const input_type &input) |
| Public synchronous execution interface. | |
| output_type | apply_operation_with_dependencies (const input_type &input) |
| Applies the operation with dependencies resolved. | |
| OutputType | apply_to_data (const InputType &data) |
| Convenience overload that extracts just the data from result. | |
| ComputeOperation () | |
| Constructor with data type validation warnings. | |
| template<typename Config> requires std::derived_from<Config, OperationConfig> | |
| const Config & | config () const |
| The stored config struct of type Config, or its defaults when none was set. | |
| output_type | execute (const input_type &input) |
| OpUnit interface - operations can act as units in dependency graphs. | |
| const auto & | get_dependencies () const |
| const ExecutionContext & | get_last_execution_context () const |
| void | merge_config (const Facets &incoming) |
Adds the config structs of incoming, replacing stored ones of the same type. | |
| output_type | operator() (const InputType &data) |
| Convenience overload for direct data processing (backward compatibility). | |
| template<typename Config> requires std::derived_from<Config, OperationConfig> | |
| const Config * | resolved_config () const |
| The stored config struct of type Config, or nullptr when absent. | |
| template<typename Config> requires std::derived_from<Config, OperationConfig> | |
| void | set_config (Config config) |
| Stores a config struct, whole, under its own type. | |
| void | set_last_execution_context (const ExecutionContext &ctx) |
| void | set_post_execution_hook (const OperationHookCallback &hook) |
| void | set_pre_execution_hook (const OperationHookCallback &hook) |
| void | set_reconstruction_callback (const ReconstructionCallback &callback) |
| template<typename... Configs> | |
| ConfigRefs | stored_configs (Supports< Configs... >) const |
| The stored configs among those listed, in listed order. | |
| virtual bool | validate_input (const input_type &) const |
| Validates if the input data meets the operation's requirements. | |
| virtual | ~ComputeOperation ()=default |
| Virtual destructor for proper cleanup of derived classes. | |
Static Public Member Functions | |
| static std::string | diagnose (const PhonemeExtractorConfig &config) |
| Why a config cannot be extracted with. | |
| static float | fitness (const input_type &) |
| One while a device is up, zero otherwise. | |
Static Public Attributes | |
| static constexpr Target | target = targets::vulkan |
Protected Member Functions | |
| output_type | extract_implementation (const input_type &input) override |
| Pure virtual extraction implementation - derived classes implement this. | |
| std::string | get_extractor_name () const override |
| Get extractor-specific name (derived classes override this). | |
| Protected Member Functions inherited from MayaFlux::Yantra::Extractor< InputType, OutputType > | |
| virtual output_type | apply_content_filtering (const output_type &raw_output) |
| Apply content-based filtering (override for custom filtering). | |
| virtual output_type | apply_data_sampling (const output_type &raw_output) |
| Apply data sampling (override for custom sampling). | |
| virtual output_type | apply_scope_filtering (const output_type &raw_output, ExtractionScope scope) |
| Apply scope filtering to results. | |
| virtual ExtractionScope | extraction_scope () const |
| The scope the stored config selects; FULL_DATA unless a family overrides it. | |
| virtual output_type | filter_to_target_regions (const output_type &raw_output) |
| Filter results to target regions (override for custom filtering). | |
| output_type | operation_function (const input_type &input) override |
| Core operation implementation - called by ComputeOperation interface. | |
| virtual bool | validate_extraction_input (const input_type &) const |
| Input validation (override for custom validation logic). | |
| Protected Member Functions inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| output_type | apply_hooks (const input_type &input, const ExecutionContext &context) |
| virtual std::future< output_type > | apply_operation_async (const input_type &input) |
| Optional async implementation - default delegates to operation_function. | |
| virtual output_type | apply_operation_chained (const input_type &input, const ExecutionContext &ctx) |
| Optional chain-aware implementation - default delegates to operation_function. | |
| virtual output_type | apply_operation_internal (const input_type &input, const ExecutionContext &context) |
| Internal execution method - ComputeMatrix can access this. | |
| virtual output_type | apply_operation_parallel (const input_type &input, const ExecutionContext &ctx) |
| Optional parallel-aware implementation - default delegates to operation_function. | |
| output_type | convert_result (std::vector< std::vector< double > > &result_data, DataStructureInfo &metadata) |
| Convert processed double data back to OutputType using metadata and optional callback. | |
Private Member Functions | |
| output_type | build_output (const PhonemeExtraction &record, const std::vector< size_t > &rows, std::span< const double > samples, DataStructureInfo &info) |
| std::vector< std::pair< size_t, size_t > > | find_voiced (std::span< const double > samples, const PhonemeExtractorConfig &cfg) |
| void | recognise_phonemes (std::span< const double > samples, PhonemeExtraction &record) |
Static Private Member Functions | |
| static output_type | empty_output () |
| static bool | gate_applies (const PhonemeExtractorConfig &cfg) |
| static bool | is_selected (const PhonemeEvent &event, PhonemeSelection selection) |
| static PhonemeAnalyzerConfig | phoneme_settings (const PhonemeExtractorConfig &config) |
| static std::vector< Piece > | pieces_of (const PhonemeExtraction &record, const std::vector< size_t > &rows) |
| static std::vector< size_t > | selected_rows (const PhonemeExtraction &record) |
Private Attributes | |
| StandardPhonemeAnalyzer | m_phonemes |
| RegionSpeechSegmenter | m_speech |
| std::optional< StandardWaveformAnalyzer > | m_waveform |
| uint32_t | m_waveform_hop { 0 } |
Additional Inherited Members | |
| Protected Attributes inherited from MayaFlux::Yantra::ComputeOperation< InputType, OutputType > | |
| ExecutionContext | m_last_execution_context |
The phonemes of a whole mono 16 kHz recording, with the pauses between speech taken out.
Finds speech with a SpeechSegmenter, then runs the phoneme network once on each speech interval, so no call exceeds its 30 s limit and nothing runs on silence. Phoneme ranges are estimates (PhonemeEvent). Pauses shorter than min_silence stay inside a range.
OutputType selects the output: Kakshya::CompositeArray (records, the default), Kakshya::RegionGroup (regions) or std::vector<Kakshya::DataVariant> (audio of the selected phonemes). Blocking, one channel, not in the default registry. It must not run on the audio thread.
Definition at line 152 of file PhonemeExtractor.hpp.