MayaFlux 1.0.0
Digital-First Multimedia Processing Framework
Loading...
Searching...
No Matches
IntervalExtractor.hpp
Go to the documentation of this file.
1#pragma once
2
4#include "Extractor.hpp"
5
11
13
14/**
15 * @file IntervalExtractor.hpp
16 * @brief Runs of a score sequence that stay above a threshold, found on the GPU in order.
17 *
18 * The scores can come from any per-window measurement: a probability, a level,
19 * a flux. The extractor knows nothing about what they mean. It answers where
20 * each column is engaged, as sample intervals that line up with the output of
21 * merge_intervals and slice_intervals. Each column has its own settings.
22 */
23
24namespace MayaFlux::Yantra {
25
26/**
27 * @enum IntervalMethod
28 * @brief How the score sequence is turned into intervals.
29 */
30enum class IntervalMethod : uint8_t {
31 HYSTERESIS_RUNS ///< Engage at or above the enter threshold, release below the leave threshold after a minimum gap
32};
33
34/**
35 * @struct IntervalScanSettings
36 * @brief How one score column is turned into intervals.
37 *
38 * Lengths are in samples and positions are 32 bit. Score i of the column sits
39 * at first_sample + i * samples_per_score. The leave threshold is
40 * max(enter_threshold - leave_margin, leave_floor). A run shorter than min_run
41 * is dropped. A gap shorter than min_gap does not end a run. Each interval is
42 * widened by padding on both sides, and a gap between neighbours smaller than
43 * twice the padding is split between them.
44 *
45 * max_run caps one interval, 0 for no cap. A capped run is cut at its longest
46 * gap when use_longest_gap is set, else at the last gap longer than
47 * gap_at_limit, else where the cap falls. It must exceed samples_per_score plus
48 * twice the padding when set.
49 *
50 * gate_channel names the column that gates this one, or no_channel for none.
51 * Where the gate's score is below gate_threshold this column is silent: an open
52 * run ends there at once whatever min_gap is, and a gap is bridged only while
53 * the gate stays at or above its threshold. The gate is another column of the
54 * same call, of the same length, and is scanned by its own settings. A column is
55 * never gated by itself.
56 *
57 * emit_on_close releases each interval when it closes, padded by the full
58 * padding on both sides. Nothing waits for a successor, so neighbouring paddings
59 * may overlap.
60 */
62 /** Value of gate_channel for no gate. */
63 static constexpr uint32_t no_channel { std::numeric_limits<uint32_t>::max() };
64
65 /** Score at or above which a run starts. */
66 double enter_threshold { 0.5 };
67 /** How far below the enter threshold a score must fall to count as a gap. */
68 double leave_margin { 0.15 };
69 /** Lowest leave threshold. */
70 double leave_floor { 0.01 };
71 /** Samples between successive scores. */
72 uint32_t samples_per_score { 512 };
73 /** Sample position of score 0. */
74 uint32_t first_sample { 0 };
75 /** Shortest run kept, in samples. */
76 uint32_t min_run { 4000 };
77 /** Shortest gap that ends a run, in samples. */
78 uint32_t min_gap { 1600 };
79 /** Samples added on each side of an interval. */
80 uint32_t padding { 480 };
81 /** Gaps longer than this are candidates for cutting a capped run, in samples. */
82 uint32_t gap_at_limit { 1568 };
83 /** Longest interval in samples, 0 for no cap. */
84 uint32_t max_run { 0 };
85 /** Cut a capped run at its longest gap instead of the last one. */
86 bool use_longest_gap { true };
87 /** Column that gates this one, or no_channel. */
89 /** Gate score below which this column is silent. */
90 double gate_threshold { 0.5 };
91 /** Emit each interval as soon as it closes, with full padding on both sides. */
92 bool emit_on_close { false };
93};
94
95/**
96 * @struct IntervalExtractorConfig
97 * @brief Settings of an IntervalExtractor.
98 *
99 * scan is the settings of every column. columns overrides them: column i uses
100 * columns[i] when the list has that entry, and scan otherwise. flush ends the
101 * stream: the open run of each column is closed at length, or at the end of its
102 * scores when length is 0, and the pending interval is emitted. Without flush
103 * the newest interval of each column stays pending until the next call. Both
104 * are the same for every column.
105 */
107 /** Selection method, as an IntervalMethod value. */
109 /** Settings of every column not listed in columns. */
111 /** Per column overrides, column i for index i. */
112 std::vector<IntervalScanSettings> columns {};
113 /** End the stream with this call. */
114 bool flush { false };
115 /** Length of the stream in samples when it ends, 0 to use the scores seen. */
116 uint32_t length { 0 };
117
118 /** @brief Settings in force for a column. */
119 [[nodiscard]] const IntervalScanSettings& settings_of(size_t column) const noexcept
120 {
121 return column < columns.size() ? columns[column] : scan;
122 }
123};
124
125MF_DESCRIBE(IntervalExtractorConfig)
126
127/**
128 * @brief The host scan configuration of one column's settings over a stream of the given length.
129 */
130[[nodiscard]] inline Kinesis::Discrete::IntervalScanConfig interval_scan_config(const IntervalScanSettings& settings, uint32_t length)
131{
132 return {
133 .enter = settings.enter_threshold,
134 .leave = std::max(settings.enter_threshold - settings.leave_margin, settings.leave_floor),
135 .gate_threshold = settings.gate_threshold,
136 .hop = settings.samples_per_score,
137 .first_sample = settings.first_sample,
138 .min_run = settings.min_run,
139 .min_gap = settings.min_gap,
140 .padding = settings.padding,
141 .gap_at_limit = settings.gap_at_limit,
142 .max_run = settings.max_run,
143 .use_longest_gap = settings.use_longest_gap,
144 .length = length,
145 .emit_on_close = settings.emit_on_close,
146 };
147}
148
149/**
150 * @struct ChannelIntervals
151 * @brief Intervals of one channel, half open sample ranges in stream order.
152 *
153 * open_start and pending describe the state after the last score of the call,
154 * so a live caller need not wait for a run to close. open_start is where the
155 * run in progress began, set from the first score at or above the enter
156 * threshold until the run closes, and so also during a gap not yet long enough
157 * to end it. pending is the closed interval waiting for its padded end: its
158 * padded start and its end before trailing padding. Both are empty after a
159 * flush, and with emit_on_close nothing is ever pending. A call that gives a
160 * channel no scores and does not flush reports neither.
161 */
162struct MAYAFLUX_API ChannelIntervals {
163 std::vector<std::pair<size_t, size_t>> intervals;
164 std::optional<size_t> open_start;
165 std::optional<std::pair<size_t, size_t>> pending;
166};
167
168/**
169 * @struct IntervalBuffers
170 * @brief Device views of one channel's results from the latest call, for a consumer that reads them on the GPU.
171 *
172 * runs holds start and end pairs and report holds interval_scan_report_words
173 * words that decode_interval_scan_report reads, all uint32. The addresses are
174 * device addresses for push constants. The extractor owns the memory and
175 * rewrites it on its next call, so a consumer recorded between two calls, in a
176 * submission made after the extractor's own, reads the latest results:
177 * submissions on one queue run in order. Addresses are 0 for a channel the call
178 * gave no scores.
179 */
180struct MAYAFLUX_API IntervalBuffers {
181 uint64_t runs_address { 0 };
182 uint64_t report_address { 0 };
183 /** Pairs the run list can hold. */
184 uint32_t capacity { 0 };
185 std::shared_ptr<Portal::Graphics::GpuBufferHandle> runs;
186 std::shared_ptr<Portal::Graphics::GpuBufferHandle> report;
187};
188
189/**
190 * @struct IntervalExtraction
191 * @brief Record of one IntervalExtractor run, stored on the output's facets.
192 */
193struct MAYAFLUX_API IntervalExtraction {
194 /** Settings the extraction was produced under. */
196 /** Intervals completed by this call, per score channel. */
197 std::vector<ChannelIntervals> channels;
198};
199
201
202/**
203 * @brief One region per interval, spanning the given channels.
204 * @param intervals Half open sample ranges. Empty ranges are skipped.
205 * @param first_channel First channel the regions cover.
206 * @param last_channel Last channel the regions cover, inclusive.
207 * @param name Name of the group.
208 */
210 const ChannelIntervals& intervals, uint32_t first_channel, uint32_t last_channel, std::string name = "intervals")
211{
212 Kakshya::RegionGroup group(std::move(name));
213 for (const auto& [start, end] : intervals.intervals) {
214 if (end > start) {
215 group.add_region(Kakshya::Region::audio_span(start, end - 1, first_channel, last_channel));
216 }
217 }
218 return group;
219}
220
221/**
222 * @class IntervalExtractor
223 * @brief Finds where each score sequence is engaged, on the GPU, as a stream.
224 *
225 * Every score depends on the state the previous one left, so one invocation
226 * reads a column's scores in order. Each column is its own scan with its own
227 * settings and its own state, and that state lives on the device and persists
228 * between calls, so successive calls continue one stream. The newest interval
229 * waits in the state until its successor closes, because its padded end depends
230 * on the gap that follows; set flush on the last call to release it. A flush
231 * ends the stream and the next call starts a new one. Call reset to start over
232 * without flushing.
233 *
234 * The input is the matrix an analyzer returns, one score column per channel, so
235 * an analyzer such as VadAnalyzer or PhonemeAnalyzer chains into it directly.
236 * Other input types are instantiations of the same template.
237 *
238 * It is on the vulkan target. Kinesis::Discrete::IntervalScanner is its CPU
239 * counterpart for scores in host memory and gives the same intervals, column by
240 * column. The output data is, per channel, the start and end of each completed
241 * interval in turn; the IntervalExtraction facet holds the same as pairs.
242 */
243template <ComputeData InputType = Eigen::MatrixXd, ComputeData OutputType = std::vector<std::vector<double>>>
244class IntervalExtractor : public Extractor<InputType, OutputType> {
245public:
250
251 static constexpr Target target = targets::vulkan;
252 static constexpr bool stateful = true;
253
254 /**
255 * @brief Construct an interval extractor from its settings.
256 * @param config Extraction settings, validated before they are stored.
257 */
259 {
260 validate_config(config);
261 this->set_config(std::move(config));
262 }
263
264 [[nodiscard]] ConfigRefs configs() const override
265 {
266 return this->stored_configs(supports {});
267 }
268
269 [[nodiscard]] ExtractionType get_extraction_type() const override
270 {
272 }
273
274 /**
275 * @brief Type-safe interval extraction
276 * @return The intervals completed by this call.
277 */
279 {
280 const auto result = this->apply_operation(data);
281 const auto* record = result.facets.template get<IntervalExtraction>();
282 if (record == nullptr) {
284 std::source_location::current(), "Interval extraction produced no result");
285 }
286 return *record;
287 }
288
290 {
291 return this->extract_intervals(input_type { data });
292 }
293
294 /**
295 * @brief Record and submit an extraction without waiting for it.
296 *
297 * For a live caller that must not block: the work is on the queue when this
298 * returns. Poll ready, then collect the result. Only one submission can be
299 * outstanding, and a synchronous extract_intervals is refused while one is.
300 *
301 * @return False when an earlier submission has not been collected, or the
302 * input could not be submitted.
303 */
304 bool submit(const input_type& data)
305 {
306 if (m_outstanding) {
307 return false;
308 }
309 try {
310 const auto& cfg = this->template config<IntervalExtractorConfig>();
311 [[maybe_unused]] auto [numeric_data, info] = OperationHelper::extract_structured_double(const_cast<input_type&>(data));
312 auto calls = record_all(numeric_data, cfg);
313 m_outstanding = Outstanding { .calls = std::move(calls), .config = cfg, .fence = m_recording->submit() };
314 return true;
315 } catch (const std::exception& e) {
316 MF_ERROR(Journal::Component::Yantra, Journal::Context::ComputeMatrix, "Interval submit failed: {}", e.what());
317 return false;
318 }
319 }
320
321 bool submit(const InputType& data)
322 {
323 return this->submit(input_type { data });
324 }
325
326 /**
327 * @brief Append to a recording the caller shares with other work.
328 *
329 * With a shared recording the extractor's scans go to the device in the same
330 * submission as the work that produced their scores. Call it before anything
331 * creates the extractor's context, that is before the first extraction or
332 * bind. The recording must be compute isolated.
333 *
334 * @throws std::logic_error when the extractor has already started.
335 */
336 void share_recording(std::shared_ptr<GpuDispatchRecorder> recording)
337 {
338 if (m_context) {
340 std::source_location::current(), "The recording can only be shared before the extractor starts");
341 }
342 m_recording = std::move(recording);
343 }
344
345 /**
346 * @brief Read the scores of a channel from a device buffer instead of uploading them.
347 *
348 * The buffer holds float32 scores and belongs to a context on the same
349 * recording, for instance SileroVadModel::probability_buffer. The scan then
350 * reads it in place. A channel that gates others is bound the same way: its
351 * scores are the gate. Run it with submit_resident or extract_resident.
352 *
353 * @param channel Channel whose scores the buffer holds.
354 * @param source Context that owns the buffer.
355 * @param set Set of the buffer.
356 * @param binding Binding of the buffer.
357 * @throws std::invalid_argument when the buffer is missing, not float32 sized, or on another recording.
358 */
359 void bind_scores(size_t channel, const GpuDispatchCore& source, uint32_t set, size_t binding)
360 {
362 m_context->import_shared(0, binding_of(channel, Scores), source, set, binding, GpuBufferBinding::ElementType::FLOAT32);
363 if (m_resident.size() <= channel) {
364 m_resident.resize(channel + 1, false);
365 }
366 m_resident[channel] = true;
367 }
368
369 /**
370 * @brief Record and submit an extraction over scores already on the device, without waiting.
371 *
372 * Everything recorded so far on the shared recording goes in the same
373 * submission, so a network recorded just before this call and the scans run
374 * together. Poll ready and collect as for submit.
375 *
376 * @param counts Number of scores in each bound channel for this call.
377 * @return False when an earlier submission has not been collected, or the call could not be recorded.
378 */
379 bool submit_resident(std::span<const uint32_t> counts)
380 {
381 if (m_outstanding) {
382 return false;
383 }
384 try {
385 const auto& cfg = this->template config<IntervalExtractorConfig>();
386 auto calls = record_resident(counts, cfg);
387 m_outstanding = Outstanding { .calls = std::move(calls), .config = cfg, .fence = m_recording->submit() };
388 return true;
389 } catch (const std::exception& e) {
390 MF_ERROR(Journal::Component::Yantra, Journal::Context::ComputeMatrix, "Interval submit failed: {}", e.what());
391 return false;
392 }
393 }
394
395 /**
396 * @brief Extraction over scores already on the device, waiting for the result.
397 * @param counts Number of scores in each bound channel for this call.
398 * @return The intervals completed by this call.
399 */
400 IntervalExtraction extract_resident(std::span<const uint32_t> counts)
401 {
402 if (m_outstanding) {
404 std::source_location::current(), "An earlier submit has not been collected");
405 }
406 const auto& cfg = this->template config<IntervalExtractorConfig>();
407 const auto calls = record_resident(counts, cfg);
408
410 record.config = cfg;
411 record.channels.resize(counts.size());
412 for (size_t channel = 0; channel < counts.size(); ++channel) {
413 record.channels[channel] = read_channel(channel, calls[channel]);
414 }
415 return record;
416 }
417
418 /**
419 * @brief True while a submission is outstanding and not yet collected.
420 */
421 [[nodiscard]] bool pending() const
422 {
423 return m_outstanding.has_value();
424 }
425
426 /**
427 * @brief True when nothing is outstanding or the outstanding submission has completed. Never waits.
428 */
429 [[nodiscard]] bool ready() const
430 {
431 return !m_outstanding || m_recording->is_complete(m_outstanding->fence);
432 }
433
434 /**
435 * @brief The result of the outstanding submission, once it has completed.
436 * @return The record, or nothing when none is outstanding or it has not completed yet. Never waits.
437 */
438 [[nodiscard]] std::optional<IntervalExtraction> collect()
439 {
440 if (!m_outstanding || !ready()) {
441 return std::nullopt;
442 }
444 record.config = m_outstanding->config;
445 record.channels.resize(m_outstanding->calls.size());
446 for (size_t channel = 0; channel < m_outstanding->calls.size(); ++channel) {
447 record.channels[channel] = read_channel(channel, m_outstanding->calls[channel]);
448 }
449 m_outstanding.reset();
450 return record;
451 }
452
453 /**
454 * @brief Device views of the results of the latest call, one per channel.
455 *
456 * Valid as soon as the call has recorded, so after submit they can be
457 * handed to a consumer before the work completes. Empty before any call.
458 */
459 [[nodiscard]] std::vector<IntervalBuffers> buffers() const
460 {
461 std::vector<IntervalBuffers> views(m_last_calls.size());
462 for (size_t channel = 0; channel < m_last_calls.size(); ++channel) {
463 if (!m_last_calls[channel].ran) {
464 continue;
465 }
466 const size_t runs = binding_of(channel, Runs);
467 const size_t report = binding_of(channel, Report);
468 views[channel] = {
469 .runs_address = m_context->shared_buffer_address(0, runs),
470 .report_address = m_context->shared_buffer_address(0, report),
471 .capacity = m_last_calls[channel].capacity,
472 .runs = std::make_shared<Portal::Graphics::GpuBufferHandle>(m_context->shared_buffer_handle(0, runs)),
473 .report = std::make_shared<Portal::Graphics::GpuBufferHandle>(m_context->shared_buffer_handle(0, report)),
474 };
475 }
476 return views;
477 }
478
479 /**
480 * @brief Start a new stream: forget the detector state of every channel.
481 *
482 * Waits for an outstanding submission and discards its result.
483 */
484 void reset()
485 {
486 if (m_recording) {
487 m_recording->wait();
488 }
489 m_outstanding.reset();
490 m_last_calls.clear();
491 m_seen.clear();
492 m_started.clear();
493 }
494
495 /**
496 * @brief One while a device is up, zero otherwise.
497 */
498 static float fitness(const input_type& /*input*/)
499 {
501 }
502
503protected:
504 [[nodiscard]] std::string get_extractor_name() const override
505 {
506 return "IntervalExtractor";
507 }
508
510 {
511 try {
512 const auto& cfg = this->template config<IntervalExtractorConfig>();
513
514 auto [numeric_data, info] = OperationHelper::extract_structured_double(const_cast<input_type&>(input));
515 DataStructureInfo structure_info = info;
516
517 if (m_outstanding) {
519 std::source_location::current(), "An earlier submit has not been collected");
520 }
521 const auto calls = record_all(numeric_data, cfg);
522
524 record.config = cfg;
525 record.channels.resize(numeric_data.size());
526
527 std::vector<std::vector<double>> flat(numeric_data.size());
528 for (size_t channel = 0; channel < numeric_data.size(); ++channel) {
529 record.channels[channel] = read_channel(channel, calls[channel]);
530 for (const auto& [start, end] : record.channels[channel].intervals) {
531 flat[channel].push_back(static_cast<double>(start));
532 flat[channel].push_back(static_cast<double>(end));
533 }
534 }
535
536 output_type output = this->convert_result(flat, structure_info);
537 output.metadata = input.metadata;
538 output.facets = input.facets;
539 output.facets.set(record);
540 return output;
541 } catch (const std::exception& e) {
542 MF_ERROR(Journal::Component::Yantra, Journal::Context::ComputeMatrix, "Interval extraction failed: {}", e.what());
543 output_type error_result;
544 error_result.metadata = input.metadata;
545 error_result.facets = input.facets;
546 return error_result;
547 }
548 }
549
550private:
558
559 struct ChannelCall {
560 bool ran { false };
561 uint32_t capacity { 0 };
562 uint32_t first { 0 };
563 uint32_t count { 0 };
564 };
565
571
572 [[nodiscard]] static size_t binding_of(size_t channel, Slot slot)
573 {
574 return channel * SlotsPerChannel + slot;
575 }
576
577 mutable std::shared_ptr<GpuDispatchRecorder> m_recording;
578 mutable std::unique_ptr<ShaderExecutionContext<>> m_context;
579 std::vector<uint32_t> m_seen;
580 std::vector<bool> m_started;
581 std::vector<bool> m_resident;
582 std::vector<ChannelCall> m_last_calls;
583 std::optional<Outstanding> m_outstanding;
584
585 template <typename Columns>
586 std::vector<ChannelCall> record_all(const Columns& columns, const IntervalExtractorConfig& cfg)
587 {
589 reset_state(columns.size());
590
591 for (size_t channel = 0; channel < columns.size(); ++channel) {
592 const auto& settings = cfg.settings_of(channel);
593 check_settings(settings, cfg.length);
594 if (gated_by_other(settings, channel)) {
595 check_gate(settings.gate_channel, columns.size(), columns[channel].size());
596 if (columns[settings.gate_channel].size() != columns[channel].size()) {
598 std::source_location::current(), "Gate column {} has {} scores and column {} has {}",
599 settings.gate_channel, columns[settings.gate_channel].size(), channel, columns[channel].size());
600 }
601 }
602 }
603
604 std::vector<ChannelCall> calls(columns.size());
605 for (size_t channel = 0; channel < columns.size(); ++channel) {
606 const std::span<const double> scores { columns[channel].data(), columns[channel].size() };
607 calls[channel] = prepare_channel(channel, scores, false, 0, cfg.flush);
608 }
609 record_scans(calls, cfg);
610 m_last_calls = calls;
611 return calls;
612 }
613
614 std::vector<ChannelCall> record_resident(std::span<const uint32_t> counts, const IntervalExtractorConfig& cfg)
615 {
617 reset_state(counts.size());
618
619 for (size_t channel = 0; channel < counts.size(); ++channel) {
620 if (channel >= m_resident.size() || !m_resident[channel]) {
622 std::source_location::current(), "Channel {} has no device scores bound", channel);
623 }
624 const auto& settings = cfg.settings_of(channel);
625 check_settings(settings, cfg.length);
626 if (gated_by_other(settings, channel)) {
627 check_gate(settings.gate_channel, counts.size(), counts[channel]);
628 if (counts[settings.gate_channel] != counts[channel]) {
630 std::source_location::current(), "Gate column {} has {} scores and column {} has {}",
631 settings.gate_channel, counts[settings.gate_channel], channel, counts[channel]);
632 }
633 if (settings.gate_channel >= m_resident.size() || !m_resident[settings.gate_channel]) {
635 std::source_location::current(), "Gate column {} has no device scores bound", settings.gate_channel);
636 }
637 }
638 }
639
640 std::vector<ChannelCall> calls(counts.size());
641 for (size_t channel = 0; channel < counts.size(); ++channel) {
642 calls[channel] = prepare_channel(channel, {}, true, counts[channel], cfg.flush);
643 }
644 record_scans(calls, cfg);
645 m_last_calls = calls;
646 return calls;
647 }
648
649 static bool gated_by_other(const IntervalScanSettings& settings, size_t channel)
650 {
651 return settings.gate_channel != IntervalScanSettings::no_channel && settings.gate_channel != channel;
652 }
653
654 static void check_gate(uint32_t gate, size_t columns, size_t length)
655 {
656 if (gate >= columns) {
658 std::source_location::current(), "Gate column {} is not a column of the {} given, for a column of {} scores",
659 gate, columns, length);
660 }
661 }
662
663 void reset_state(size_t channels)
664 {
665 if (m_seen.size() != channels) {
666 m_seen.assign(channels, 0);
667 m_started.assign(channels, false);
668 }
669 }
670
671 static void check_settings(const IntervalScanSettings& settings, uint32_t length)
672 {
673 if (!std::isfinite(settings.leave_margin) || !std::isfinite(settings.leave_floor)) {
675 std::source_location::current(), "Thresholds must be finite");
676 }
677 if (const std::string reason = Kinesis::Discrete::diagnose_interval_scan(interval_scan_config(settings, length)); !reason.empty()) {
679 std::source_location::current(), "{}", reason);
680 }
681 }
682
684 {
685 check_settings(cfg.scan, cfg.length);
686 for (const auto& settings : cfg.columns) {
687 check_settings(settings, cfg.length);
688 }
689 }
690
691 void ensure_context() const
692 {
693 if (m_context) {
694 return;
695 }
696 if (!m_recording) {
697 m_recording = std::make_shared<GpuDispatchRecorder>(true);
698 }
699 m_context = std::make_unique<ShaderExecutionContext<>>(
700 GpuComputeConfig { .shader_path = interval_scan_shader, .workgroup_size = { 1, 1, 1 }, .push_constant_size = sizeof(IntervalScanPC) });
701 m_context->set_recording(m_recording);
702 if (!m_context->ensure_gpu_ready()) {
704 std::source_location::current(), "IntervalExtractor needs a graphics device and none is available");
705 }
706 }
707
708 /**
709 * Makes a column's buffers, uploads its scores and state, and counts the call.
710 * Every column is prepared before any scan is recorded, so a gate column's
711 * scores exist when the scans that read them are recorded.
712 */
713 ChannelCall prepare_channel(size_t channel, std::span<const double> scores, bool resident, uint32_t resident_count, bool flush)
714 {
715 const auto n = resident ? resident_count : static_cast<uint32_t>(scores.size());
716 if (n == 0 && !flush) {
717 return {};
718 }
719
721 constexpr auto word = GpuBufferBinding::ElementType::UINT32;
722 const size_t scores_binding = binding_of(channel, Scores);
723 const size_t runs_binding = binding_of(channel, Runs);
724 const size_t report_binding = binding_of(channel, Report);
725 const size_t state_binding = binding_of(channel, State);
726
727 m_context->ensure_shared_buffer(0, scores_binding, std::max<size_t>(n, 1), GpuBufferBinding::ElementType::FLOAT32, BufferUsageHint::COMPUTE);
728 m_context->ensure_shared_buffer(0, runs_binding, interval_scan_runs_words(n), word);
729 m_context->ensure_shared_buffer(0, report_binding, interval_scan_report_words, word);
730 m_context->ensure_shared_buffer(0, state_binding, interval_scan_state_words, word, BufferUsageHint::COMPUTE);
731
732 if (!m_started[channel]) {
733 const std::vector<uint32_t> zeros(interval_scan_state_words, 0U);
734 m_context->upload_shared_raw(0, state_binding, reinterpret_cast<const uint8_t*>(zeros.data()), zeros.size() * sizeof(uint32_t));
735 m_started[channel] = true;
736 }
737 if (!resident && n > 0) {
738 const std::vector<float> floats(scores.begin(), scores.end());
739 m_context->upload_shared_raw(0, scores_binding, reinterpret_cast<const uint8_t*>(floats.data()), floats.size() * sizeof(float));
740 }
741 const uint32_t none = 0;
742 m_context->upload_shared_raw(0, report_binding, reinterpret_cast<const uint8_t*>(&none), sizeof(none));
743
744 const uint32_t first = m_seen[channel];
745 if (flush) {
746 m_seen[channel] = 0;
747 m_started[channel] = false;
748 } else {
749 m_seen[channel] = first + n;
750 }
751 return { .ran = true, .capacity = n + 4, .first = first, .count = n };
752 }
753
754 /**
755 * Records one scan per column that ran, each with its own settings. A gated
756 * column reads its gate's scores from the gate column's own buffer.
757 */
758 void record_scans(const std::vector<ChannelCall>& calls, const IntervalExtractorConfig& cfg)
759 {
760 std::vector<DependencyStage> stages;
761 for (size_t channel = 0; channel < calls.size(); ++channel) {
762 if (!calls[channel].ran) {
763 continue;
764 }
765 const auto& settings = cfg.settings_of(channel);
766 const uint64_t gate = gated_by_other(settings, channel)
767 ? m_context->shared_buffer_address(0, binding_of(settings.gate_channel, Scores))
768 : uint64_t { 0 };
769 stages.push_back(interval_scan_stage(interval_scan_config(settings, cfg.length),
770 m_context->shared_buffer_address(0, binding_of(channel, Scores)),
771 gate,
772 m_context->shared_buffer_address(0, binding_of(channel, State)),
773 m_context->shared_buffer_address(0, binding_of(channel, Runs)),
774 m_context->shared_buffer_address(0, binding_of(channel, Report)),
775 calls[channel].first, calls[channel].count, cfg.flush));
776 }
777 if (!stages.empty()) {
778 m_context->record_stages(stages);
779 }
780 }
781
782 ChannelIntervals read_channel(size_t channel, const ChannelCall& call) const
783 {
784 ChannelIntervals result;
785 if (!call.ran) {
786 return result;
787 }
788
789 std::array<uint32_t, interval_scan_report_words> words {};
790 m_context->download_shared(0, binding_of(channel, Report), words.data(), sizeof(words));
791 const auto report = decode_interval_scan_report(words);
792
793 if (report.emitted > 0) {
794 std::vector<uint32_t> runs(static_cast<size_t>(report.emitted) * 2);
795 m_context->download_shared(0, binding_of(channel, Runs), runs.data(), runs.size() * sizeof(uint32_t));
796 result.intervals.reserve(report.emitted);
797 for (uint32_t i = 0; i < report.emitted; ++i) {
798 result.intervals.emplace_back(runs[2 * i], runs[2 * i + 1]);
799 }
800 }
801 result.open_start = report.open_start;
802 result.pending = report.pending;
803 return result;
804 }
805};
806
807/// Matrix interval extractor: the MatrixXd an analyzer returns, one score channel per column -> interval pairs per channel
809
810/// Standard interval extractor: DataVariant scores -> interval pairs per channel
812
813} // namespace MayaFlux::Yantra
#define MF_ERROR(comp, ctx,...)
#define MF_DESCRIBE(...)
Requests a generated member description for a struct at namespace scope.
Definition Describe.hpp:14
Modern, digital-first extractor framework for Maya Flux.
The stage that runs the interval scan on device memory, its push layout and its buffer contract.
Runs of a score sequence that stay above a threshold, found in order as a stream, for MayaFlux::Kines...
void set(T value)
Stores a struct under its type identity.
Definition Facets.hpp:107
bool is_initialized() const
Check if compiler is initialized.
ConfigRefs stored_configs(Supports< Configs... >) const
output_type convert_result(std::vector< std::vector< double > > &result_data, DataStructureInfo &metadata)
Template-flexible extractor base with instance-defined I/O types.
Definition Extractor.hpp:98
Non-template base that owns all type-independent GPU dispatch logic.
Extractor< InputType, OutputType > base_type
static void check_gate(uint32_t gate, size_t columns, size_t length)
IntervalExtraction extract_intervals(const input_type &data)
Type-safe interval extraction.
static void check_settings(const IntervalScanSettings &settings, uint32_t length)
bool ready() const
True when nothing is outstanding or the outstanding submission has completed.
IntervalExtraction extract_intervals(const InputType &data)
output_type extract_implementation(const input_type &input) override
std::vector< ChannelCall > record_resident(std::span< const uint32_t > counts, const IntervalExtractorConfig &cfg)
bool pending() const
True while a submission is outstanding and not yet collected.
std::string get_extractor_name() const override
ChannelIntervals read_channel(size_t channel, const ChannelCall &call) const
std::vector< ChannelCall > record_all(const Columns &columns, const IntervalExtractorConfig &cfg)
void reset()
Start a new stream: forget the detector state of every channel.
Supports< IntervalExtractorConfig > supports
ChannelCall prepare_channel(size_t channel, std::span< const double > scores, bool resident, uint32_t resident_count, bool flush)
Makes a column's buffers, uploads its scores and state, and counts the call.
std::vector< IntervalBuffers > buffers() const
Device views of the results of the latest call, one per channel.
bool submit_resident(std::span< const uint32_t > counts)
Record and submit an extraction over scores already on the device, without waiting.
ConfigRefs configs() const override
IntervalExtraction extract_resident(std::span< const uint32_t > counts)
Extraction over scores already on the device, waiting for the result.
bool submit(const input_type &data)
Record and submit an extraction without waiting for it.
static size_t binding_of(size_t channel, Slot slot)
static void validate_config(const IntervalExtractorConfig &cfg)
void record_scans(const std::vector< ChannelCall > &calls, const IntervalExtractorConfig &cfg)
Records one scan per column that ran, each with its own settings.
void bind_scores(size_t channel, const GpuDispatchCore &source, uint32_t set, size_t binding)
Read the scores of a channel from a device buffer instead of uploading them.
ExtractionType get_extraction_type() const override
static float fitness(const input_type &)
One while a device is up, zero otherwise.
IntervalExtractor(IntervalExtractorConfig config={})
Construct an interval extractor from its settings.
void share_recording(std::shared_ptr< GpuDispatchRecorder > recording)
Append to a recording the caller shares with other work.
static bool gated_by_other(const IntervalScanSettings &settings, size_t channel)
std::optional< IntervalExtraction > collect()
The result of the outstanding submission, once it has completed.
Finds where each score sequence is engaged, on the GPU, as a stream.
static std::tuple< NumericChannels, DataStructureInfo > extract_structured_double(T &compute_data)
Extract structured double data from Datum container or direct ComputeData with automatic container ha...
@ ComputeMatrix
Compute operations (Yantra - algorithms, matrices, DSP).
@ Yantra
DSP algorithms, computational units, matrix operations, Grammar.
std::string diagnose_interval_scan(const IntervalScanConfig &config)
Why a config cannot be scanned.
constexpr FenceID INVALID_FENCE
MAYAFLUX_API ShaderFoundry & get_shader_foundry()
Get the global shader compiler instance.
BufferUsageHint
Semantic usage hint for buffer allocation and memory properties.
constexpr uint32_t interval_scan_state_words
Words of the state buffer.
ExtractionType
Categories of extraction operations for discovery and organization.
Definition Extractor.hpp:47
@ PATTERN_BASED
Extract based on pattern recognition.
Definition Extractor.hpp:51
IntervalExtractor< Eigen::MatrixXd, std::vector< std::vector< double > > > MatrixIntervalExtractor
Matrix interval extractor: the MatrixXd an analyzer returns, one score channel per column -> interval...
DependencyStage interval_scan_stage(const Kinesis::Discrete::IntervalScanConfig &config, uint64_t scores, uint64_t gate, uint64_t state, uint64_t runs, uint64_t report, uint32_t first_index, uint32_t n, bool flush)
One interval_scan dispatch: continue a stream with n scores from device memory.
IntervalScanReport decode_interval_scan_report(std::span< const uint32_t > words)
Decode the report buffer.
Kinesis::Discrete::IntervalScanConfig interval_scan_config(const IntervalScanSettings &settings, uint32_t length)
The host scan configuration of one column's settings over a stream of the given length.
constexpr uint32_t interval_scan_report_words
Words of the report buffer: the run count, then the open run and the pending interval.
constexpr size_t interval_scan_runs_words(size_t n) noexcept
Words the run list needs for n scores: a start and an end for each run that can close,...
IntervalExtractor< std::vector< Kakshya::DataVariant >, std::vector< std::vector< double > > > StandardIntervalExtractor
Standard interval extractor: DataVariant scores -> interval pairs per channel.
IntervalMethod
How the score sequence is turned into intervals.
@ HYSTERESIS_RUNS
Engage at or above the enter threshold, release below the leave threshold after a minimum gap.
std::vector< std::reference_wrapper< const OperationConfig > > ConfigRefs
Borrowed views of the configs an operation holds.
Kakshya::RegionGroup interval_regions(const ChannelIntervals &intervals, uint32_t first_channel, uint32_t last_channel, std::string name="intervals")
One region per interval, spanning the given channels.
constexpr const char * interval_scan_shader
std::pair< std::shared_ptr< Kriya::BufferPipeline >, std::shared_ptr< Buffers::AudioBuffer > > record(const std::shared_ptr< Kakshya::DynamicSoundStream > &stream, uint32_t input_channel, double seconds, std::optional< uint32_t > monitor, uint32_t channel)
Record a live input channel into a stream for a number of seconds.
Definition Rigs.cpp:262
void add_region(const Region &region)
Organizes related signal regions into a categorized collection.
static Region audio_span(uint64_t start_frame, uint64_t end_frame, uint32_t start_channel, uint32_t end_channel, const std::string &label="")
Create a Region representing a span in audio (frames and channels).
Definition Region.hpp:196
enum MayaFlux::Portal::Graphics::GpuBufferBinding::ElementType FLOAT32
Plain-data description of the compute shader to dispatch.
Stores a scoped enum in its underlying integer type.
std::optional< std::pair< size_t, size_t > > pending
std::vector< std::pair< size_t, size_t > > intervals
Intervals of one channel, half open sample ranges in stream order.
Dimensions, modality, source type, and optional Composite field schema.
Facets facets
Typed results that travel with the data, read by type.
Definition DataIO.hpp:68
std::unordered_map< std::string, std::any > metadata
Associated metadata.
Definition DataIO.hpp:67
Input/Output container for computation pipeline data flow with structure preservation.
Definition DataIO.hpp:63
Settings every extractor family accepts.
Definition Extractor.hpp:75
uint32_t capacity
Pairs the run list can hold.
std::shared_ptr< Portal::Graphics::GpuBufferHandle > report
std::shared_ptr< Portal::Graphics::GpuBufferHandle > runs
Device views of one channel's results from the latest call, for a consumer that reads them on the GPU...
std::vector< ChannelIntervals > channels
Intervals completed by this call, per score channel.
IntervalExtractorConfig config
Settings the extraction was produced under.
Record of one IntervalExtractor run, stored on the output's facets.
bool flush
End the stream with this call.
IntervalScanSettings scan
Settings of every column not listed in columns.
std::vector< IntervalScanSettings > columns
Per column overrides, column i for index i.
uint32_t length
Length of the stream in samples when it ends, 0 to use the scores seen.
const IntervalScanSettings & settings_of(size_t column) const noexcept
Settings in force for a column.
Reflect::EnumByte< IntervalMethod > method
Selection method, as an IntervalMethod value.
Settings of an IntervalExtractor.
Push constants of interval_scan.comp.
uint32_t first_sample
Sample position of score 0.
bool emit_on_close
Emit each interval as soon as it closes, with full padding on both sides.
double leave_margin
How far below the enter threshold a score must fall to count as a gap.
uint32_t padding
Samples added on each side of an interval.
static constexpr uint32_t no_channel
Value of gate_channel for no gate.
bool use_longest_gap
Cut a capped run at its longest gap instead of the last one.
uint32_t min_gap
Shortest gap that ends a run, in samples.
uint32_t samples_per_score
Samples between successive scores.
uint32_t gate_channel
Column that gates this one, or no_channel.
double gate_threshold
Gate score below which this column is silent.
uint32_t max_run
Longest interval in samples, 0 for no cap.
double enter_threshold
Score at or above which a run starts.
uint32_t gap_at_limit
Gaps longer than this are candidates for cutting a capped run, in samples.
double leave_floor
Lowest leave threshold.
uint32_t min_run
Shortest run kept, in samples.
How one score column is turned into intervals.
The config structs an operation class accepts, declared as using supports.
A set of (device, runtime) pairs.