The previous document Domain and Control demonstrated true digital-first multimedia coordination where computational domains interact through shared timing references, data transformation pipelines, and temporal coordination mechanisms. Rather than separating audio, visual, and temporal processing into isolated systems, MayaFlux treats them as unified aspects of a single computational-creative expression.
The Engine's default automation provides optimal performance for common workflows, while explicit control mechanisms enable precise customization when specific computational patterns are required. This default automation with expressive override philosophy ensures that creative flow remains uninterrupted while advanced techniques remain accessible when needed.
This document explores replacing Engine defaults with custom managers, handlers, subsystems, backends, and the default Engine itself—enabling complete architectural customization while maintaining the digital-first computational paradigm.
Note Many currently non existent subsystems will me mentioned to showcase possible actions, workflows and overrides. All referenced methods are available in the ISubsystem interface, but the specific concrete class may not be part of the default library.
Note This document offers less textural or verbose explanations as most low level c++ code itself contains explanations within its structure and via comments. Textual context is added only where it is necessary to describe intent of operation beyond whats made obvious by the code. As the title suggests, this is an advanced section.
- Processing Handles: Token-Scoped System Access
- BufferProcessingHandle
- NodeProcessingHandle
- TaskSchedulerHandle
- SubsystemProcessingHandle
- Handle composition and custom contexts
- Why Processing Handles Instead of Direct Manager Access?
- Handle Performance Characteristics
- SubsystemManager: Computational Domain Orchestration
- Subsystem Registration and Lifecycle
- Cross-Subsystem Data Access Control
- Processing Hooks and Custom Integration
- Direct SubsystemManager Control
- Subsystems: Computational Domain Implementation
- ISubsystem Interface and Implementation Patterns
- Specialized Subsystem Examples
- AudioSubsystem
- GraphicsSubsystem
- Subsystem Communication and Coordination
- Direct Subsystem Management
- Backends: Hardware and Platform Abstraction
- Audio Backend Architecture
- IAudioBackend Interface
- Backend Factory Expansion
- Graphics Backend Architecture
- IGraphicsBackend Interface
- Custom Rendering Pipeline
- Network Backend for Distributed Processing%
Processing Handles: Token-Scoped System Access
Processing handles provide token-scoped access to the core processing systems (Nodes, Buffers, Coroutines), ensuring proper domain isolation and thread safety. Rather than direct access to managers, handles act as computational interfaces that enforce processing boundaries while enabling cross-domain coordination.
Each handle represents a specific computational context defined by processing tokens, providing controlled access to the underlying systems while maintaining the engine's coordination patterns.
BufferProcessingHandle
Provides token-scoped access to buffer operations within a specific processing domain:
auto buffer_handle = audio_subsystem->get_processing_context_handle()->buffers;
auto audio_buffer = buffer_handle.create_buffer<AudioBuffer>(0, 1024);
buffer_handle.process(512);
auto buffer_data = buffer_handle.get_buffer_data(0);
std::shared_ptr< SubsystemManager > get_subsystem_manager()
Gets the subsystem manager for advanced component access.
The handle automatically routes operations through the appropriate token-specific processing paths, ensuring that buffer operations remain within their designated computational domain while enabling optimal performance.
Direct handle creation
Handles can be created independently for custom processing scenarios:
auto custom_handle = BufferProcessingHandle(
custom_buffer_manager,
Buffers::ProcessingToken::CUSTOM | Buffers::ProcessingToken::PARALLEL
);
custom_handle.add_buffer(gpu_accelerated_buffer);
custom_handle.process(1024);
auto gpu_buffer_data = custom_handle.read_channel_data(0);
NodeProcessingHandle
Provides token-scoped access to node graph operations within specific processing domains:
auto node_handle = graphics_subsystem->get_processing_context_handle()->nodes;
auto visual_output = node_handle.process_channel(0, 1024);
auto sample_output = node_handle.process_sample(0);
node_handle.register_node(visual_processor_node, 0);
Node handles ensure that processing operations respect the temporal characteristics of their assigned domain while providing unified access patterns across different processing contexts.
TaskSchedulerHandle
Provides token-scoped access to coroutine scheduling within specific temporal domains:
auto sync_subsystem = get_engine()->get_subsystem_manager()->get_sync_subsystem();
auto task_handle = sync_subsystem->get_processing_context_handle()->tasks;
task_handle.process(1024);
auto routine = create_sync_routine();
task_handle.add_task(routine, "cross_domain_sync");
SubsystemProcessingHandle
Combines all three processing handles into a unified computational context for subsystem operations:
class CustomMultimediaSubsystem : public ISubsystem {
private:
SubsystemProcessingHandle* m_handle;
public:
void initialize(SubsystemProcessingHandle& handle)
override {
m_handle = &handle;
auto audio_buffer = m_handle->buffers.create_buffer<AudioBuffer>(0, 512);
auto visual_node = create_visual_processor();
m_handle->nodes.register_node(visual_node, 0);
auto sync_routine = create_coordination_routine();
m_handle->tasks.add_task(sync_routine, "multimedia_sync");
}
void process_multimedia_frame() {
m_handle->buffers.process_all_buffers(512);
auto visual_data = m_handle->nodes.process_channel(0, 512);
m_handle->tasks.process(512);
coordinate_audio_visual_data(visual_data);
}
};
Handle composition and custom contexts
Processing handles can be composed for specialized computational requirements:
auto multimedia_tokens = SubsystemTokens{
.Buffer = Buffers::ProcessingToken::AUDIO_BACKEND | Buffers::ProcessingToken::GRAPHICS_BACKEND,
.Node = Nodes::ProcessingToken::AUDIO_RATE | Nodes::ProcessingToken::VISUAL_RATE,
.Task = Vruta::ProcessingToken::MULTI_RATE
};
auto multimedia_handle = SubsystemProcessingHandle(
buffer_manager, node_manager, task_scheduler, multimedia_tokens
);
multimedia_handle.buffers.transfer_data_between_domains(audio_buffer, graphics_buffer);
multimedia_handle.nodes.coordinate_processing_rates(audio_nodes, visual_nodes);
multimedia_handle.tasks.sync_temporal_domains();
Why Processing Handles Instead of Direct Manager Access?
Processing handles enforce computational boundaries that prevent:
- Cross-domain memory corruption
- Token privilege escalation
- Uncontrolled resource access
auto buffer_data = buffer_handle.get_buffer_data(0);
Direct manager access would require manual boundary checking in every subsystem.
Handle Performance Characteristics
- Token validation: O(1) bitfield operations
- Cross-domain access: Single atomic read with permission check
- Memory overhead: ~64 bytes per handle (pointer + token state)
auto data = handle.get_buffer_data(0);
auto cross_data = handle.read_cross_subsystem_buffer(...);
SubsystemManager: Computational Domain Orchestration
The SubsystemManager serves as the central coordinator for all computational domains in MayaFlux, managing subsystem lifecycle, processing handle distribution, and cross-domain data access. Rather than a simple container, it acts as a computational orchestrator that ensures proper isolation while enabling coordinated operation across different processing contexts.
Subsystem Registration and Lifecycle
auto subsystem_manager = std::make_shared<SubsystemManager>(
node_graph_manager, buffer_manager, task_scheduler
);
subsystem_manager->create_audio_subsystem(stream_info, AudioBackendType::PORTAUDIO);
subsystem_manager->create_graphics_subsystem(display_config, GraphicsBackendType::VULKAN);
subsystem_manager->create_sync_subsystem(sync_config);
subsystem_manager->start_all_subsystems();
The manager automatically creates appropriate processing handles for each subsystem based on their token requirements, ensuring that each computational domain receives properly scoped access to the core processing systems.
Cross-Subsystem Data Access Control
subsystem_manager->allow_cross_access(SubsystemType::AUDIO, SubsystemType::GRAPHICS);
subsystem_manager->allow_cross_access(SubsystemType::GRAPHICS, SubsystemType::AUDIO);
auto spectral_data = subsystem_manager->read_cross_subsystem_buffer(
SubsystemType::GRAPHICS,
SubsystemType::AUDIO,
0
);
if (spectral_data) {
apply_spectral_visualization(*spectral_data);
}
Cross-access permissions ensure that data flow between computational domains remains controlled and explicit, preventing unintended interference while enabling intentional coordination.
Processing Hooks and Custom Integration
subsystem_manager->register_process_hook(
SubsystemType::AUDIO,
"spectral_analysis",
[](SubsystemProcessingHandle& handle) {
auto audio_data = handle.buffers.get_buffer_data(0);
perform_spectral_analysis(audio_data);
},
HookPosition::PRE_PROCESS
);
subsystem_manager->register_process_hook(
SubsystemType::GRAPHICS,
"visual_coordination",
[](SubsystemProcessingHandle& handle) {
coordinate_visual_elements();
update_display_parameters();
},
HookPosition::POST_PROCESS
);
Processing hooks enable custom integration points without modifying subsystem internals, supporting the expressive override philosophy by allowing precise customization when specific computational patterns are required.
Direct SubsystemManager Control
auto custom_manager = std::make_shared<SubsystemManager>(
custom_node_manager, custom_buffer_manager, custom_scheduler
);
auto custom_subsystem = std::make_shared<CustomProcessingSubsystem>();
custom_manager->add_subsystem(SubsystemType::CUSTOM, custom_subsystem);
auto subsystem = custom_manager->get_subsystem(SubsystemType::CUSTOM);
auto processing_handle = subsystem->get_processing_context_handle();
processing_handle->buffers.process_all_buffers(1024);
processing_handle->nodes.process_channel(0, 1024);
processing_handle->tasks.process(1024);
Subsystems: Computational Domain Implementation
Subsystems represent complete computational domains that encapsulate specific processing characteristics, resource management, and coordination patterns. Each subsystem implements the ISubsystem interface, defining how it integrates with the broader MayaFlux processing architecture.
ISubsystem Interface and Implementation Patterns
class CustomAudioVisualSubsystem : public ISubsystem {
private:
SubsystemProcessingHandle* m_handle;
std::atomic<bool> m_running{false};
std::atomic<bool> m_ready{false};
std::unique_ptr<SpectralAnalyzer> m_spectral_analyzer;
std::unique_ptr<VisualRenderer> m_visual_renderer;
public:
void initialize(SubsystemProcessingHandle& handle)
override {
m_handle = &handle;
m_spectral_analyzer = std::make_unique<SpectralAnalyzer>();
m_visual_renderer = std::make_unique<VisualRenderer>();
setup_audio_visual_coordination();
m_ready = true;
}
void start() override {
if (!m_ready) return;
m_running = true;
m_spectral_analyzer->start();
m_visual_renderer->start();
register_audio_callback();
register_visual_callback();
}
SubsystemTokens get_tokens() const override {
return {
.Buffer = Buffers::ProcessingToken::AUDIO_BACKEND |
Buffers::ProcessingToken::GRAPHICS_BACKEND,
.Node = Nodes::ProcessingToken::AUDIO_RATE |
Nodes::ProcessingToken::VISUAL_RATE,
.Task = Vruta::ProcessingToken::MULTI_RATE
};
}
private:
void setup_audio_visual_coordination() {
auto audio_buffer = m_handle->buffers.create_buffer<AudioBuffer>(0, 1024);
auto texture_buffer = m_handle->buffers.create_buffer<TextureBuffer>(0, 1024);
auto fft_node = create_spectral_analyzer_node();
auto visual_node = create_visual_processor_node();
m_handle->nodes.register_node(fft_node, 0);
m_handle->nodes.register_node(visual_node, 1);
auto sync_routine = create_audio_visual_sync_routine();
m_handle->tasks.add_task(sync_routine, "audio_visual_sync");
}
};
Specialized Subsystem Examples
AudioSubsystem
class AudioSubsystem : public ISubsystem {
private:
Utils::AudioBackendType m_backend_type;
std::unique_ptr<IAudioBackend> m_backend;
std::unique_ptr<AudioDevice> m_audio_device;
std::unique_ptr<AudioStream> m_audio_stream;
GlobalStreamInfo m_stream_info;
public:
AudioSubsystem(GlobalStreamInfo& stream_info, Utils::AudioBackendType backend_type)
: m_stream_info(stream_info), m_backend_type(backend_type) {
m_backend = AudioBackendFactory::create_backend(backend_type);
m_audio_device = m_backend->create_device_manager();
}
void initialize(SubsystemProcessingHandle& handle)
override {
m_handle = &handle;
setup_audio_buffers(handle);
setup_audio_nodes(handle);
setup_sample_accurate_timing(handle);
m_audio_stream = m_backend->create_stream(
m_audio_device->get_default_output_device(),
m_audio_device->get_default_input_device(),
m_stream_info,
this
);
m_ready = true;
}
SubsystemTokens get_tokens() const override {
return {
.Buffer = Buffers::ProcessingToken::AUDIO_BACKEND,
.Node = Nodes::ProcessingToken::AUDIO_RATE,
.Task = Vruta::ProcessingToken::SAMPLE_ACCURATE
};
}
private:
void setup_audio_buffers(SubsystemProcessingHandle& handle) {
for (uint32_t channel = 0; channel < m_stream_info.output.channels; ++channel) {
auto input_buffer = handle.buffers.create_buffer<AudioInputBuffer>(channel, 512);
auto output_buffer = handle.buffers.create_buffer<AudioOutputBuffer>(channel, 512);
}
}
};
GraphicsSubsystem
class GraphicsSubsystem : public ISubsystem {
private:
Utils::GraphicsBackendType m_backend_type;
std::unique_ptr<IGraphicsBackend> m_backend;
std::unique_ptr<VulkanRenderer> m_renderer;
std::unique_ptr<DisplayManager> m_display_manager;
public:
GraphicsSubsystem(DisplayConfig& display_config, Utils::GraphicsBackendType backend_type)
: m_display_config(display_config), m_backend_type(backend_type) {
m_backend = GraphicsBackendFactory::create_backend(backend_type);
m_display_manager = m_backend->create_display_manager();
}
void initialize(SubsystemProcessingHandle& handle)
override {
m_handle = &handle;
setup_texture_buffers(handle);
setup_visual_nodes(handle);
setup_frame_timing(handle);
m_renderer = m_backend->create_renderer(m_display_config);
m_ready = true;
}
SubsystemTokens get_tokens() const override {
return {
.Buffer = Buffers::ProcessingToken::GRAPHICS_BACKEND,
.Node = Nodes::ProcessingToken::VISUAL_RATE,
.Task = Vruta::ProcessingToken::FRAME_ACCURATE
};
}
private:
void setup_texture_buffers(SubsystemProcessingHandle& handle) {
auto color_buffer = handle.buffers.create_buffer<ColorBuffer>(0, 1920*1080);
auto depth_buffer = handle.buffers.create_buffer<DepthBuffer>(0, 1920*1080);
auto compute_buffer = handle.buffers.create_buffer<ComputeBuffer>(0, 1024*1024);
}
};
Subsystem Communication and Coordination
class SynchronizationSubsystem : public ISubsystem {
private:
SubsystemManager* m_manager;
public:
void initialize(SubsystemProcessingHandle& handle)
override {
m_handle = &handle;
setup_audio_visual_sync(handle);
setup_temporal_coordination(handle);
m_ready = true;
}
SubsystemTokens get_tokens() const override {
return {
.Buffer = Buffers::ProcessingToken::CUSTOM,
.Node = Nodes::ProcessingToken::CUSTOM_RATE,
.Task = Vruta::ProcessingToken::MULTI_RATE
};
}
private:
void setup_audio_visual_sync(SubsystemProcessingHandle& handle) {
auto sync_routine = [this, &handle](Vruta::TaskScheduler& scheduler) -> Vruta::SoundRoutine {
while (true) {
co_await Kriya::ProcessingGate{scheduler,
[this]() { return m_manager->get_audio_subsystem()->frame_complete(); }};
auto visual_subsystem = m_manager->get_graphics_subsystem();
visual_subsystem->trigger_frame_render();
co_await Kriya::SampleDelay{scheduler.seconds_to_samples(1.0/60.0)};
}
};
handle.tasks.add_task(sync_routine(scheduler), "cross_domain_sync");
}
};
Direct Subsystem Management
Subsystems can operate independently of the SubsystemManager when specific computational patterns require direct control:
auto custom_subsystem = std::make_shared<CustomProcessingSubsystem>();
auto custom_tokens = SubsystemTokens{
.Buffer = Buffers::ProcessingToken::CUSTOM,
.Node = Nodes::ProcessingToken::CUSTOM_RATE,
.Task = Vruta::ProcessingToken::CUSTOM
};
auto custom_handle = SubsystemProcessingHandle(
buffer_manager, node_manager, task_scheduler, custom_tokens
);
custom_subsystem->initialize(custom_handle);
custom_subsystem->start();
auto processing_handle = custom_subsystem->get_processing_context_handle();
processing_handle->buffers.process_all_buffers(512);
processing_handle->nodes.process_channel(0, 512);
processing_handle->tasks.process(512);
});
void schedule_metro(double interval_seconds, std::function< void()> callback, std::string name)
Creates a metronome task and addes it to the default scheduler list for evaluation.
Backends: Hardware and Platform Abstraction
Backends provide the abstraction layer between MayaFlux's computational architecture and platform-specific hardware or software interfaces. Each backend type handles a specific category of external interface: audio hardware, graphics systems, file I/O, networking, etc.
Audio Backend Architecture
The audio backend system abstracts different audio APIs (RtAudio, JACK, ASIO, CoreAudio, etc.) behind a unified interface:
IAudioBackend Interface
class CustomAudioBackend : public IAudioBackend {
private:
std::unique_ptr<CustomAudioContext> m_context;
std::unique_ptr<CustomDeviceManager> m_device_manager;
public:
CustomAudioBackend() {
m_context = std::make_unique<CustomAudioContext>();
m_device_manager = std::make_unique<CustomDeviceManager>(m_context.get());
}
std::unique_ptr<AudioDevice> create_device_manager() override {
return std::make_unique<CustomAudioDevice>(m_context.get());
}
std::unique_ptr<AudioStream> create_stream(
unsigned int output_device_id,
unsigned int input_device_id,
const GlobalStreamInfo& stream_info,
void* user_data) override {
return std::make_unique<CustomAudioStream>(
m_context.get(), output_device_id, input_device_id, stream_info, user_data
);
}
std::string get_version_string() const override {
return "CustomAudioBackend v1.0.0";
}
int get_api_type() const override {
return static_cast<int>(Utils::AudioBackendType::CUSTOM);
}
void cleanup() override {
if (m_context) {
m_context->shutdown();
}
}
};
Backend Factory Expansion
class AudioBackendFactory {
public:
static std::unique_ptr<IAudioBackend> create_backend(Utils::AudioBackendType type) {
switch (type) {
case Utils::AudioBackendType::RTAUDIO:
return std::make_unique<RtAudioBackend>();
case Utils::AudioBackendType::JACK:
return std::make_unique<JackAudioBackend>();
case Utils::AudioBackendType::ASIO:
return std::make_unique<AsioAudioBackend>();
case Utils::AudioBackendType::CUSTOM:
return std::make_unique<CustomAudioBackend>();
default:
throw std::runtime_error("Unsupported audio backend type");
}
}
static void register_custom_backend(
Utils::AudioBackendType type,
std::function<std::unique_ptr<IAudioBackend>()> factory) {
custom_factories[type] = factory;
}
private:
static std::map<Utils::AudioBackendType,
std::function<std::unique_ptr<IAudioBackend>()>> custom_factories;
};
Graphics Backend Architecture
Similar to audio backends, graphics backends abstract different rendering APIs:
IGraphicsBackend Interface
class VulkanGraphicsBackend : public IGraphicsBackend {
private:
std::unique_ptr<VulkanContext> m_vulkan_context;
std::unique_ptr<VulkanRenderer> m_renderer;
public:
VulkanGraphicsBackend() {
m_vulkan_context = std::make_unique<VulkanContext>();
}
std::unique_ptr<DisplayManager> create_display_manager() override {
return std::make_unique<VulkanDisplayManager>(m_vulkan_context.get());
}
std::unique_ptr<IRenderer> create_renderer(const DisplayConfig& config) override {
return std::make_unique<VulkanRenderer>(m_vulkan_context.get(), config);
}
std::unique_ptr<ComputeManager> create_compute_manager() override {
return std::make_unique<VulkanComputeManager>(m_vulkan_context.get());
}
std::string get_version_string() const override {
return "Vulkan Backend v1.3.0";
}
void cleanup() override {
if (m_vulkan_context) {
m_vulkan_context->shutdown();
}
}
};
Custom Rendering Pipeline
class SpectralVisualizationRenderer : public IRenderer {
private:
std::unique_ptr<VulkanComputeShader> m_fft_shader;
std::unique_ptr<VulkanRenderPass> m_spectral_pass;
public:
void initialize(
const DisplayConfig& config)
override {
m_fft_shader = create_fft_compute_shader();
m_spectral_pass = create_spectral_render_pass();
}
void render_frame(const RenderData& data) override {
m_fft_shader->dispatch(data.audio_buffer);
m_spectral_pass->render(data.spectral_data);
}
void coordinate_with_audio(const AudioFrameData& audio_data) {
update_spectral_parameters(audio_data.frequency_data);
trigger_visual_events(audio_data.onset_detection);
}
};
Network Backend for Distributed Processing
class NetworkAudioBackend : public IAudioBackend {
private:
std::unique_ptr<NetworkManager> m_network;
std::unique_ptr<AudioCodec> m_codec;
public:
NetworkAudioBackend(const NetworkConfig& config) {
m_network = std::make_unique<NetworkManager>(config);
m_codec = std::make_unique<AudioCodec>(CodecType::OPUS);
}
std::unique_ptr<AudioStream> create_stream(
unsigned int output_device_id,
unsigned int input_device_id,
const GlobalStreamInfo& stream_info,
void* user_data) override {
return std::make_unique<NetworkAudioStream>(
m_network.get(), m_codec.get(), stream_info, user_data
);
}
void connect_to_peer(const std::string& address) {
m_network->connect(address);
}
void start_collaboration_session() {
m_network->enable_multicast();
}
};
This comprehensive architecture enables complete computational paradigm customization while maintaining the digital-first creative philosophy. Every layer—from processing handles through backends to the engine itself—can be replaced with custom implementations that serve specific creative or technical requirements.
The three-tier architecture (Processing Handles → SubsystemManager → Subsystems) combined with replaceable backends and engine contexts provides ultimate flexibility for specialized computational workflows. Whether implementing quantum-inspired processing, neural adaptation, biomimetic algorithms, or entirely novel paradigms, the architecture supports seamless integration while preserving the unified data transformation approach that defines MayaFlux's digital-first creative environment.
The expressive override philosophy extends through every architectural level, ensuring that creative expression never encounters artificial limitations while computational precision remains accessible at every scale of interaction.