224 lines
5.7 KiB
Markdown
224 lines
5.7 KiB
Markdown
# Fractal Audio Visualizer
|
|
|
|
## Technical Specification
|
|
|
|
**Platform:** Windows 10/11\
|
|
**Runtime:** .NET 10\
|
|
**Framework:** MonoGame (WindowsDX)\
|
|
**Audio Library:** NAudio (WASAPI Capture + WASAPI Loopback)
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# 1. Project Overview
|
|
|
|
Fractal Audio Visualizer is a real-time Windows desktop application that
|
|
captures audio and generates GPU-accelerated fractal visualizations that
|
|
react dynamically to sound.
|
|
|
|
The application uses exactly 2 audio input presets:
|
|
|
|
1. **Microphone (Default):** Listen to microphone input in real time.
|
|
2. **System Output:** Listen to sound played by any application on the
|
|
PC (Spotify, YouTube, games, browser audio, etc.).
|
|
|
|
Captured audio drives procedural fractal graphics via frequency spectrum
|
|
analysis.
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# 2. Core Functional Requirements
|
|
|
|
## 2.1 Audio Capture
|
|
|
|
Input preset modes (exactly 2):
|
|
|
|
- **Microphone (Default):** Capture default microphone in real time
|
|
using NAudio WASAPI capture (shared mode, with WaveIn fallback if
|
|
needed)
|
|
- **System Output:** Use NAudio WASAPI Loopback Capture for audio
|
|
played by any application on the PC
|
|
|
|
Common requirements:
|
|
|
|
- Support 44.1kHz and 48kHz
|
|
- Automatic device rebind if default input/output device changes
|
|
- Buffer size: 1024--4096 samples
|
|
- PCM float format internally
|
|
|
|
### Output
|
|
|
|
Audio sample stream → Ring buffer → AudioAnalyzer
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
## 2.2 Audio Analysis
|
|
|
|
Performed continuously in background thread.
|
|
|
|
### Processing Steps:
|
|
|
|
1. Apply Hann window
|
|
2. FFT (2048 or 4096)
|
|
3. Compute:
|
|
- RMS (Level)
|
|
- Bass band energy (20--200Hz)
|
|
- Mid band energy (200--2000Hz)
|
|
- High band energy (2000--16000Hz)
|
|
4. Beat detection (energy spike detection)
|
|
5. Exponential smoothing (attack/release)
|
|
|
|
### Output Model
|
|
|
|
``` csharp
|
|
public struct AudioFeatures
|
|
{
|
|
public float Level;
|
|
public float Bass;
|
|
public float Mid;
|
|
public float High;
|
|
public float Beat;
|
|
public float[] Spectrum; // 64 bins
|
|
public float TimeSeconds;
|
|
}
|
|
```
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# 3. Rendering Architecture
|
|
|
|
## 3.1 Framework
|
|
|
|
MonoGame WindowsDX template\
|
|
Rendering via HLSL shaders (.fx)
|
|
|
|
## 3.2 Rendering Pipeline
|
|
|
|
Game Loop:
|
|
|
|
Update(): - Pull latest AudioFeatures snapshot - Update shader
|
|
parameters
|
|
|
|
Draw(): - Render fullscreen quad - Apply fractal shader - Optional
|
|
post-processing
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
## 3.3 Shader Uniforms
|
|
|
|
float Time;
|
|
float Level;
|
|
float Bass;
|
|
float Mid;
|
|
float High;
|
|
float Beat;
|
|
float2 Resolution;
|
|
float Spectrum[64];
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# 4. Fractal Presets
|
|
|
|
Minimum 3 presets:
|
|
|
|
## 4.1 Julia Set (2D)
|
|
|
|
- Parameter C modified by Mid
|
|
- Zoom pulse from Bass
|
|
- Glow from High
|
|
|
|
## 4.2 Mandelbulb (3D Raymarch)
|
|
|
|
- Power modulated by Bass
|
|
- Rotation by Mid
|
|
- Detail iterations influenced by High
|
|
- Beat triggers camera pulse
|
|
|
|
## 4.3 Kaleidoscope Tunnel
|
|
|
|
- Spectrum drives radial distortion
|
|
- Beat triggers color shift
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# 5. Performance Strategy
|
|
|
|
- Render to scaled RenderTarget (0.5x--1x resolution)
|
|
- Adjustable iteration count
|
|
- Target: 60 FPS at 1080p (Medium preset)
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# 6. UI Controls
|
|
|
|
Keyboard: - F11 → Fullscreen toggle - ESC → Exit fullscreen - SPACE →
|
|
Pause animation - LEFT/RIGHT → Switch fractal preset - TAB → Toggle HUD
|
|
- M → Toggle audio input mode (Microphone/System Output)
|
|
|
|
Optional: Myra UI for settings overlay
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# 7. Threading Model
|
|
|
|
Audio Thread: - Active input callback (Microphone or Loopback) - Push
|
|
samples to ring buffer
|
|
|
|
Analysis Thread: - FFT + feature extraction - Publish snapshot
|
|
(lock-free)
|
|
|
|
Main Thread: - MonoGame Update/Draw
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# 8. Project Structure
|
|
|
|
FractalVisualizer.sln
|
|
├── FractalVisualizer
|
|
│ ├── Game1.cs
|
|
│ ├── Audio/
|
|
│ │ AudioCapture.cs
|
|
│ │ AudioAnalyzer.cs
|
|
│ ├── Rendering/
|
|
│ │ FractalRenderer.cs
|
|
│ │ Shaders/
|
|
│ ├── Models/
|
|
│ │ AudioFeatures.cs
|
|
│ ├── UI/
|
|
│ └── SettingsManager.cs
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# 9. MVP Scope
|
|
|
|
MVP Includes: - Exactly 2 input presets: Microphone (default) and
|
|
System Output (switchable) - FFT + 3-band energy extraction - Julia
|
|
shader - Preset switching - Fullscreen mode
|
|
|
|
Estimated implementation time: 5--7 development days.
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# 10. Future Enhancements
|
|
|
|
- Advanced beat & tempo detection
|
|
- Preset editor (live tweak UI)
|
|
- Video export (visual-only recording)
|
|
- MIDI input support
|
|
- Multi-monitor support
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# 11. Acceptance Criteria
|
|
|
|
1. App starts in Microphone mode and reacts to live mic input.
|
|
2. User can switch to System Output mode and app reacts to
|
|
Spotify/YouTube playback without microphone.
|
|
3. Bass drops visibly affect zoom/intensity.
|
|
4. Preset switch occurs without frame drop.
|
|
5. 60 FPS at 1080p (medium quality).
|
|
6. No crashes on device change.
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
# End of Specification
|