TuneRipper/TuneRipper.Api/README_UPDATED.md

TuneRipper API - yt-dlp Integration

Overview

The TuneRipper API has been updated to use yt-dlp.exe for real YouTube video processing with progress monitoring. Videos are downloaded and converted to MP3 files, stored in the wwwroot/downloads directory with a GUID-based filename format, and served directly through the API.

Setup

Prerequisites

1. .NET 10 installed
2. yt-dlp.exe placed in TuneRipper.Api/wwwroot/yt-dlp.exe
3. ffmpeg available in PATH (required by yt-dlp for audio conversion)

File Structure

TuneRipper.Api/
??? wwwroot/
?   ??? yt-dlp.exe          # YouTube downloader executable
?   ??? downloads/          # Downloaded MP3 files (auto-created)
??? Controllers/
??? Services/
??? Program.cs

Filename Format

Downloaded files use the structured format: {jobId}_{title}.mp3

Format Breakdown:

• {jobId}: GUID identifier for the processing job (e.g., 550e8400-e29b-41d4-a716-446655440000)
• _{title}: Sanitized video title with spaces replaced by underscores
• .mp3: Audio format extension

Examples:

• 550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3
• 6ba7b810-9dad-11d1-80b4-00c04fd430c8_How_to_Build_a_Web_API.mp3
• 6ba7b811-9dad-11d1-80b4-00c04fd430c8_Amazing_Guitar_Solo_Performance.mp3

Benefits:

• Unique Identification: GUID ensures no filename conflicts
• Direct Job Mapping: Easy file lookup using job ID
• Human Readable: Title part provides context
• System Friendly: Sanitized for filesystem compatibility

Title Sanitization:

• Invalid filename characters are replaced with safe alternatives
• Spaces become underscores (_)
• Special characters: < > | : * ? " / are replaced with safe chars
• Maximum title length: 50 characters
• Fallback to "video" if title cannot be determined

Key Changes

1. Program.cs Updates

• Added UseStaticFiles() middleware for serving downloaded files
• Added AddHttpClient() for VideoService HTTP requests
• Updated logging and endpoint information

2. VideoService Updates

• GUID-Based Filenames: Uses job ID as primary identifier in filename
• Title Sanitization: Converts video titles to safe filenames
• Direct Job Mapping: Files can be found using {jobId}_*.* pattern
• Real yt-dlp Integration: Uses actual yt-dlp.exe with progress monitoring
• Progress Parsing: Real-time progress from yt-dlp output streams
• Error Handling: Comprehensive error handling with 5-minute timeout

3. VideoController Updates

• Passes job ID to VideoService for filename generation
• Uses actual yt-dlp processing instead of simulation
• Stores downloaded file information with GUID-based naming
• Returns real download URLs for completed files

4. DownloadController Updates

• Precise Job Lookup: Uses {jobId}_*.* pattern for exact file matching
• User-Friendly Downloads: Converts filenames back to readable format
• Example: 550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3 ? Rick Astley Never Gonna Give You Up.mp3
• Reliable File Operations: No ambiguity in file identification
• Serves actual MP3 files from wwwroot/downloads/
• Proper MIME type detection for audio files

API Workflow

1. Start Processing

POST /api/video/process
Content-Type: application/json

{
  "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
  "format": "mp3",
  "quality": "128"
}

Response:

{
  "success": true,
  "message": "Processing started",
  "fileName": "550e8400-e29b-41d4-a716-446655440000",
  "downloadUrl": null,
  "videoDetails": null
}

2. Monitor Progress

GET /api/video/status/550e8400-e29b-41d4-a716-446655440000

Response (Processing):

{
  "percentageComplete": 45,
  "status": "Processing", 
  "message": "Downloading audio...",
  "videoDetails": {
    "id": "dQw4w9WgXcQ",
    "title": "Rick Astley - Never Gonna Give You Up",
    "thumbnailUrl": "https://img.youtube.com/vi/dQw4w9WgXcQ/maxresdefault.jpg"
  }
}

3. Get Final Result

GET /api/video/result/550e8400-e29b-41d4-a716-446655440000

Response:

{
  "success": true,
  "message": "Video processed successfully",
  "fileName": "550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3",
  "downloadUrl": "/downloads/550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3",
  "videoDetails": { ... }
}

4. Download File

GET /api/download/550e8400-e29b-41d4-a716-446655440000

Response: Binary MP3 file download with user-friendly filename Download Name: Rick Astley Never Gonna Give You Up.mp3

Alternative Static File Access:

GET /downloads/550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3

File Management

Download Directory

• Location: wwwroot/downloads/
• Naming Pattern: {jobId}_{title}.ext
• Job Lookup: Files found using Directory.GetFiles(dir, "{jobId}_*.*")
• User Download: Converted back to readable names
• Cleanup: Files can be deleted by job ID

File Serving Options

1. API Download: Controlled access via /api/download/{jobId} (recommended)
   • Uses job ID for secure file lookup
   • Returns user-friendly filename
   • Proper error handling
2. Static Files: Direct access via /downloads/{actual_filename}
   • Requires knowing the full filename
   • Less secure but faster
3. Content-Type: Automatic detection based on file extension

File Operations by Job ID

• Find: GET /api/download/info/{jobId} - Get file metadata
• Download: GET /api/download/{jobId} - Download the file
• Delete: DELETE /api/download/{jobId} - Remove the file

Progress Monitoring

The VideoService monitors yt-dlp output and translates it to progress updates:

• 5-15%: Fetching video details
• 15-70%: Downloading audio (yt-dlp download progress × 0.7)
• 70-85%: Converting to MP3 (ffmpeg processing)
• 85-95%: Finalizing (cleanup)
• 100%: Complete

Error Handling

Common Errors:

1. yt-dlp.exe not found: Ensure yt-dlp.exe is in wwwroot directory
2. ffmpeg not available: Install ffmpeg and add to PATH
3. Invalid YouTube URL: URL format validation
4. Download timeout: 5-minute timeout for processing
5. File not found: File may have been cleaned up or job ID invalid

Error Response Format:

{
  "success": false,
  "message": "Error description",
  "fileName": null,
  "downloadUrl": null,
  "videoDetails": null
}

Configuration

appsettings.json

{
  "Logging": {
    "LogLevel": {
      "TuneRipper.Api.Services.VideoService": "Debug"
    }
  }
}

yt-dlp Arguments Used:

• --extract-audio: Extract audio only
• --audio-format mp3: Convert to MP3
• --audio-quality 0: Best quality
• --no-playlist: Single video only
• --no-warnings: Clean output
• --newline: Line-by-line progress
• --output "{jobId}_{title}.%(ext)s": Uses job ID-based filename

Testing

Manual Testing:

1. Start the API: dotnet run --project TuneRipper.Api
2. Use API testing tools or the Blazor client
3. Check logs for yt-dlp progress
4. Verify files in wwwroot/downloads/ with GUID-based naming

Example Test Files:

wwwroot/downloads/
??? 550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3
??? 6ba7b810-9dad-11d1-80b4-00c04fd430c8_How_to_Build_a_Web_API.mp3
??? 6ba7b811-9dad-11d1-80b4-00c04fd430c8_Amazing_Guitar_Solo_Performance.mp3

Health Check:

GET /api/health

Job Listing:

GET /api/video/jobs

Production Considerations

1. File Storage: Consider cloud storage for production
2. Cleanup Strategy: Implement automatic file cleanup/retention using job completion timestamps
3. Security: Job IDs provide secure file access (no directory traversal)
4. Performance: Consider queue-based processing for high load
5. Monitoring: Add detailed logging and metrics
6. Updates: Keep yt-dlp.exe updated for compatibility
7. Scalability: GUID-based naming prevents conflicts in distributed systems

Troubleshooting

Common Issues:

1. Permissions: Ensure API can write to wwwroot/downloads
2. Antivirus: yt-dlp.exe may be flagged by antivirus
3. Network: YouTube may block requests from some IPs
4. Format Support: Some videos may not support audio extraction
5. Long Titles: Titles are truncated to 50 characters for filesystem compatibility

Debug Logging:

Enable debug logging for VideoService to see detailed yt-dlp output:

{
  "Logging": {
    "LogLevel": {
      "TuneRipper.Api.Services.VideoService": "Debug"
    }
  }

File Lookup Issues:

• Verify job ID matches the GUID format
• Check that files exist with the {jobId}_ prefix
• Look at directory contents if files are not found
• Ensure proper case sensitivity on Linux systems

Advantages of GUID-Based Naming:

1. Unique: No filename conflicts possible
2. Secure: Job ID acts as access token
3. Traceable: Direct mapping from API calls to files
4. Scalable: Works in distributed environments
5. Clean: Easy cleanup by job ID pattern