Michal/TuneRipper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor: Use jobId_title.ext for downloads & improve UX

Browse Source 
Adopt GUID-based {jobId}_{title}.ext filename convention for all downloads, ensuring unique, human-readable, and traceable files. Update DownloadController to use precise jobId-based file matching and generate user-friendly download names. Pass jobId to VideoService for consistent naming; add robust title sanitization. Remove fallback file search logic for predictability. Update README and API docs to reflect new format and workflow. Improve logging and error messages for clarity.
This commit is contained in:
Michal Vrba
2025-12-12 10:28:48 +01:00
parent 406cf31b49
commit 3ecff7ca73
4 changed files with 247 additions and 102 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TuneRipper.Api/README_UPDATED.md
+108 -50
View File
@@ -2,7 +2,7 @@
## Overview
The TuneRipper API has been updated to use `yt-dlp.exe` for real YouTube video processing with progress monitoring. Videos are downloaded and converted to MP3 files, stored in the `wwwroot/downloads` directory, and served directly through the API.
The TuneRipper API has been updated to use `yt-dlp.exe` for real YouTube video processing with progress monitoring. Videos are downloaded and converted to MP3 files, stored in the `wwwroot/downloads` directory with a GUID-based filename format, and served directly through the API.
## Setup
@@ -22,6 +22,33 @@ TuneRipper.Api/
??? Program.cs
```
## Filename Format
Downloaded files use the structured format: **`{jobId}_{title}.mp3`**
### Format Breakdown:
- **`{jobId}`**: GUID identifier for the processing job (e.g., `550e8400-e29b-41d4-a716-446655440000`)
- **`_{title}`**: Sanitized video title with spaces replaced by underscores
- **`.mp3`**: Audio format extension
### Examples:
- `550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3`
- `6ba7b810-9dad-11d1-80b4-00c04fd430c8_How_to_Build_a_Web_API.mp3`
- `6ba7b811-9dad-11d1-80b4-00c04fd430c8_Amazing_Guitar_Solo_Performance.mp3`
### Benefits:
- **Unique Identification**: GUID ensures no filename conflicts
- **Direct Job Mapping**: Easy file lookup using job ID
- **Human Readable**: Title part provides context
- **System Friendly**: Sanitized for filesystem compatibility
### Title Sanitization:
- Invalid filename characters are replaced with safe alternatives
- Spaces become underscores (`_`)
- Special characters: `< > | : * ? " /` are replaced with safe chars
- Maximum title length: 50 characters
- Fallback to "video" if title cannot be determined
## Key Changes
### 1. Program.cs Updates
@@ -30,21 +57,26 @@ TuneRipper.Api/
- Updated logging and endpoint information
### 2. VideoService Updates
- Integrated real `yt-dlp.exe` processing with progress monitoring
- Real-time progress parsing from yt-dlp output
- File storage in `wwwroot/downloads/` directory
- Progress callbacks for UI updates
- **GUID-Based Filenames**: Uses job ID as primary identifier in filename
- **Title Sanitization**: Converts video titles to safe filenames
- **Direct Job Mapping**: Files can be found using `{jobId}_*.*` pattern
- **Real yt-dlp Integration**: Uses actual yt-dlp.exe with progress monitoring
- **Progress Parsing**: Real-time progress from yt-dlp output streams
- **Error Handling**: Comprehensive error handling with 5-minute timeout
### 3. VideoController Updates
- Passes job ID to VideoService for filename generation
- Uses actual yt-dlp processing instead of simulation
- Stores downloaded file information in job tracking
- Stores downloaded file information with GUID-based naming
- Returns real download URLs for completed files
### 4. DownloadController Updates
- **Precise Job Lookup**: Uses `{jobId}_*.*` pattern for exact file matching
- **User-Friendly Downloads**: Converts filenames back to readable format
- **Example**: `550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3` ? `Rick Astley Never Gonna Give You Up.mp3`
- **Reliable File Operations**: No ambiguity in file identification
- Serves actual MP3 files from `wwwroot/downloads/`
- Proper content type detection
- File cleanup capabilities
- Download metadata endpoints
- Proper MIME type detection for audio files
## API Workflow
@@ -65,7 +97,7 @@ Content-Type: application/json
{
"success": true,
"message": "Processing started",
"fileName": "job-id-guid",
"fileName": "550e8400-e29b-41d4-a716-446655440000",
"downloadUrl": null,
"videoDetails": null
}
@@ -73,21 +105,7 @@ Content-Type: application/json
### 2. Monitor Progress
```http
GET /api/video/status/{jobId}
```
**Response (Early Stage):**
```json
{
"percentageComplete": 15,
"status": "Processing",
"message": "Video details retrieved successfully",
"videoDetails": {
"id": "dQw4w9WgXcQ",
"title": "Rick Astley - Never Gonna Give You Up",
"thumbnailUrl": "https://img.youtube.com/vi/dQw4w9WgXcQ/maxresdefault.jpg"
}
}
GET /api/video/status/550e8400-e29b-41d4-a716-446655440000
```
**Response (Processing):**
@@ -96,13 +114,17 @@ GET /api/video/status/{jobId}
"percentageComplete": 45,
"status": "Processing",
"message": "Downloading audio...",
"videoDetails": { ... }
"videoDetails": {
"id": "dQw4w9WgXcQ",
"title": "Rick Astley - Never Gonna Give You Up",
"thumbnailUrl": "https://img.youtube.com/vi/dQw4w9WgXcQ/maxresdefault.jpg"
}
}
```
### 3. Get Final Result
```http
GET /api/video/result/{jobId}
GET /api/video/result/550e8400-e29b-41d4-a716-446655440000
```
**Response:**
@@ -110,23 +132,48 @@ GET /api/video/result/{jobId}
{
"success": true,
"message": "Video processed successfully",
"fileName": "dQw4w9WgXcQ_20241212_093022.mp3",
"downloadUrl": "/downloads/dQw4w9WgXcQ_20241212_093022.mp3",
"fileName": "550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3",
"downloadUrl": "/downloads/550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3",
"videoDetails": { ... }
}
```
### 4. Download File
```http
GET /api/download/{jobId}
GET /api/download/550e8400-e29b-41d4-a716-446655440000
```
**Response:** Binary MP3 file download
**Response:** Binary MP3 file download with user-friendly filename
**Download Name:** `Rick Astley Never Gonna Give You Up.mp3`
**Alternative Static File Access:**
```http
GET /downloads/{fileName}
GET /downloads/550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3
```
## File Management
### Download Directory
- **Location**: `wwwroot/downloads/`
- **Naming Pattern**: `{jobId}_{title}.ext`
- **Job Lookup**: Files found using `Directory.GetFiles(dir, "{jobId}_*.*")`
- **User Download**: Converted back to readable names
- **Cleanup**: Files can be deleted by job ID
### File Serving Options
1. **API Download**: Controlled access via `/api/download/{jobId}` (recommended)
- Uses job ID for secure file lookup
- Returns user-friendly filename
- Proper error handling
2. **Static Files**: Direct access via `/downloads/{actual_filename}`
- Requires knowing the full filename
- Less secure but faster
3. **Content-Type**: Automatic detection based on file extension
### File Operations by Job ID
- **Find**: `GET /api/download/info/{jobId}` - Get file metadata
- **Download**: `GET /api/download/{jobId}` - Download the file
- **Delete**: `DELETE /api/download/{jobId}` - Remove the file
## Progress Monitoring
The VideoService monitors yt-dlp output and translates it to progress updates:
@@ -144,7 +191,7 @@ The VideoService monitors yt-dlp output and translates it to progress updates:
2. **ffmpeg not available**: Install ffmpeg and add to PATH
3. **Invalid YouTube URL**: URL format validation
4. **Download timeout**: 5-minute timeout for processing
5. **File not found**: Cleanup or missing download directory
5. **File not found**: File may have been cleaned up or job ID invalid
### Error Response Format:
```json
@@ -157,18 +204,6 @@ The VideoService monitors yt-dlp output and translates it to progress updates:
}
```
## File Management
### Download Directory
- Location: `wwwroot/downloads/`
- Naming: `{videoId}_{timestamp}.{ext}`
- Auto-cleanup: Implement retention policies as needed
### File Serving
- Static files: Direct access via `/downloads/{fileName}`
- API download: Controlled access via `/api/download/{jobId}`
- Content-Type: Automatic detection based on file extension
## Configuration
### appsettings.json
@@ -189,6 +224,7 @@ The VideoService monitors yt-dlp output and translates it to progress updates:
- `--no-playlist`: Single video only
- `--no-warnings`: Clean output
- `--newline`: Line-by-line progress
- `--output "{jobId}_{title}.%(ext)s"`: Uses job ID-based filename
## Testing
@@ -196,7 +232,15 @@ The VideoService monitors yt-dlp output and translates it to progress updates:
1. Start the API: `dotnet run --project TuneRipper.Api`
2. Use API testing tools or the Blazor client
3. Check logs for yt-dlp progress
4. Verify files in `wwwroot/downloads/`
4. Verify files in `wwwroot/downloads/` with GUID-based naming
### Example Test Files:
```
wwwroot/downloads/
??? 550e8400-e29b-41d4-a716-446655440000_Rick_Astley_Never_Gonna_Give_You_Up.mp3
??? 6ba7b810-9dad-11d1-80b4-00c04fd430c8_How_to_Build_a_Web_API.mp3
??? 6ba7b811-9dad-11d1-80b4-00c04fd430c8_Amazing_Guitar_Solo_Performance.mp3
```
### Health Check:
```http
@@ -211,11 +255,12 @@ GET /api/video/jobs
## Production Considerations
1. **File Storage**: Consider cloud storage for production
2. **Cleanup**: Implement automatic file cleanup/retention
3. **Security**: Validate URLs and sanitize filenames
2. **Cleanup Strategy**: Implement automatic file cleanup/retention using job completion timestamps
3. **Security**: Job IDs provide secure file access (no directory traversal)
4. **Performance**: Consider queue-based processing for high load
5. **Monitoring**: Add detailed logging and metrics
6. **Updates**: Keep yt-dlp.exe updated for compatibility
7. **Scalability**: GUID-based naming prevents conflicts in distributed systems
## Troubleshooting
@@ -224,6 +269,7 @@ GET /api/video/jobs
2. **Antivirus**: yt-dlp.exe may be flagged by antivirus
3. **Network**: YouTube may block requests from some IPs
4. **Format Support**: Some videos may not support audio extraction
5. **Long Titles**: Titles are truncated to 50 characters for filesystem compatibility
### Debug Logging:
Enable debug logging for VideoService to see detailed yt-dlp output:
@@ -234,5 +280,17 @@ Enable debug logging for VideoService to see detailed yt-dlp output:
"TuneRipper.Api.Services.VideoService": "Debug"
}
}
}
```
```
### File Lookup Issues:
- Verify job ID matches the GUID format
- Check that files exist with the `{jobId}_` prefix
- Look at directory contents if files are not found
- Ensure proper case sensitivity on Linux systems
### Advantages of GUID-Based Naming:
1. **Unique**: No filename conflicts possible
2. **Secure**: Job ID acts as access token
3. **Traceable**: Direct mapping from API calls to files
4. **Scalable**: Works in distributed environments
5. **Clean**: Easy cleanup by job ID pattern