Real YouTube download & MP3 conversion with yt-dlp
- Integrate yt-dlp.exe for real YouTube audio download/conversion - Serve actual MP3 files from wwwroot/downloads via API and static files - Report real-time progress from yt-dlp to frontend - Enhance DownloadController: file serving, metadata, deletion endpoints - Update VideoController to track and return real file info/URLs - Enable static file serving in Program.cs; update API info/logging - Update Blazor frontend to use real download URLs and remove old JS logic - Add README_UPDATED.md with new workflow and troubleshooting - Improve error handling, logging, and file cleanup throughout backend - Update .gitignore for downloads and yt-dlp.exe
This commit is contained in:
Michal Vrba
@@ -0,0 +1,238 @@
|
||||
# TuneRipper API - yt-dlp Integration
|
||||
|
||||
## Overview
|
||||
|
||||
The TuneRipper API has been updated to use `yt-dlp.exe` for real YouTube video processing with progress monitoring. Videos are downloaded and converted to MP3 files, stored in the `wwwroot/downloads` directory, and served directly through the API.
|
||||
|
||||
## Setup
|
||||
|
||||
### Prerequisites
|
||||
1. **.NET 10** installed
|
||||
2. **yt-dlp.exe** placed in `TuneRipper.Api/wwwroot/yt-dlp.exe`
|
||||
3. **ffmpeg** available in PATH (required by yt-dlp for audio conversion)
|
||||
|
||||
### File Structure
|
||||
```
|
||||
TuneRipper.Api/
|
||||
??? wwwroot/
|
||||
? ??? yt-dlp.exe # YouTube downloader executable
|
||||
? ??? downloads/ # Downloaded MP3 files (auto-created)
|
||||
??? Controllers/
|
||||
??? Services/
|
||||
??? Program.cs
|
||||
```
|
||||
|
||||
## Key Changes
|
||||
|
||||
### 1. Program.cs Updates
|
||||
- Added `UseStaticFiles()` middleware for serving downloaded files
|
||||
- Added `AddHttpClient()` for VideoService HTTP requests
|
||||
- Updated logging and endpoint information
|
||||
|
||||
### 2. VideoService Updates
|
||||
- Integrated real `yt-dlp.exe` processing with progress monitoring
|
||||
- Real-time progress parsing from yt-dlp output
|
||||
- File storage in `wwwroot/downloads/` directory
|
||||
- Progress callbacks for UI updates
|
||||
|
||||
### 3. VideoController Updates
|
||||
- Uses actual yt-dlp processing instead of simulation
|
||||
- Stores downloaded file information in job tracking
|
||||
- Returns real download URLs for completed files
|
||||
|
||||
### 4. DownloadController Updates
|
||||
- Serves actual MP3 files from `wwwroot/downloads/`
|
||||
- Proper content type detection
|
||||
- File cleanup capabilities
|
||||
- Download metadata endpoints
|
||||
|
||||
## API Workflow
|
||||
|
||||
### 1. Start Processing
|
||||
```http
|
||||
POST /api/video/process
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
|
||||
"format": "mp3",
|
||||
"quality": "128"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "Processing started",
|
||||
"fileName": "job-id-guid",
|
||||
"downloadUrl": null,
|
||||
"videoDetails": null
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Monitor Progress
|
||||
```http
|
||||
GET /api/video/status/{jobId}
|
||||
```
|
||||
|
||||
**Response (Early Stage):**
|
||||
```json
|
||||
{
|
||||
"percentageComplete": 15,
|
||||
"status": "Processing",
|
||||
"message": "Video details retrieved successfully",
|
||||
"videoDetails": {
|
||||
"id": "dQw4w9WgXcQ",
|
||||
"title": "Rick Astley - Never Gonna Give You Up",
|
||||
"thumbnailUrl": "https://img.youtube.com/vi/dQw4w9WgXcQ/maxresdefault.jpg"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response (Processing):**
|
||||
```json
|
||||
{
|
||||
"percentageComplete": 45,
|
||||
"status": "Processing",
|
||||
"message": "Downloading audio...",
|
||||
"videoDetails": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Get Final Result
|
||||
```http
|
||||
GET /api/video/result/{jobId}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "Video processed successfully",
|
||||
"fileName": "dQw4w9WgXcQ_20241212_093022.mp3",
|
||||
"downloadUrl": "/downloads/dQw4w9WgXcQ_20241212_093022.mp3",
|
||||
"videoDetails": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Download File
|
||||
```http
|
||||
GET /api/download/{jobId}
|
||||
```
|
||||
**Response:** Binary MP3 file download
|
||||
|
||||
**Alternative Static File Access:**
|
||||
```http
|
||||
GET /downloads/{fileName}
|
||||
```
|
||||
|
||||
## Progress Monitoring
|
||||
|
||||
The VideoService monitors yt-dlp output and translates it to progress updates:
|
||||
|
||||
- **5-15%**: Fetching video details
|
||||
- **15-70%**: Downloading audio (yt-dlp download progress × 0.7)
|
||||
- **70-85%**: Converting to MP3 (ffmpeg processing)
|
||||
- **85-95%**: Finalizing (cleanup)
|
||||
- **100%**: Complete
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Common Errors:
|
||||
1. **yt-dlp.exe not found**: Ensure yt-dlp.exe is in wwwroot directory
|
||||
2. **ffmpeg not available**: Install ffmpeg and add to PATH
|
||||
3. **Invalid YouTube URL**: URL format validation
|
||||
4. **Download timeout**: 5-minute timeout for processing
|
||||
5. **File not found**: Cleanup or missing download directory
|
||||
|
||||
### Error Response Format:
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"message": "Error description",
|
||||
"fileName": null,
|
||||
"downloadUrl": null,
|
||||
"videoDetails": null
|
||||
}
|
||||
```
|
||||
|
||||
## File Management
|
||||
|
||||
### Download Directory
|
||||
- Location: `wwwroot/downloads/`
|
||||
- Naming: `{videoId}_{timestamp}.{ext}`
|
||||
- Auto-cleanup: Implement retention policies as needed
|
||||
|
||||
### File Serving
|
||||
- Static files: Direct access via `/downloads/{fileName}`
|
||||
- API download: Controlled access via `/api/download/{jobId}`
|
||||
- Content-Type: Automatic detection based on file extension
|
||||
|
||||
## Configuration
|
||||
|
||||
### appsettings.json
|
||||
```json
|
||||
{
|
||||
"Logging": {
|
||||
"LogLevel": {
|
||||
"TuneRipper.Api.Services.VideoService": "Debug"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### yt-dlp Arguments Used:
|
||||
- `--extract-audio`: Extract audio only
|
||||
- `--audio-format mp3`: Convert to MP3
|
||||
- `--audio-quality 0`: Best quality
|
||||
- `--no-playlist`: Single video only
|
||||
- `--no-warnings`: Clean output
|
||||
- `--newline`: Line-by-line progress
|
||||
|
||||
## Testing
|
||||
|
||||
### Manual Testing:
|
||||
1. Start the API: `dotnet run --project TuneRipper.Api`
|
||||
2. Use API testing tools or the Blazor client
|
||||
3. Check logs for yt-dlp progress
|
||||
4. Verify files in `wwwroot/downloads/`
|
||||
|
||||
### Health Check:
|
||||
```http
|
||||
GET /api/health
|
||||
```
|
||||
|
||||
### Job Listing:
|
||||
```http
|
||||
GET /api/video/jobs
|
||||
```
|
||||
|
||||
## Production Considerations
|
||||
|
||||
1. **File Storage**: Consider cloud storage for production
|
||||
2. **Cleanup**: Implement automatic file cleanup/retention
|
||||
3. **Security**: Validate URLs and sanitize filenames
|
||||
4. **Performance**: Consider queue-based processing for high load
|
||||
5. **Monitoring**: Add detailed logging and metrics
|
||||
6. **Updates**: Keep yt-dlp.exe updated for compatibility
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues:
|
||||
1. **Permissions**: Ensure API can write to wwwroot/downloads
|
||||
2. **Antivirus**: yt-dlp.exe may be flagged by antivirus
|
||||
3. **Network**: YouTube may block requests from some IPs
|
||||
4. **Format Support**: Some videos may not support audio extraction
|
||||
|
||||
### Debug Logging:
|
||||
Enable debug logging for VideoService to see detailed yt-dlp output:
|
||||
```json
|
||||
{
|
||||
"Logging": {
|
||||
"LogLevel": {
|
||||
"TuneRipper.Api.Services.VideoService": "Debug"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user