YouTube Scraper

A YouTube scraper powered by yt-dlp - available in both Python and Node.js. No API key required.

How It Works

The scraper wraps yt-dlp CLI to extract YouTube metadata without downloading videos.

┌─────────────┐      ┌─────────────┐      ┌─────────────┐
│  Your Code  │ ───► │  yt-dlp CLI │ ───► │   YouTube   │
│  (scraper)  │ ◄─── │  (JSON out) │ ◄─── │   Servers   │
└─────────────┘      └─────────────┘      └─────────────┘

Flow:

1. You call a method (e.g., get_video(url))
2. Scraper runs: yt-dlp --dump-json --no-download "URL"
3. yt-dlp fetches data from YouTube and returns raw JSON
4. Scraper transforms it into a clean, unified format

Project Structure

youtube-scraper/
├── python/
│   ├── scraper.py
│   └── requirements.txt
├── nodejs/
│   ├── scraper.js
│   └── package.json
└── README.md

Features

Feature	Python	Node.js	Description
Video metadata	✅	✅	Get title, views, likes, duration, thumbnail, etc.
Search	✅	✅	Search YouTube and get video results
Channel videos	✅	✅	Get all videos from a channel
Playlists	✅	✅	Get all videos in a playlist
Download	✅	✅	Download video or audio (MP3)
Transcripts	✅	✅	Get video subtitles/captions
Comments	✅	✅	Get video comments
JSON Export	✅	✅	Export results to JSON file
No API key	✅	✅	Works without YouTube API credentials

Setup

# Install yt-dlp (required)
pip install yt-dlp

# For Node.js, also install dependencies
cd nodejs && npm install

Usage

Command Line

Python

cd python

# Get video metadata
python scraper.py video "https://youtube.com/watch?v=VIDEO_ID"

# Search YouTube
python scraper.py search "python tutorial" -n 10

# Get channel videos
python scraper.py channel "@channelname" -n 20

# Get playlist
python scraper.py playlist "https://youtube.com/playlist?list=PLAYLIST_ID"

# Get transcript/subtitles
python scraper.py transcript "VIDEO_URL" -l en

# Get comments
python scraper.py comments "VIDEO_URL" -n 100

# Download video
python scraper.py download "VIDEO_URL"

# Download audio only (MP3)
python scraper.py download "VIDEO_URL" --audio

# Export to JSON file
python scraper.py video "VIDEO_URL" -o output.json

Node.js

cd nodejs

# Get video metadata
node scraper.js video "https://youtube.com/watch?v=VIDEO_ID"

# Search YouTube
node scraper.js search "python tutorial" --max 10

# Get channel videos
node scraper.js channel "@channelname" --max 20

# Get playlist
node scraper.js playlist "https://youtube.com/playlist?list=PLAYLIST_ID"

# Get transcript/subtitles
node scraper.js transcript "VIDEO_URL" --lang en

# Get comments
node scraper.js comments "VIDEO_URL" --max 100

# Download video
node scraper.js download "VIDEO_URL"

# Download audio only (MP3)
node scraper.js download "VIDEO_URL" --audio

# Export to JSON file
node scraper.js video "VIDEO_URL" --output output.json

As a Module

Python

from scraper import YTDLPScraper

scraper = YTDLPScraper()

# Get video metadata
video = scraper.get_video("https://youtube.com/watch?v=VIDEO_ID")
print(video['title'], video['views'])

# Search YouTube
results = scraper.search("python tutorial", max_results=10)
for video in results:
    print(video['title'])

# Get channel videos
videos = scraper.get_channel("@channelname", max_videos=20)

# Get playlist
playlist = scraper.get_playlist("PLAYLIST_URL")
print(f"Playlist: {playlist['title']} ({playlist['video_count']} videos)")

# Get transcript
transcript = scraper.get_transcript("VIDEO_URL", lang="en")
for segment in transcript:
    print(f"[{segment['start']:.1f}s] {segment['text']}")

# Get comments
comments = scraper.get_comments("VIDEO_URL", max_comments=100)
for comment in comments:
    print(f"@{comment['author']}: {comment['text']}")

# Download
scraper.download("VIDEO_URL", audio_only=True)

Node.js

const { YTDLPScraper } = require('./scraper');

const scraper = new YTDLPScraper();

// Get video metadata
const video = await scraper.getVideo('https://youtube.com/watch?v=VIDEO_ID');
console.log(video.title, video.views);

// Search YouTube
const results = await scraper.search('python tutorial', 10);
results.forEach(v => console.log(v.title));

// Get channel videos
const videos = await scraper.getChannel('@channelname', 20);

// Get playlist
const playlist = await scraper.getPlaylist('PLAYLIST_URL');
console.log(`Playlist: ${playlist.title} (${playlist.video_count} videos)`);

// Get transcript
const transcript = await scraper.getTranscript('VIDEO_URL', 'en');
transcript.forEach(s => console.log(`[${s.start}s] ${s.text}`));

// Get comments
const comments = await scraper.getComments('VIDEO_URL', 100);
comments.forEach(c => console.log(`@${c.author}: ${c.text}`));

// Download
await scraper.download('VIDEO_URL', { audioOnly: true });

// Export to JSON
scraper.exportJson(results, 'results.json');

Data Formats

All methods return a unified data format - consistent across Python and Node.js.

Video Metadata

{
  "video_id": "dQw4w9WgXcQ",
  "title": "Rick Astley - Never Gonna Give You Up",
  "description": "The official video for...",
  "channel_name": "Rick Astley",
  "channel_id": "UCuAXFkgsw1L7xaCfnd5JJOw",
  "channel_url": "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw",
  "views": 1500000000,
  "likes": 15000000,
  "comments_count": 3000000,
  "duration_seconds": 212,
  "duration_string": "3:32",
  "upload_date": "20091025",
  "thumbnail_url": "https://i.ytimg.com/vi/dQw4w9WgXcQ/maxresdefault.jpg",
  "tags": ["rick astley", "never gonna give you up", "80s"],
  "is_live": false,
  "is_short": false,
  "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
}

Search Results / Channel Videos

[
  {
    "video_id": "abc123",
    "title": "Video Title",
    "channel_name": "Channel Name",
    "channel_id": "UCxxxx",
    "views": 10000,
    "duration_seconds": 180,
    "duration_string": "3:00",
    "thumbnail_url": "https://i.ytimg.com/vi/abc123/hqdefault.jpg",
    "published_time": "20230101",
    "url": "https://www.youtube.com/watch?v=abc123"
  }
]

Playlist

{
  "playlist_id": "PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf",
  "title": "My Playlist",
  "description": "A collection of videos",
  "channel_name": "Channel Name",
  "video_count": 25,
  "videos": [
    {
      "index": 1,
      "video_id": "abc123",
      "title": "First Video",
      "channel_name": "Channel",
      "duration_seconds": 240,
      "duration_string": "4:00",
      "thumbnail_url": "https://...",
      "url": "https://www.youtube.com/watch?v=abc123"
    }
  ],
  "url": "https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf"
}

Transcript

[
  {
    "start": 0.0,
    "duration": 2.5,
    "text": "Hello and welcome"
  },
  {
    "start": 2.5,
    "duration": 3.0,
    "text": "to this video"
  }
]

Comments

[
  {
    "author": "Username",
    "author_id": "UCxxxx",
    "author_url": "https://www.youtube.com/channel/UCxxxx",
    "text": "Great video!",
    "likes": 150,
    "published_time": "2 months ago",
    "reply_count": 5,
    "is_reply": false
  }
]

API Reference

Python Methods

Method	Parameters	Returns
get_video(url)	Video URL	Video metadata dict
search(query, max_results=20)	Search query, limit	List of video dicts
get_channel(channel, max_videos=30)	@username or URL, limit	List of video dicts
get_playlist(url)	Playlist URL	Playlist dict with videos
get_transcript(url, lang="en")	Video URL, language code	List of transcript segments
get_comments(url, max_comments=100)	Video URL, limit	List of comment dicts
download(url, output, audio_only)	Video URL, filename, audio flag	Boolean success

Node.js Methods

Method	Parameters	Returns
getVideo(url)	Video URL	Promise
search(query, maxResults=20)	Search query, limit	Promise<Video[]>
getChannel(channel, maxVideos=30)	@username or URL, limit	Promise<Video[]>
getPlaylist(url)	Playlist URL	Promise
getTranscript(url, lang="en")	Video URL, language code	Promise<Transcript[]>
getComments(url, maxComments=100)	Video URL, limit	Promise<Comment[]>
download(url, options)	Video URL, {output, audioOnly}	Promise
exportJson(data, filename)	Data object, filepath	void

Notes

• Channel URLs: Accepts both @username format and full URLs
• Shorts detection: Automatically detects YouTube Shorts via URL path
• Duration formatting: Converts seconds to MM:SS or HH:MM:SS automatically
• Rate limiting: yt-dlp handles rate limiting internally
• Transcripts: Falls back to auto-generated captions if manual subtitles unavailable
• Comments: May take time to fetch on videos with many comments

Links

• yt-dlp GitHub
• yt-dlp Documentation

License

MIT