This is a bot that crawls over YouTube channels to collect videos from the channel
and also their descriptions.
This code is highly reusable. Although it is written to crawl over 2 Min papers channel, the functions in the code can be used independently.
In order to make selenium work with Firefox
$ wget https://github.com/mozilla/geckodriver/releases/download/v0.25.0/geckodriver-v0.25.0-linux64.tar.gz -O /tmp/geckodriver.tar.gz
$ tar -C /opt -xzf /tmp/geckodriver.tar.gz
$ chmod 755 /opt/geckodriver
$ ln -fs /opt/geckodriver /usr/bin/geckodriver
$ ln -fs /opt/geckodriver /usr/local/bin/geckodriver
[ ] save description for the failed ones and set a var to be false [ ] look at the failed ones and check more keyword like "Our paper"