using podsync for people who don't know very well programming

[ahmedahmedo]

1. get YouTube API key
2. download and install docker
3. install youtube-dl for your operation system
4. create configuration file(config.toml) in default folder
   a. insert YouTube API key
   b. insert YouTube channel url
5. pull and run podsync files via docker
6. open web browser and go http://localhost:8080, you will see xml files, copy link and paste your rss reader.

this tutor is very basic, please someone edit by means of programming and English

[elsiehupp]

Does Podsync strictly require Docker? If no, is there a reason I should use Docker? And if I'm not using Docker, how would I install Podsync without it?

I tried using git and make, and it failed on a ton of dependencies from GitHub; then I tried using Docker, and I didn't even know where to start...

How realistic would it be for me to be able to install Podsync from something like apt?

[elsiehupp]

Here's what I got when I tried to make the git clone:

$ make
go build -o bin/podsync ./cmd/podsync
cmd/podsync/updater.go:13:2: cannot find package "github.com/hashicorp/go-multierror" in any of:
/usr/lib/go-1.10/src/github.com/hashicorp/go-multierror (from $GOROOT)
/home/rss/go/src/github.com/hashicorp/go-multierror (from $GOPATH)
cmd/podsync/main.go:12:2: cannot find package "github.com/jessevdk/go-flags" in any of:
/usr/lib/go-1.10/src/github.com/jessevdk/go-flags (from $GOROOT)
/home/rss/go/src/github.com/jessevdk/go-flags (from $GOPATH)
cmd/podsync/updater.go:17:2: cannot find package "github.com/mxpv/podsync/pkg/builder" in any of:
/usr/lib/go-1.10/src/github.com/mxpv/podsync/pkg/builder (from $GOROOT)
/home/rss/go/src/github.com/mxpv/podsync/pkg/builder (from $GOPATH)
cmd/podsync/main.go:17:2: cannot find package "github.com/mxpv/podsync/pkg/config" in any of:
/usr/lib/go-1.10/src/github.com/mxpv/podsync/pkg/config (from $GOROOT)
/home/rss/go/src/github.com/mxpv/podsync/pkg/config (from $GOPATH)
cmd/podsync/main.go:18:2: cannot find package "github.com/mxpv/podsync/pkg/db" in any of:
/usr/lib/go-1.10/src/github.com/mxpv/podsync/pkg/db (from $GOROOT)
/home/rss/go/src/github.com/mxpv/podsync/pkg/db (from $GOPATH)
cmd/podsync/updater.go:20:2: cannot find package "github.com/mxpv/podsync/pkg/feed" in any of:
/usr/lib/go-1.10/src/github.com/mxpv/podsync/pkg/feed (from $GOROOT)
/home/rss/go/src/github.com/mxpv/podsync/pkg/feed (from $GOPATH)
cmd/podsync/main.go:19:2: cannot find package "github.com/mxpv/podsync/pkg/fs" in any of:
/usr/lib/go-1.10/src/github.com/mxpv/podsync/pkg/fs (from $GOROOT)
/home/rss/go/src/github.com/mxpv/podsync/pkg/fs (from $GOPATH)
cmd/podsync/updater.go:22:2: cannot find package "github.com/mxpv/podsync/pkg/model" in any of:
/usr/lib/go-1.10/src/github.com/mxpv/podsync/pkg/model (from $GOROOT)
/home/rss/go/src/github.com/mxpv/podsync/pkg/model (from $GOPATH)
cmd/podsync/main.go:20:2: cannot find package "github.com/mxpv/podsync/pkg/ytdl" in any of:
/usr/lib/go-1.10/src/github.com/mxpv/podsync/pkg/ytdl (from $GOROOT)
/home/rss/go/src/github.com/mxpv/podsync/pkg/ytdl (from $GOPATH)
cmd/podsync/updater.go:14:2: cannot find package "github.com/pkg/errors" in any of:
/usr/lib/go-1.10/src/github.com/pkg/errors (from $GOROOT)
/home/rss/go/src/github.com/pkg/errors (from $GOPATH)
cmd/podsync/main.go:13:2: cannot find package "github.com/robfig/cron/v3" in any of:
/usr/lib/go-1.10/src/github.com/robfig/cron/v3 (from $GOROOT)
/home/rss/go/src/github.com/robfig/cron/v3 (from $GOPATH)
cmd/podsync/main.go:14:2: cannot find package "github.com/sirupsen/logrus" in any of:
/usr/lib/go-1.10/src/github.com/sirupsen/logrus (from $GOROOT)
/home/rss/go/src/github.com/sirupsen/logrus (from $GOPATH)
cmd/podsync/main.go:15:2: cannot find package "golang.org/x/sync/errgroup" in any of:
/usr/lib/go-1.10/src/golang.org/x/sync/errgroup (from $GOROOT)
/home/rss/go/src/golang.org/x/sync/errgroup (from $GOPATH)
cmd/podsync/main.go:22:2: cannot find package "gopkg.in/natefinch/lumberjack.v2" in any of:
/usr/lib/go-1.10/src/gopkg.in/natefinch/lumberjack.v2 (from $GOROOT)
/home/rss/go/src/gopkg.in/natefinch/lumberjack.v2 (from $GOPATH)
Makefile:12: recipe for target 'build' failed
make: *** [build] Error 1

Well, I went ahead and painstakingly downloaded each and every one of the dependencies with go get, then tried make again only to get this:

$ make
go build -o bin/podsync ./cmd/podsync
# github.com/hashicorp/go-multierror
../go/src/github.com/hashicorp/go-multierror/multierror.go:112:9: undefined: errors.As
../go/src/github.com/hashicorp/go-multierror/multierror.go:117:9: undefined: errors.Is
Makefile:12: recipe for target 'build' failed
make: *** [build] Error 2

Any thoughts on where I should go from here?

[elsiehupp]

(Note that I have no idea what I'm doing when it comes to go.)

[ahmedahmedo]

this procedure actually for Mac OS user, I don't know your operation system but you can apply this procedure your operation system too.
I think firstly you should search how to install docker and how to install any app into the docker.
docker likes operation system.
first install docker after that install podsync into the docker.

maybe @mxpv can prepare elaborate how to guide

[s39n]

(Note that I have no idea what I'm doing when it comes to go.)

What operating system are you using. I have mine running on windows. But have got Linux in the past so I can help

[elsiehupp]

What operating system are you using. I have mine running on windows. But have got Linux in the past so I can help

I'm using Ubuntu 18.04 LTS. I'm trying again with Docker—the problem I had before seems to have been that the user I was logged in with was not part of the docker group—and I'll see if I can get it to work this time. It would still be nice to have proper instructions for installing outside of Docker even if I get Docker to work, though.

[elsiehupp]

I have Podsync (apparently) running now in Docker on Ubuntu, and I have a few questions:

1. Where is the correct location for the config.toml file? Why does the install seemingly create a directory with that name rather than a file?
2. How do I test my first feed on a headless server, i.e. not localhost?
3. What does "404 page not found" (not the regular Apache 404 page) in my web browser mean? Is this a firewall issue?

For the time being you can reference these URLs:
http://rss.elsiehupp.com (Apache virtual host on port 80)
https://rss.elsiehupp.com (Apache virtual host on port 443)
http://rss.elsiehupp.com:8080 (blank page)
http://rss.elsiehupp.com:8080/ID1.xml ("404 page not found")

4. Is it possible to get Podsync RSS feeds running on port 80 or 443 alongside (or inside) Apache?

[s39n]

What operating system are you using. I have mine running on windows. But have got Linux in the past so I can help

I'm using Ubuntu 18.04 LTS. I'm trying again with Docker—the problem I had before seems to have been that the user I was logged in with was not part of the docker group—and I'll see if I can get it to work this time. It would still be nice to have proper instructions for installing outside of Docker even if I get Docker to work, though.

Ya that would definitely be an issue. I think the instructions are not to detailed that way you can install it on any operating system.

[elsiehupp]

To reproduce where I am right now, follow these steps on an Ubuntu 18.04 x64 system:

Install prerequisites from apt

1. sudo apt-get install ffmpeg
2. sudo apt-get install youtube-dl
3. sudo apt-get install make

Set up correct version of go

4. sudo apt-get install golang-go !! Installs wrong version of Go. Podsync requires 1.13.6.
   wget https://dl.google.com/go/go1.13.6.linux-amd64.tar.gz
5. sudo tar -C /usr/local -xzf go1.13.6.linux-amd64.tar.gz
6. nano ~/.profile
   at end of file add:
   export GOROOT=/usr/local/go
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
   Save and exit.
   (This could probably be done better with echo "something" >> ~./profile, but I haven't tested it.)

Install podsync from source

7. git clone https://github.com/mxpv/podsync
8. nano ./podsync/Makefile
9. After build: add a new line with go get ./... (to dynamically include dependencies), save and exit.
10. cd ./podsync
11. make
12. go test -v ./… (should run diagnostic tests)

Set up and run podsync

13. nano ./bin/config.toml
14. Paste sample config.tomlfrom README.md, save and exit.
15. To run podsync run ~/podsync/bin/podsync (It appears the contents of the bin folder can be placed wherever you want them.)

At this point I am running into several problems:

1. The podsync.log file is formatted for cat, with strings like ^[[31m, ^[[36m, and ^[[0m that don't work in editors like nano. These should probably be stripped out.
2. Auto-updating youtube-dl doesn't seem to work when it has been installed from apt-get. I feel like this should be more explicit.

INFO[2020-06-22T23:04:08Z] updating youtube-dl
ERRO[2020-06-22T23:04:09Z] [debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--update', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.03.14
[debug] Python version 3.6.9 (CPython) - Linux-4.15.0-106-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.6-0ubuntu0.18.04.1, ffprobe 3.4.6-0ubuntu0.18.04.1, phantomjs ., rtmpdump 2.4
[debug] Proxy map: {}
Usage: youtube-dl [OPTIONS] URL [URL...]

youtube-dl: error: youtube-dl's self-update mechanism is disabled on Debian.
Please update youtube-dl using apt(8).
See https://packages.debian.org/sid/youtube-dl for the latest packaged version.
error="failed to execute youtube-dl: exit status 2"
ERRO[2020-06-22T23:04:09Z] failed to update youtube-dl error="failed to self update youtube-dl: failed to execute youtube-dl: exit status 2"

3. I'm getting the following error when trying to update a feed:

ERRO[2020-06-22T23:11:18Z] failed to update feed: https://www.youtube.com/user/LinusTechTips error="update failed: failed to query channel: context canceled"

Is the feed URL properly formatted? I tried using a YouTube RSS feed, and podsync didn't seem to like this, either.

Thoughts

I don't know about all y'all, but the fact that the installation requires a specific version of go, and I had to find this out through trial and error is kind of a problem.

The missing dependencies call in the Makefile might be a bug, or it might be an artifact of the authors of podsync intending for it to be built in a different environment, perhaps using go get github.com/mxpv/podsync instead of git and make.

Similarly, if podsync is going to use its own update mechanism for youtube-dl it should be smarter about not conflicting with apt.

For initial setup it would be ideal for there to be a preexisting sample config.toml with real values so that the user can confirm that the setup is, in fact, working. There is a test script go test -v ./… that uses the source, so perhaps that could be reproduced as a shell script for testing the binary. (I would kind of prefer using a shell script rather than a configuration file for production, and it would be nice to have fully documented command-line settings.)

Finally, this is still not working for me. Any thoughts of how I could resolve the failed to query channel error? Once I get the YouTube query working hopefully the only step remaining will be to get the RSS feeds running and at the URLs I want. (I'll probably think of or figure more things out on my own, but this is where I am at this exact point in time.)

[elsiehupp]

Trying to load podsync using go...

~$ go get github.com/mxpv/podsync
can't load package: package github.com/mxpv/podsync: no Go files in /usr/local/go/src/github.com/mxpv/podsync

Trying to load podsync in the go src directory:

/usr/local/go/src$ go get github.com/mxpv/podsync
go: finding github.com/mxpv/podsync latest
go: downloading github.com/mxpv/podsync v0.0.0-20200609053654-00c7bce6dc69
go: extracting github.com/mxpv/podsync v0.0.0-20200609053654-00c7bce6dc69
go: writing go.sum: open /usr/local/go/src/go.sum87694162.tmp: permission denied

Trying to build podsync in the go src directory rather than in ~:

/usr/local/go/src/github.com/mxpv/podsync$ make
go build -o bin/podsync ./cmd/podsync
go: inconsistent vendoring in /usr/local/go/src/github.com/mxpv/podsync:
        go.mod requires github.com/mxpv/podsync but vendor/modules.txt does not include it.
        run 'go mod tidy; go mod vendor' to sync
Makefile:12: recipe for target 'build' failed
make: *** [build] Error 1

I tried go mod tidy; go mod vendor, and it didn't make any difference in terms of the above error.

[elsiehupp]

I did some fussing, and it seems like there were two problems with my above setup:

1. youtube-dl is constantly being updated, and older versions are generally broken. apt-get installs a version of youtube-dl from 2018, which obviously no longer works.
2. youtube-dl is rather fussy with URLs. In my limited experimentation, it seems to work well with URLs formatted as https://www.youtube.com/playlist?list=XXXXXXX.

I was able to get podsync up and running, but I have a few remaining issues:

1. The built-in update mechanism for youtube-dl just... doesn't seem to work (because I'm not using Docker?). It appears I will need to set up youtube-dl updates with cron, I guess? I could migrate my podsync scheduling to cron, as well, just to keep everything in one place. I installed the latest youtube-dl using curl, and podsync.log included the following message:

It looks like you installed youtube-dl with a package manager, pip, setup.py or a tarball. Please use that to update.

2. The way I'm using podsync is that it's just dumping the RSS feeds and episode files in an Apache server, so I would rather not be running podsync's built-in server. It seems like I can ignore it, but podsync seems designed to run in the background as a daemon, so I will have to see how migrating to cron affects that.
3. Executing the podsync binary locks up the Terminal, even if I try to run it in the background with podsync &. This may not be a problem if I run podsync via cron, and closing the terminal window doesn't seem to interrupt the podsync process, but it's still rather annoying.
4. There isn't an obvious place to put the podsync executable, especially since it requires the config.toml file to be in the same directory and proceeds to dump podsync.log in the same directory, as well. It would be nice if podsync could be installed in /usr/bin/ like a normal application and take command line arguments from a shell script, like $podsync --config=PATH_TO_TOML --log=PATH_TO_LOG, ya know?

I will try and follow up with updated Ubuntu installation instructions once I'm able to fiddle some more.

[bcarney66]

@ahmedahmedo:

4. create configuration file(config.toml) in default folder

I've followed these steps, but can't figure out where the "default folder" is on OS X. Can you point out where it should go? Thank you.