Fails to find youtube-dl when run from a startup script

[nuTTeLLo]

I am trying to get this to run as a service on a Synology NAS (DSM6) but am running into issues where it is not finding youtube-dl on the path even though it is specifically stated. Even when running youtube-dl within the script runs fine.

Here's my startup script podsync.sh (in /usr/local/etc/rc.d):

#!/bin/sh

start()
{
    echo "Using youtube-dl version `youtube-dl --version`"
    cd /volume1/Podsync
    /volume1/Podsync/podsync --config /volume1/Podsync/config.toml
}

stop()
{
    killall podsync
}

restart()
{
    stop
    start
}

case $1 in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
    *)
        echo $"Usage: $0 {start|stop|restart}"
        exit 1
esac

When running /usr/local/etc/rc.d/podsync.sh start I get this console output:

Using youtube-dl version 2020.01.01
INFO[2020-03-03T13:21:15+11:00]
 _______  _______  ______   _______           _        _______
(  ____ )(  ___  )(  __  \ (  ____ \|\     /|( (    /|(  ____ \
| (    )|| (   ) || (  \  )| (    \/( \   / )|  \  ( || (    \/
| (____)|| |   | || |   ) || (_____  \ (_) / |   \ | || |
|  _____)| |   | || |   | |(_____  )  \   /  | (\ \) || |
| (      | |   | || |   ) |      ) |   ) (   | | \   || |
| )      | (___) || (__/  )/\____) |   | |   | )  \  || (____/\
|/       (_______)(______/ \_______)   \_/   |/    )_)(_______/
INFO[2020-03-03T13:21:15+11:00] running podsync                               commit=9f4c926af1482fd1d22cc68275fee8b55548a775 date="2020-01-27T01:05:24Z" version=2.0.7
FATA[2020-03-03T13:21:15+11:00] youtube-dl error                              error="could not find youtube-dl: failed to execute youtube-dl: exec: \"youtube-dl\": executable file not found in $PATH"

We see that youtube-dl is pathed correctly from the first line, giving us back the version, but podsync itself fails to find it and errors out.

I am unable to use Docker on my NAS as it is not applicable. Running the Linux_i386 release version 2.0.7 of podsync.

[mxpv]

Hello. Can you share PATH env var and youtube-dl location on your system?

[nuTTeLLo]

Updated the startup script to print PATH and youtube-dl location.

#!/bin/sh

start()
{
    echo "Using youtube-dl version `youtube-dl --version` located in `which youtube-dl`"
    echo "Current PATH: " $PATH
    cd /volume1/Podsync
    /volume1/Podsync/podsync --config /volume1/Podsync/config.toml
}

stop()
{
    killall podsync
}

restart()
{
    stop
    start
}

case $1 in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
    *)
        echo $"Usage: $0 {start|stop|restart}"
        exit 1
esac

Running this directly with /usr/local/bin/podsync.sh start yields this output

Using youtube-dl version 2020.01.01 located in /usr/local/bin/youtube-dl
Current PATH:  /opt/bin:/opt/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/syno/sbin:/usr/syno/bin:/usr/local/sbin:/usr/local/bin
INFO[2020-03-11T18:00:08+11:00]
 _______  _______  ______   _______           _        _______
(  ____ )(  ___  )(  __  \ (  ____ \|\     /|( (    /|(  ____ \
| (    )|| (   ) || (  \  )| (    \/( \   / )|  \  ( || (    \/
| (____)|| |   | || |   ) || (_____  \ (_) / |   \ | || |
|  _____)| |   | || |   | |(_____  )  \   /  | (\ \) || |
| (      | |   | || |   ) |      ) |   ) (   | | \   || |
| )      | (___) || (__/  )/\____) |   | |   | )  \  || (____/\
|/       (_______)(______/ \_______)   \_/   |/    )_)(_______/
INFO[2020-03-11T18:00:08+11:00] running podsync                               commit=9f4c926af1482fd1d22cc68275fee8b55548a775 date="2020-01-27T01:05:24Z" version=2.0.7
FATA[2020-03-11T18:00:08+11:00] youtube-dl error                              error="could not find youtube-dl: failed to execute youtube-dl: exec: \"youtube-dl\": executable file not found in $PATH"

I want to also mention that the youtube-dl binary is symlinked to /usr/local/bin/youtube-dl.

[mxpv]

I'm not sure what's the reason of the problem. I've added the code to lookup for youtube-dl binary. Let's see if that helps.

[mxpv]

@nuTTeLLo can you try 2.1.0? (https://github.com/mxpv/podsync/releases/tag/v2.1.0)

[nuTTeLLo]

Just tried the latest version (2.2.2), still getting the same error I'm afraid.
Not sure why it won't work either.
I've done a workaround by running the podsync binary directly at boot time but its not the best case scenario but works.

[netravnen]

Bugger stupid error. Failed to notice the service had stopped on my node.

$ tail -n 2 /var/log/podsync.d/podsync.log 
time="2021-03-16T13:05:23Z" level=info msg="running podsync" commit=none date=unknown version=dev
time="2021-03-16T13:05:23Z" level=fatal msg="youtube-dl error" error="could not find youtube-dl: failed to execute youtube-dl: exit status 1"

Found this open thread about it. And got me thinking why not implement a runtime variable to be able to hardcode the absolute path to youtube-dl at runtime?

E.g. ExecStart=/usr/local/bin/podsync --config /etc/podsync.d/podsync.toml --youtubedl /usr/bin/youtube-dl

My workaround to get the service running again was modifying the system service unit. Adding in the Environment knob. Afterwards, I was able to get the service running again without issues.

# /etc/systemd/system/podsync.service
[Unit]
Description=Podsync
After=network.target

[Service]
ExecStart=/usr/local/bin/podsync --config /etc/podsync.d/podsync.toml
ExecStop=pkill -TERM podsync
StandardOutput=file:/var/log/podsync.d/podsync-stdout.log
StandardError=file:/var/log/podsync.d/podsync-stderr.log
Restart=on-failure
RestartSec=5
Environment="PATH=/usr/bin/youtube-dl:${PATH}"

[Install]
WantedBy=multi-user.target

${PATH} in the above example is set equals to the output of echo ${PATH} for which system user podsync is running under.

Version should be a commit around the time of the release of v2.4.0... ¯(°_o)/¯ Not entirely sure, as podsync does not implement a --version / -v flag. And log only mentions level=info msg="running podsync" commit=none date=unknown version=dev due to having had compiled it from source. Only the branch is mentioned, not the actual git commit from which the binary was compiled from. :(

$ ls -lah /usr/local/bin/podsync
-rwxr-xr-x 1 root staff 15M Oct 31 22:07 /usr/local/bin/podsync

[netravnen]

... That was short-live success. Still running into the issue. → Recompiling the binary from master branch to see if that changes anything.

... Nope :(

Seems the issue was with youtube-dl binary in my case.

/usr/bin/youtube-dl --version
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 6, in <module>
    from pkg_resources import load_entry_point
ModuleNotFoundError: No module named 'pkg_resources'

Forcing an re-install from debian testing repository did the trick.

/usr/bin/youtube-dl --version                   
2021.02.10

The Environment="PATH=/usr/bin/youtube-dl:${PATH}" variable in the system service file turned out to be unnessecary. ¯_(ツ)_/¯

.. Probably should pin that version to avoid automatic package upgrade of youtube-dl...(?)

[netravnen]

improvement idea

Output a more clear error message when youtube-dl binary is corrupted - as e.g. in my case - to the podsync log file.

Instead of the more misleading log msg “could not find youtube-dl”. Which make the issue seem to be something else than a corrupted youtube-dl binary.

[pclark1985]

I THINK the issue I'm running into is related to this. I used to just launch PodSync using a cron job, but it hasn't worked for a while. Running the exact same commend manually does work though. I just modified my cron job to save a log and it is saying:
level=fatal msg="youtube-dl error" error="youtube-dl binary not found: exec: \"youtube-dl\": executable file not found in $PATH"

I am running PodSync 2.4.0 on Ubuntu 20.10.

My cron job is:
@reboot sleep 300 && /home/peter/podsync/podsync --config /home/peter/podsync/config.toml > /home/peter/podsync/log/cron.log 2>&1

[sean89503]

This is due to windows missing a service pack ,the service pack has been removed from Microsoft website but the *.dll that youtube-dl need to run is available in a zip listed in this article https://www.winhelponline.com/blog/youtube-dl-error-msvcr100-dll-runtime-missing/