Increase "fragment stuck" timeout from 10s to 30s [try to avoid erroneously labeling slow video downloads as "stuck"]

[deldesir]

Fixes many videos failures due to progress polling hanging for some amount of time.

Tested on Ubuntu 24.10 (t4) but needs further testing from @EMG70 and others.

Related to #168

[holta]

This seems very safe to merge:

Is the "cost" a slower UX during downloads, while monitoring the "Tasks" view?

[deldesir]

The cost is real stuck videos will take longer to detect.

[EMG70]

A new VM was created after PR #169 was merged.
A huge improvement on stuck /failed downloads .The following pottery playlist https://www.youtube.com/playlist?list=PLbC14Txh1KZdaX5DkYYX8BBF3j9Bowhce previously had many failed downloads but now 29/30 videos downloaded OK after merging 169.
The only failed video https://www.youtube.com/watch?v=7B_GljX52w8 showed that error "Download appears to be stuck" as on screenshot.

Another attempt to download the failed video https://www.youtube.com/watch?v=7B_GljX52w8 individually also resulted in download stuck error as shown below suggesting there is a deeper problem with this single video.It however plays well on Youtube.

[holta]

The only failed video https://www.youtube.com/watch?v=7B_GljX52w8 showed that error "Download appears to be stuck" as on screenshot.

@deldesir plz investigate and aim for 100% of videos downloaded 🎯

Or if that's impossible, provide more clear explanations (with a "Googlable" explanation in "Tasks" view) for each of the main kinds of video download failure, allowing every intelligent teacher worldwide to self-empower their community to learn why exactly they're being blocked. 🌱

[deldesir]

Some unknown factors are at play there. Can I get access to the VM where the video failed or at least see the log? T4 tests still "PASS".

[EMG70]

I have shared OpenVpn details with Adam for you to access my machine

[holta]

Can I get access to the VM where the video failed or at least see the log?

1. @EMG70 can you share /var/log/calibre-web.log and /var/log/xklb.log as @deldesir needs? Run the following: (as root of course!)

   pastebinit -b dpaste.com /var/log/calibre-web.log
   pastebinit -b dpaste.com /var/log/xklb.log
   

2. Could https://youtu.be/7B_GljX52w8 download failures appear to be somewhat random — depending on network traffic congestion at that moment — or some such?

[deldesir]

I have shared OpenVpn details with Adam for you to access my machine

Thanks

[EMG70]

root@box:/var/log# pastebinit -b dpaste.com /var/log/calibre-web.log
https://dpaste.com/FZ4VBWP85

root@box:/var/log# pastebinit -b dpaste.com /var/log/xklb.log
https://dpaste.com/65S6CDVEW
root@box:/var/log#

[holta]

I have shared OpenVpn details with Adam for you to access my machine

@EMG70 also needed to configure SSH for incoming connections (recent versions of Ubuntu have tightened security!) as he's now done:

sudo -i
rm /etc/ssh/sshd_config.d/60-cloudimg-settings.conf
echo 'PasswordAuthentication yes' >> /etc/ssh/sshd_config
systemctl reload ssh

ASIDE: Thanks to PR iiab/iiab#3748, his iiab-diagnostics are now posted here: https://dpaste.com/A93HCTC8J

[holta]

root@box:/var/log# pastebinit -b dpaste.com /var/log/calibre-web.log https://dpaste.com/FZ4VBWP85

root@box:/var/log# pastebinit -b dpaste.com /var/log/xklb.log https://dpaste.com/65S6CDVEW

@deldesir also requested his /library/calibre-web/xklb-metadata.db so I zipped that up and attached it here:

xklb-metadata.db.zip

[deldesir]

Thanks @holta and @EMG70, the db file helped. The video downloaded but was not moved to /library/calibre-web.

[holta]

video downloaded but was not moved to /library/calibre-web.

Thanks @deldesir for confirming! Looks like:

• Some videos failed to move [from /library/downloads/calibre-web] to /library/calibre-web #165

As originally discussed here:

• Handle hanging due to unavailable fragments #137
• Remove residual fragments from failed video #157 (comment)