Can't seem to authenticate

[ArtyomFR]

Hi,
I wan't use this plugin with a calibre-web generated opds.
My server is private and authentification must be use.
Does this plugin support basic authentification from calibre-web?
Thanks for your help!

[steinarb]

Hi, This is no such functionality right now. Also, since this isn't something that I need, it is unfortunately not likely to be implmenented by me. :-) But I agree this would be a reasonable thing to have, and I will take and merge reasonable PRs. A reasonable PR would be to pop up a dialog prompting for credentials when receiving a 401 HTTP code on loading a feed or downloading a book. It probably be best not to persist the credentials (unless calibre already has a mechanism for this...? I've no idea if it does). Note also that urllib and feedparser may allow the workaround listed at the start of this page (adding username and password to the URL). https://pythonhosted.org/feedparser/http-authentication.html#downloading-a-feed-protected-by-basic-authentication-the-easy-way

[OdinVex]

Feedback, adding auth details to uri does not work, unfortunately. A mechanism to check for HTTP 401 should be added.

[steinarb]

Yes, but not something I need, so not something I will spend time on.

[qbit]

FWIW, you can authenticate with http basic auth. Set the url to: http://username:password@server.local:8080/opds and it should work as expected (assuming your service is using http basic auth :P).

[qbit]

Sorry, I was mistaken, it partially works with the http auth trick. The urls returned within the opds data don't have the auth info associated with them.

[qbit]

Took a crack at this. I got the plugin working (via non-commitable hacks :P) with http basic auth, however, I hit a wall because the calibre download_ebook function doesn't properly handle URLs with the auth parameters set.

Here is a diff of what I have so far:

diff --git a/calibre_plugin/main.py b/calibre_plugin/main.py
index 1b6e6ff..1b1ca98 100644
--- a/calibre_plugin/main.py
+++ b/calibre_plugin/main.py
@@ -202,6 +202,7 @@ class OpdsDialog(QDialog):
 
     def downloadBook(self, book):
         if len(book.links) > 0:
+            # TODO: make a browser() implementation that works with urllib.parse.urlparse'd urls
             self.gui.download_ebook(book.links[0])
 
     def fixBookTimestamps(self):
diff --git a/calibre_plugin/model.py b/calibre_plugin/model.py
index 0e905e2..f5d02a3 100644
--- a/calibre_plugin/model.py
+++ b/calibre_plugin/model.py
@@ -66,6 +66,7 @@ class OpdsBooksModel(QAbstractTableModel):
         return None
 
     def downloadOpdsRootCatalog(self, gui, opdsUrl, displayDialogOnErrors):
+        self.urlInfo = urllib.parse.urlparse(opdsUrl)
         feed = feedparser.parse(opdsUrl)
         if 'bozo_exception' in feed:
             exception = feed['bozo_exception']
@@ -95,6 +96,7 @@ class OpdsBooksModel(QAbstractTableModel):
         return (firstTitle, catalogEntries)
 
     def downloadOpdsCatalog(self, gui, opdsCatalogUrl):
+        opdsCatalogUrl = re.sub(":\/\/", "://"+self.urlInfo.username + ":" + self.urlInfo.password + "@", opdsCatalogUrl)
         print("downloading catalog: %s" % opdsCatalogUrl)
         opdsCatalogFeed = feedparser.parse(opdsCatalogUrl)
         self.books = self.makeMetadataFromParsedOpds(opdsCatalogFeed.entries)
@@ -102,6 +104,7 @@ class OpdsBooksModel(QAbstractTableModel):
         QCoreApplication.processEvents()
         nextUrl = self.findNextUrl(opdsCatalogFeed.feed)
         while nextUrl is not None:
+            nextUrl = re.sub(":\/\/", "://"+self.urlInfo.username + ":" + self.urlInfo.password + "@", nextUrl)
             nextFeed = feedparser.parse(nextUrl)
             self.books = self.books + self.makeMetadataFromParsedOpds(nextFeed.entries)
             self.filterBooks()
@@ -156,7 +159,11 @@ class OpdsBooksModel(QAbstractTableModel):
         except AttributeError:
             rawTimestamp = "1980-01-01T00:00:00+00:00"
         parsableTimestamp = re.sub('((\.[0-9]+)?\+0[0-9]:00|Z)$', '', rawTimestamp)
-        metadata.timestamp = datetime.datetime.strptime(parsableTimestamp, '%Y-%m-%dT%H:%M:%S')
+        fmt = '%Y-%m-%dT%H:%M:%S'
+        try:
+            metadata.timestamp = datetime.datetime.strptime(parsableTimestamp, fmt)
+        except:
+            metadata.timestamp = datetime.datetime.strptime(parsableTimestamp, fmt+'%z')
         tags = []
         summary = opdsBookStructure.get(u'summary', u'')
         summarylines = summary.splitlines()
@@ -171,6 +178,8 @@ class OpdsBooksModel(QAbstractTableModel):
         links = opdsBookStructure.get('links', [])
         for link in links:
             url = link.get('href', '')
+            url = re.sub(":\/\/", "://"+self.urlInfo.username + ":" + self.urlInfo.password + "@", url)
+            # url = urllib.parse.urlparse(url)
             bookType = link.get('type', '')
             # Skip covers and thumbnails
             if not bookType.startswith('image/'):
@@ -184,9 +193,10 @@ class OpdsBooksModel(QAbstractTableModel):
         return metadata
 
     def findNextUrl(self, feed):
-        for link in feed.links:
-            if link.rel == u'next':
-                return link.href
+        if "links" in feed:
+            for link in feed.links:
+                if link.rel == u'next':
+                    return link.href
         return None
 
     def downloadMetadataUsingCalibreRestApi(self, opdsUrl):
@@ -200,6 +210,7 @@ class OpdsBooksModel(QAbstractTableModel):
         # a meaningful timestamp for the books
 
         # Get the base of the web server, from the OPDS URL
+        print("HERE", opdsUrl)
         parsedOpdsUrl = urllib.parse.urlparse(opdsUrl)
 
         # GET the search URL twice: the first time is to get the total number
@@ -233,6 +244,6 @@ class OpdsBooksModel(QAbstractTableModel):
             bookMetadata = bookMetadataById[book.uuid]
             rawTimestamp = bookMetadata['timestamp']
             parsableTimestamp = re.sub('(\.[0-9]+)?\+00:00$', '', rawTimestamp)
-            timestamp = datetime.datetime.strptime(parsableTimestamp, '%Y-%m-%dT%H:%M:%S')
+            timestamp = datetime.datetime.strptime(parsableTimestamp, '%Y-%m-%dT%H:%M:%S%z')
             book.timestamp = timestamp
         self.filterBooks()

There is some stuff mixed in to make things work with Readeck. I think (not familiar with python or the calibre apis) a proper fix would be to implement a Browser that can properly handle properly parsed URLs.

[qbit]

I had also tried to pass feedparser.parse a handlers=[auth] parameter, where auth was an instance of HTTPBasicAuthHandler (outlined here: https://feedparser.readthedocs.io/en/latest/http-authentication.html#downloading-a-feed-protected-by-http-basic-authentication-the-hard-way), but that didn't work for some reason.