Skip to content

IIAB Calibre‐Web

Jump to bottom Edit New page
deldesir edited this page Feb 5, 2025 · 58 revisions

Build your own "YouTube/Vimeo[*] Learning Library" using Internet-in-a-Box!

Calibre-Web (initial release 2015) is an open-source, web-based eBook server based on the popular Calibre eBook management tool (initial release 2006). It provides a user-friendly interface for organizing, converting, reading and listening to e-books.

Starting in 2023/2024, Calibre-Web is being modified for Internet-in-a-Box (IIAB) to not only allow eBooks' collection and organization, but also the learning videos, audiocasts and images. This fork called IIAB Calibre-Web aims to help educators and communities to download the videos and audio recordings they truly need, from almost 2000 websites.

  1. Getting Started ✅✅✅

  2. Media Management ✅❌❌

  3. Metadata Management ✅✅❌

  4. User Guide ✅❌❌

  5. Installation and Setup ✅❌❌

  6. Troubleshooting and Known Issues ✅✅❌

  7. Community and Feedback ❌❌❌

  8. Developer Guide ❌❌❌

  9. Resources ❌❌❌

1. Getting Started

Getting familiar with IIAB Calibre-Web is straightforward. Once you understand the layout and know where to find key options and buttons, you'll be ready to make the most of its features.

In this section, you'll learn how to:

Accessing IIAB Calibre-Web

To open IIAB Calibre-Web, you’ll need the IP address pointing to the virtual machine (VM) where it’s installed. This guide assumes you’ve set up IIAB Calibre-Web on Windows or Linux using a virtual machine with Multipass. If you’re new to this process, refer to our installation with Multipass instructions for step-by-step guidance.

🍒 IIAB on Raspberry Pi: If your IIAB Calibre-Web is installed as part of an Internet-in-a-Box (IIAB) installation on a Raspberry Pi, this information is still applicable. However, instead of an IP address, it's more convenient to use http://box.lan/books to access it.

Navigating the Interface

1. Access the Web Interface

Open a web browser and go to:

http://<VM-IP-Address>/books

Replace <VM-IP-Address> with the actual IP address of your virtual machine (retrievable via multipass list or hostname -I commands).

🍒 Guest Account: The homepage you see initially is configured for the Guest account. It has limited access but allows you to browse and view the available books and videos in the library.

2. Log In for Full Access

To enable advanced features like uploading books and videos, downloading content from YouTube, or editing metadata and managing application settings, you'll need to log in with the admin account.

Click the "Guest" button in the top-right corner and enter the following credentials:

  • Username: Admin
  • Password: changeme

Animation2

Once logged in, you can:

  • Explore your library
  • Read books or watch videos
  • Upload new content
  • Download videos directly from YouTube

For more detailed instructions on managing your library, refer to the next section. You can also start customizing various aspects of your IIAB Calibre-Web by referring to the "Customizing your preferences" instructions in the User Guide section.

2. Media Management

Effectively managing your digital media is at the core of IIAB Calibre-Web. It allows you to add ebooks, videos, and other files in various formats (EPUB, PDF, MP4, MP3, JPEG, PNG) through simple uploading or downloading from online sources.

Uploading Media

Use the Upload button to upload any ebook, audiobook or video.

Uploading Process

  1. Select the desired file(s) from your source directory.
  2. Assign relevant metadata or edit existing details before confirming the upload.
  3. Review the uploaded media in your library.

Downloading Media

We support downloading from popular videos platforms like YouTube and Vimeo.

Supported URLs

For YouTube, our support extends to a wide range of URLs, making it incredibly versatile:

For Vimeo, while experimental, we can handle basic URLs. Do note that Vimeo covers are not currently working.

Tweaking Video Quality

Download Process and Steps

After you log into Calibre-Web (at http://box/books), a Download to IIAB button is displayed near the top of the page (if your Calibre-Web user's role allows for it), allowing you to directly import the educational videos (and their metadata) that your students need.

  1. Enter the URL of the desired video via the Download to IIAB button.

    🍒 Bookshelf creation: If you specify the URL for a YouTube channel or playlist or similar, a Calibre-Web bookshelf will be auto-created to store them. If the bookshelf name is already taken, a numbered suffix will be added to create a unique bookshelf name. Finally, a suitable subset of the channel or playlist's videos (e.g., 100 videos) will be added to the bookshelf, one at a time, over ensuing minutes (or hours if necessary!)

  1. Monitor the download progress in Tasks page and review the item once complete.

3. Metadata Management

Effective media organization wouldn’t be possible without metadata. Each educational resource is associated with key information such as titles, descriptions, authors, and more.

This section covers:

Metadata Extraction and Preservation

Whether it's an ebook, audiobook, or video, IIAB Calibre-Web is designed to automatically extract and store metadata whenever it's available. The extraction methods depend on the file format and the context of the action—upload or download.

  • For Uploaded Files: Metadata is read directly from the file, provided it is embedded within the resource.
  • For Downloaded Files: Metadata is extracted from the source’s available information. For example, in the case of YouTube or similar platforms, metadata is collected based on the details provided at the time of download.

For a deeper dive into the tools and database schema used to manage this metadata, refer to the Developer Guide.

Editing Metadata

Metadata in IIAB Calibre-Web is not immutable. If your account has the necessary permissions (e.g., an Admin account), you can edit the metadata for any item in your library. Simply click on the media item and select the "Edit Metadata" button on its information page.

  • Ensure the modifications you make are accurate and adhere to copyright guidelines.
  • For popular ebooks, you can fetch metadata automatically from online services like Google Books, Amazon, and others.

Metadata Handling

Metadata serves more than just organizational purposes; it also powers IIAB Calibre-Web's advanced search functionality. This enables you to search not only by title but also by description, author, and other metadata fields.

For videos, if you imported the item using the "Download to IIAB" button, you can even search captions linked to the video.

Development is ongoing to allow metadata export and manipulation via APIs. For more information, consult the API Documentation.

Cover Art Extraction

When uploading or downloading a video, IIAB Calibre-Web attempts to use the associated cover art if one exists. If no cover art is found, a default placeholder is used.

  • For videos, FFmpeg is utilized to generate a cover from the video content itself.

For further details, visit the Cover Generation section.

4. User Guide

IIAB Calibre-Web provides a bookshelf-like experience for your videos, making it easy to organize and manage your different collections.

This section covers:

Browsing and Searching Media

Uploaded and downloaded videos are automatically organized by date. You can browse them by 15 different criteria listed under "BROWSE" section on the home page:

Books Hot Books Downloaded Books
Top Rated Books Read Books Unread Books
Discover Categories Series
Authors Publishers Languages
Ratings File formats Archived Books

You can search for videos by typing in titles, in the search bar (available in the top of every page of IIAB Calibre-Web). If your search is criteria-based, you can click on the "Advanced Search" button instead.

NOT YET IMPLEMENTED: Another useful feature is "captions" search. If a video contains captions or subtitles, IIAB Calibre-Web can identify them by key terms. For example, if you download a video from YouTube and you forget its title, you can search for a keyword from the captions or subtitles, and IIAB Calibre-Web will find the relevant video for you.

Managing Collections and Bookshelves

Bookshelves help you group related videos, making it easier to find, share, and manage your collections. This is especially useful for large libraries or when sharing categorized videos with specific audiences. You can add individual videos to an existing bookshelf, or have them automatically included when you download playlists and channels.

Only administrators and users with the required permissions can create bookshelves. Once logged in with the correct permissions, the "Create a Bookshelf" button will appear on the homepage. Clicking the button will open a form where you can enter a title for your bookshelf and choose whether to make it publicly accessible.

Adding Videos Manually

To add a video to a bookshelf manually, first click on the video. This will open a modal dialog displaying the video's metadata (title, description, etc.). At the bottom of this dialog, you'll find an "Add to Shelf" button. Click this button, and you'll be presented with a list of your existing bookshelves. Select the one you want to add the video to. A single video can belong to multiple bookshelves.

Automatic Bookshelf Creation and Population

IIAB Calibre-Web can automatically create and populate bookshelves when you download content from certain sources. This typically happens when you click "Download to IIAB" button and you specify the URL for a YouTube channel, playlist, or similar source.

If a bookshelf with the suggested name already exists, a numbered suffix will be added to the name to ensure uniqueness (e.g., "My Playlist" becomes "My Playlist 1"). IIAB Calibre-Web will then begin adding videos from the channel or playlist to the newly created bookshelf. By default, only a subset of the videos (the top 100) will be added initially. You can adjust this limit if needed.

Once created, you can always edit the bookshelf by adding more videos, removing unwanted ones, or renaming it to reflect its content better.

Sorting Media in Bookshelves

Sorting your media within bookshelves can make browsing through them more efficient.

By default, videos added as part of a playlist or channel download are automatically sorted according to the original ordering of that playlist or channel. This usually reflects the video's popularity or upload date, based on views per year.

IIAB Calibre-Web also allows you to reorder your videos manually within bookshelves by dragging them into place, or you can sort them by date, by title or by author using the appropriate buttons.

Customizing Preferences

IIAB Calibre-Web offers various configuration options and system behavior to meet your needs. These settings can be adjusted through the Admin menu.

Users

  • Edit Users
  • Add New User

Email Server Settings

  • Edit Email Server Settings

Configuration

  • Edit Calibre Database Configuration: Specify the path to your Calibre metadata.db file. This file, created by Calibre, stores your book metadata. Ensure the database file resides on a mounted drive accessible to the server running IIAB Calibre-Web. Use either forward slashes (/) or backslashes () as directory separators; quotes are not required. Example: /library/calibre-web/metadata.db

  • Edit Basic Configuration: Adjust fundamental system settings, including server parameters, and logging options.

    • Server Configuration

      • Server Port: Define the port on which IIAB Calibre-Web listens for connections. Changes to this setting require a server restart to take effect. Remember to update the URL in your browser accordingly.

      • SSL Configuration: Configure IIAB Calibre-Web for secure connections using SSL. Provide the paths to your certificate file and key file. Incorrect configuration can be overridden using command-line options: -c [certfile location] -k [keyfile location]. Setting both values to an empty string ("") will disable SSL.

      • Update Channel: Choose between two update channels: CAUTION - Use instructions for Upgrading IIAB Calibre-Web instead.

        • Stable Channel (default): Recommended for production environments, offering reliable and tested updates.
        • Nightly Channel: Provides access to the latest features and bug fixes but may be less stable.

      • Trusted Hosts: Specify a comma-separated list of additional trusted hosts. This is useful for integration with third-party theming or proxies.

    • Logfile Configuration

      • Log Level: Control the verbosity of the logs:

        • DEBUG: Provides the most detailed logs, ideal for troubleshooting.
        • INFO (default): The standard logging level for normal operation.
        • WARNING and ERROR: Minimal logging, only recording significant issues.

      • Logfile Location and Name: Customize the path and name of the log file. The default location is /var/log/calibre-web.log.

        • Example (Windows): C:\logs\calibre-web.log
        • Example (Linux): /var/log/calibre-web.log

      • Enable Access Log: Enable a separate log file (access.log) to track all incoming requests. The path to this log file can be customized similarly to the main log file.

    • Feature Configuration: This section lets you enable or disable various features, such as uploads, anonymous browsing, and integrations.

      • Enable Uploading: Allows users to upload media files (PDF, EPUB, MP4, MP3, DJVU, CBZ, CBT, and more). The allowed file types can be further customized under "Allowed Upload File Formats." If ImageMagick is installed, cover images can be extracted from certain file types (e.g., PDF, EPUB, CBZ, CBT). Similarly, if rarfile is installed, covers can be extracted from CBR files.

      • Enable Anonymous Browsing: Permits users to browse the library without logging in. Administrators can configure permissions for guest users.

      • Enable Public Registration: Allows new users to register themselves. This requires SMTP configuration for email verification. You can optionally require email-based usernames and restrict registration to specific email domains.

      • Enable Remote Login (Magic Link): Enables users to log in on one device (e.g., an e-reader) by scanning a QR code or entering a magic link generated on another logged-in device.

      • Enable Kobo Sync: Enables synchronization of books with Kobo e-readers. See the Kobo Sync Configuration section for details.

      • Allow Reverse Proxy Authentication: Enables authentication to be handled by an upstream proxy. Use this option only in trusted Single Sign-On (SSO) environments.

  • Edit UI Configuration: Customize the look and feel of the IIAB Calibre-Web interface.

    • Title Customization: Change the instance name displayed in the top-left corner.

    • Books per Page: Control the number of books displayed per page. Adjust this setting to use pagination instead of infinite scrolling.

    • Number of Random Books Displayed: Set the number of books shown in the Random Books section.

    • Number of Authors Displayed Before Hiding: Limit the number of authors displayed for books with multiple authors to prevent clutter. Set to 0 to show all authors.

    • Theme Selection: Choose between the Standard (Light) Theme and Dark Mode (Caliblur).

    • Regular Expression for Ignoring Columns: Use regular expressions to exclude specific custom columns from the interface. Example: .* (excludes all custom columns), Read (excludes only the "Read" column).

    • Link Read/Unread Status to Calibre Column: In single-user environments, you can synchronize read/unread status with a custom Boolean column in Calibre. Do not use this in multi-user setups.

    • Visibility Restrictions Based on Custom Columns: Restrict book visibility based on custom column values. For example, hide books tagged "Adult" from child users.

    • Visibility Restrictions Based on Tags: Similar to custom column restrictions, but based on book tags.

  • Scheduled Tasks:

  • Edit Scheduled Tasks Settings

Administration

  • Download Debug Package
  • View Logs
  • Reconnect Calibre Database
  • Restart
  • Shutdown

Version Information

  • Check for Update

5. Installation and Setup

As of December 2024, the Multipass virtual machine (VM) system is the easiest way to install IIAB Calibre-Web.

System Requirements

Before you dive in the installation process, ensure the following system requirements are met:

  • Operating System: Linux, Windows 10+ (Pro preferred), or macOS
  • Hardware: A computer with at least 8 GB RAM
  • Multipass: Version 1.14.1+ is required. Ensure your version is up-to-date by running: 
    multipass version

    ⚠️ Note for Windows Users: Windows Home requires VirtualBox for Multipass. Ensure hardware virtualization (e.g., Intel VT-x or AMD-V) is enabled in your BIOS. Follow this guide for enabling virtualization.

Installation Steps

[text placeholder]

  1. Install Multipass
    Download and install Multipass 1.14.1+ from multipass.run.

    • On Windows or macOS: Ensure your system is fully updated before installation.
    • On Linux: Verify compatibility and install via the official package manager.

  2. Prepare the Configuration File [text placeholder]

    • On Linux, download the configuration file by running: 
      curl https://raw.githubusercontent.com/iiab/calibre-web/master/scripts/omg.yml > omg.yml
    • On Windows or macOS, manually create a file named omg.yml and copy/paste the content from this link.

  3. Launch the Multipass VM [text placeholder]

    • For Linux, run: 
      multipass launch 25.04 -c 2 -m 2G -d 20G -n box --cloud-init omg.yml
    • For Windows/macOS, use: 
      multipass launch -c 2 -m 2G -d 20G -n box --cloud-init omg.yml

    🍒 Tip for Windows Users: Keep the VM named box to avoid potential issues. Changing the hostname can break VM reboots (details here).

  4. Verify the VM's IP Address
    After a few minutes, check your VM's IP address by running:

multipass list

⚠️ Note: On Windows, avoid using the top-most IP address in the list after rebooting your VM—it may be stale.

Monitoring Installation

Once the Multipass VM is running, installation of IIAB Calibre-Web will begin automatically.

  1. Log into the VM in another terminal window: 
    multipass shell box
  2. Monitor installation progress: 
    sudo tail -f /var/log/cloud-init-output.log
    Installation typically takes 2 to 20 minutes. Look for the completion message:
    "INTERNET-IN-A-BOX (IIAB) SOFTWARE INSTALL IS COMPLETE."

🍒 Optional Debugging Commands
If you're curious about additional ways to monitor installation, try the following:

sudo -i  # Switch to root
pgrep iiab -a
tail -f /opt/iiab/iiab/iiab-install.log

Configuration Options

Enable Uploads

The Enable uploads checkbox should already be set, for the Admin account (e.g. teachers!) But feel free to also enable uploads/downloads on a per-user basis, e.g. if a student co-librarian is helping everyone manage the collection? (N.B. This checkbox might be renamed to Enable adding content in future.)

Upgrading IIAB Calibre-Web

If you already have a recent version of IIAB Calibre-Web, upgrading is simple:

  1. Update the software:

    sudo iiab-update -f

    If your VM is named box, you can automate this from your host machine:

    multipass exec box -- sudo iiab-update -f

  2. For comprehensive updates, including OS and Ansible:

    sudo iiab-update

    Again, this can be automated from your host machine using:

    multipass exec box sudo iiab-update && multipass restart box

6. Troubleshooting and Known Issues

If the Download to IIAB button (or similar features) fail for you, in a repeated way:

  • Please open your browser's console by pressing F12 (or CTRL+SHIFT+J, or ⌘+OPTION+J) and then click Console.

  • Please post screenshots of BOTH (1) what is not working, and (2) the console's error messages — into a New issue (click the green button) after you log into: github.com/iiab/calibre-web/issues

  • You can also inspect Network activity, in your browser console's Network tab:

    • Look for requests related to the download process.
    • Check for any requests marked as failed or with error status codes (4xx or 5xx).
    • Pay attention to the response headers and content for potential error messages.

Common Issues and Solutions

Known Issues and Workarounds

How to Check Logs

First, click the Tasks button (to the right of Download to IIAB) to see error messages from particular downloads that might have failed.

Also, look into these log files:

  • /var/log/calibre-web.log (tips below, if you want to change this)
  • /var/log/xklb.log
  • Run journalctl -u calibre-web
  • See also Calibre-Web's -o <logfile> command-line option, that's officially part of Calibre-Web 0.6.21+ since 2023-10-21
  • See also Calibre-Web's logging documention as mentioned within github.com/iiab/iiab/tree/master/roles/calibre-web#backend
    1. To change Calibre-Web logging, log in to Calibre-Web (e.g. http://box.lan/books) then click Admin (by Tasks button, on top) > Configuration / Edit Basic Configuration > Logfile Configuration
    2. Then change these 2 settings:
      1. Log Level: DEBUG
      2. Location and name of logfile: /var/log/calibre-web.log

IIAB Directories Used

  • /tmp/calibre_web/ e.g. for covers/thumbnails, and possibly also eBooks like test.pdf or test.epub etc, using hash-like filenames [e.g. random-sequence of characters] initially?
  • /library/downloads/calibre-web/ e.g. to download larger items like videos, before moving them into /library/calibre-web/*

FAQ

Can I transplant an old Calibre-Web and its content, to a new IIAB?

The safest approach is to transplant the older Calibre-Web and all of your content together.

WARNING: Do not upgrade Calibre-Web!

  1. Install the latest IIAB Calibre-Web onto your NEW IIAB. (Ironically, it won't actually be used. Most crucially, this installs xklb and yt-dlp.) Then:
    1. Run sudo systemctl stop calibre-web
    2. Run sudo mv /library/calibre-web /library/calibre-web.bkp
  2. On your OLD IIAB, run sudo systemctl stop calibre-web
    1. Copy the older /library/calibre-web (including all of its contents) to your NEW IIAB, ideally using rsync -av to carefully preserve all permissions etc.
  3. Reboot your NEW IIAB, and try it!
Add a custom footer
Add a custom sidebar

Clone this wiki locally