Skip to content

README.md

Latest commit

 

History

History
109 lines (85 loc) · 10.1 KB

README.md

File metadata and controls

109 lines (85 loc) · 10.1 KB
Raw

penguindex-go: Command-Line Uploader & Manager (Go Rewrite)

1. Overview

penguindex-go is a command-line application developed in Go, engineered for efficient uploading of files to Google Drive and management of existing Drive files through deletion. It is a rewrite of the original Rust project, rust_index_uploader (available at https://github.com/jendermine/rust-index-uploader).

A core design principle is security, achieved by utilizing an encrypted bundle for sensitive credentials such as the Google Service Account key and Telegram Bot Token. The tool offers a streamlined command-line interface (CLI) experience, incorporating features like upload progress indicators and Telegram notifications for operational status.

The application fetches its primary configuration—the encrypted Service Account key and Bot Token—from a secure, user-defined URL. Decryption of this bundle at runtime requires a user-provided PIN. Additionally, the Telegram Chat ID for notifications is fetched from a separate, distinct URL where it is stored in plaintext, facilitating on-the-fly modifications to the notification destination without the need to re-encrypt the primary secrets bundle.

2. Technical Features & Architecture

  • Secure Credential Handling & Decryption:

    • Retrieves an encrypted data bundle (JSON format, containing the Google Service Account JSON as a string and the Telegram Bot Token) from a user-configured URL specified by the EMBEDDED_BUNDLE_URL constant. This bundle is expected to be generated by a companion encrypt_util utility (which may also be a Go version or a compatible tool).
    • Mandates a user-provided PIN for the decryption of this bundle.
    • Employs PBKDF2-HMAC-SHA256 (Password-Based Key Derivation Function 2) with a high iteration count (consistent with the original encrypt_util) to derive a 256-bit decryption key from the user's PIN and the salt retrieved from the fetched bundle. This would be implemented using Go's golang.org/x/crypto/pbkdf2 package.
    • Utilizes AES-256-GCM (Advanced Encryption Standard in Galois/Counter Mode) for decrypting the bundle's ciphertext, likely using Go's crypto/aes and crypto/cipher packages. This AEAD (Authenticated Encryption with Associated Data) cipher ensures both confidentiality and integrity of the Service Account key and Bot Token.
    • Decrypted secrets (Service Account key, Bot Token) are strictly held in the application's memory and only for the duration of its operational lifecycle, minimizing exposure. They are not persisted to disk in unencrypted form by this tool.

  • Dynamic Telegram Chat ID Configuration:

    • Fetches the target Telegram Chat ID from a separate, user-configured URL (TELEGRAM_CHAT_ID_URL). The Chat ID at this URL is expected to be in plaintext.
    • This design allows administrators to change the Telegram notification recipient without needing to re-encrypt and re-upload the primary secrets bundle.

  • Google Drive API Integration (v3):

    • Authenticates with the Google Drive API using the decrypted Service Account credentials via Google's Go client libraries (e.g., google.golang.org/api/drive/v3 and golang.org/x/oauth2/google).
    • Upload Functionality:
      • Supports two modes of operation via the upload command:
        1. Upload to a default "test" folder, whose ID is hardcoded as DEFAULT_TEST_FOLDER_ID.
        2. Upload to a user-specified folder by providing its direct Google Drive Folder ID.
      • The tool does not perform searches for folders by name or undertake automatic folder creation. The responsibility lies with the user or administrator to ensure that the target folder ID is valid and that the service account possesses 'Editor' (or equivalent write) permissions on said folder.
      • Leverages Google Drive's resumable upload protocol for robust transfer of large files.
      • Features an interactive upload progress bar, displaying transfer speed and estimated time remaining (ETA), implemented using a Go library such as github.com/schollz/progressbar/v3 or similar. The progress bar would wrap the file reader to monitor byte transfer.
    • Delete Functionality:
      • Enables deletion of files from Google Drive using either the unique Google Drive File ID or a shareable Google Drive link. The tool includes regex-based parsing (using Go's regexp package) to attempt extraction of the File ID from common link formats.
      • API calls for deletion include SupportsAllDrives: true to ensure compatibility with files located in Shared Drives.

  • User Interface & Notifications:

    • Provides clear, text-based feedback in the command-line for all operations, indicating status (e.g., INFO:, OK:, ERROR:) and errors.
    • Upon successful file upload, generates and displays:
      • A direct Google Drive download link (https://drive.google.com/uc?id=...).
      • A custom Direct Download Link (DDL), the format of which is determined by an embedded base URL and the uploaded file's target folder (name or ID) and file name.
    • Sends a formatted notification message to the configured Telegram chat using the decrypted Bot Token and the dynamically fetched Chat ID. The message includes details such as file name, target folder, size, MIME type, and the aforementioned download links, using MarkdownV2 for formatting.

  • Auxiliary Features:

    • MIME Type Detection: Uses Go's standard library mime package or a third-party library to automatically determine the MIME type of the local file being uploaded.
    • URL Encoding: Employs Go's net/url package for constructing well-formed and safe DDLs, particularly for file and folder names that may contain special characters.
    • Argument Parsing: Uses a Go library like cobra, urfave/cli, or the standard library's flag package for robust command-line argument parsing and generation of help messages.

3. Command-Line Interface (CLI)

The tool utilizes a Go CLI library (e.g., cobra or flag) for parsing command-line arguments and generating help messages.

General Invocation Syntax:

./penguindex-go <COMMAND> [ARGUMENTS...]

(Replace ./penguindex-go with the actual path to your compiled binary if it's not in the current directory or your GOBIN.)

3.1. upload Command

Initiates the file upload process.

Syntax 1: Upload to Default "test" Folder

./penguindex-go upload <FILE_PATH>
<FILE_PATH>: (Required) The local path to the file intended for upload.

Action: The specified file is uploaded to the Google Drive folder identified by the DEFAULT_TEST_FOLDER_ID constant. The DDL and Telegram notification will use "test" as the folder identifier. Note: The associated service account must have 'Editor' (or equivalent write) permissions on the Google Drive folder corresponding to DEFAULT_TEST_FOLDER_ID.

Syntax 2: Upload to a Specific Folder by its ID

./penguindex-go upload <FILE_PATH> <OPTIONAL_FOLDER_ID>
<FILE_PATH>: (Required) The local path to the file intended for upload.
<OPTIONAL_FOLDER_ID>: (Optional) The direct Google Drive ID of the target destination folder.

Action: The specified file is uploaded to the Google Drive folder identified by <OPTIONAL_FOLDER_ID>. The DDL and Telegram notification will use this ID as the folder identifier. Note: Ensure the provided <OPTIONAL_FOLDER_ID> is a valid Google Drive Folder ID and that the service account possesses 'Editor' (or equivalent write) permissions for that specific folder. The tool does not perform validation of folder existence by name.

3.2. delete Command

Removes a specified file from Google Drive.

Syntax:

./penguindex-go delete <ID_OR_LINK>
<ID_OR_LINK>: (Required) Either the unique Google Drive File ID of the file to be deleted or a full shareable Google Drive link pointing to the file (e.g., https://drive.google.com/file/d/YOUR_FILE_ID/view).

4. Configuration (Embedded Constants)

The operational parameters of the tool are primarily defined by constants embedded within its source code. These must be correctly configured prior to compilation:

EMBEDDED_BUNDLE_URL: The raw HTTPS URL pointing to the encrypted_bundle.json file. This file, generated by the companion encrypt_util utility, contains the encrypted Google Service Account key (as a JSON string) and the encrypted Telegram Bot Token. TELEGRAM_CHAT_ID_URL: The raw HTTPS URL pointing to a plain text file containing solely the target Telegram Chat ID (e.g., -1001234567890). DEFAULT_TEST_FOLDER_ID: The Google Drive Folder ID that serves as the default upload destination when the upload command is invoked with only the <FILE_PATH> argument.

5. Compilation

The penguindex-go project is built using standard Go tooling. To produce an optimized release binary (e.g., stripping debug symbols):

go build -ldflags="-s -w" .

The resulting executable will typically be located in the current directory (e.g., penguindex-go or penguindex-go.exe on Windows) or in $GOPATH/bin or $GOBIN if installed globally. For comprehensive details on build optimization and cross-compilation, consult Go's official documentation.

6. Security Considerations

PIN Management: The security of the encrypted credentials hinges on the strength and confidentiality of the user-provided PIN. This PIN is used for key derivation via PBKDF2 and is not stored by the application. In-Memory Secret Handling: Decrypted sensitive data (Google Service Account key, Telegram Bot Token) is exclusively held within the application's memory during its runtime and is not persisted to disk. Secure Transport (HTTPS): All external network communications initiated by the tool—including interactions with the Google Drive API, fetching of the encrypted bundle and Chat ID from URLs, and sending Telegram notifications—are conducted over HTTPS, ensuring data in transit is encrypted. Permissions: It is critical that the Google Service Account used has the minimum necessary permissions on Google Drive. For uploading, 'Editor' rights on the target folder(s) are required. For deletion, appropriate permissions on the file are needed.

This tool is designed to offer a robust, secure, and efficient command-line solution.