Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
221 changes: 143 additions & 78 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,126 +1,191 @@
# Openwhisp
# OpenWhisp Enhanced

![Openwhisp](assets/cover.png)
![OpenWhisp](assets/cover.png)

Voice to text, entirely on your machine. Hold **Fn**, speak, release — your words are transcribed, polished, and pasted right where you need them. No cloud, no account, no latency.
**Free WisprFlow alternative.** Hold **Fn**, speak, release. Your words are transcribed via cloud AI, polished by a local LLM, and pasted right where you need them. German-optimized, works with any language.

Built in a weekend because I kept getting ads for Wispr Flow and thought — why not build it myself?
> Enhanced fork of [OpenWhisp](https://github.com/giusmarci/openwhisp) by [Raelume](https://raelume.ai). Adds cloud transcription, smaller/faster models, and multi-language support.

## How it works
## What's different from the original?

| Feature | Original OpenWhisp | Enhanced |
|---|---|---|
| **Transcription** | Local only (Whisper Base, 150 MB) | Cloud via Groq (Whisper Large v3) + local fallback |
| **Accuracy** | Basic | Significantly better (Large v3 vs Base) |
| **Text model** | gemma4:e4b (9.6 GB) | qwen3.5:2b (2.7 GB), 3.5x smaller |
| **Language** | English-focused | German-optimized (configurable to any language) |
| **Cloud cost** | None (all local) | Free (Groq Free Tier: 2 hours of audio per day) |
| **RAM usage** | ~12 GB (Whisper + Gemma 4) | ~3 GB (only Ollama LLM, transcription runs in the cloud) |
| **API key security** | N/A | Encrypted via macOS Keychain |
| **Offline mode** | Yes | Yes (automatic fallback to local Whisper) |

1. **Hold Fn** — OpenWhisp starts listening
2. **Speak** — your voice is captured locally
3. **Release Fn** — Whisper transcribes your speech, a local LLM polishes the text, and the result is pasted into whatever app you were using
## How it works

The entire pipeline runs locally via [Whisper](https://github.com/openai/whisper) (speech-to-text) and [Ollama](https://ollama.com) (text enhancement).
1. **Hold Fn** to start recording
2. **Speak** naturally
3. **Release Fn** and the pipeline kicks in:

## Features
```
Audio -> Groq Whisper Large v3 (cloud, free)
|
v
Raw text (in your language)
|
v
Ollama qwen3.5:2b (local, 2.7 GB)
|
v
Polished text -> Clipboard -> Auto-paste
```

- **Fully local** — no data leaves your Mac
- **Styles** — switch between Conversation and Vibe Coding modes depending on context
- **Enhancement levels** — from raw transcription (No Filter) to professional polish (High)
- **Intent resolution** — if you change your mind mid-sentence ("make it white... actually, black"), OpenWhisp resolves to your final intent
- **Auto-paste** — refined text is pasted directly into the active app
- **Auto-launch Ollama** — if Ollama is installed, OpenWhisp starts it automatically
- **Setup wizard** — guided first-launch experience for permissions, models, and configuration
- **Minimal overlay** — a small audio-reactive grid appears at the bottom of your screen during dictation
No internet? No problem. OpenWhisp automatically falls back to local Whisper.

## Styles
## Features

| Style | Use case |
|-------|----------|
| **Conversation** | Messages, emails, notes, everyday writing |
| **Vibe Coding** | Developer communication — translates casual speech into proper engineering language |
- **Cloud + Local Hybrid**: Groq for best accuracy, local Whisper as offline fallback
- **Tiny LLM**: qwen3.5:2b (2.7 GB) instead of 9.6 GB, runs on any Mac
- **3 Transcription Modes**: Auto (cloud + fallback), Cloud-only, Local-only
- **Styles**: Conversation and Vibe Coding modes
- **4 Enhancement Levels**: No Filter, Soft, Medium, High
- **Intent Resolution**: "Make it white... actually, black" resolves to final intent only
- **Auto-Paste**: Text is pasted directly into the active app
- **Encrypted API Key**: Stored via macOS Keychain, never in plaintext
- **Configurable Provider**: Groq, OpenAI, Lemonfox.ai, or any OpenAI-compatible provider
- **Language Selector**: German, English, French, Spanish, and 90+ more

Each style has four enhancement levels: **No Filter**, **Soft**, **Medium**, and **High**.
## Why Groq?

## Requirements
| Provider | Price/min | Model | Free Tier |
|---|---|---|---|
| **Groq** | $0.0002 | Whisper Large v3 | 7,200 sec/hr (~2 hrs of audio per day, free) |
| OpenAI | $0.006 | Whisper v2 | None |
| Lemonfox | $0.003 | Whisper Large v3 | 1 month free |

- macOS (Apple Silicon recommended)
- [Ollama](https://ollama.com/download/mac) — OpenWhisp auto-launches it if installed
- ~10 GB disk space for models (downloaded on first launch)
Groq is **30x cheaper than OpenAI** and offers a generous free tier. For normal usage (a few minutes of dictation per day), it is **completely free**.

## Getting started
## Quick Start

**1. Install Ollama and download the text model first:**
### 1. Install Ollama and pull the text model

```bash
# Install Ollama from https://ollama.com/download/mac, then:
# Install Ollama: https://ollama.com/download/mac
ollama serve

# In a new terminal, pull the text enhancement model (~9.6 GB)
ollama pull gemma4:e4b
# Pull the text enhancement model (only 2.7 GB!)
ollama pull qwen3.5:2b
```

**2. Clone and run Openwhisp:**
### 2. Get a Groq API key (free)

1. Go to [console.groq.com](https://console.groq.com)
2. Create an account (free)
3. Generate an API key

### 3. Clone and run

```bash
git clone https://github.com/giusmarci/openwhisp.git
cd openwhisp
git clone https://github.com/nicremo/openwhisp-enhanced.git
cd openwhisp-enhanced
npm install
npm run build:native
npm run dev
```

On first launch, the setup wizard will walk you through:
### 4. Setup Wizard

1. **Ollama** — verifies the connection. If Ollama is running, it connects automatically.
2. **Speech model** — downloads Whisper Base Multilingual (~150 MB) automatically.
3. **Text model** — detects the Gemma 4 model you already pulled.
4. **Permissions** — microphone access for recording, plus Accessibility and Input Monitoring for Fn key listening and auto-paste.
The setup wizard walks you through:

After setup, click into the text field where you want the text to go (an email, chat, code editor, etc.), then hold **Fn** and speak. When you release, the transcribed and enhanced text is automatically pasted into that field. If you move away or no text field is selected, the text is still copied to your clipboard — just use **Cmd+V** to paste it wherever you need.
1. **Transcription Engine**: Enter your Groq API key (or download local Whisper as fallback)
2. **Ollama**: Verify the connection
3. **Permissions**: Microphone, Accessibility, Input Monitoring

## Default models
After setup: hold **Fn**, speak, release. Done.

| Purpose | Model | Size |
|---------|-------|------|
| Speech-to-text | `onnx-community/whisper-base` | ~150 MB |
| Text enhancement | `gemma4:e4b` | ~9.6 GB |
## Changing the language

You can switch to any Ollama-compatible model from the Models page.
The app defaults to German. To switch to English (or any other language):

## Tech stack
1. Open the **Models** page
2. Change the **Language** dropdown to your language
3. Done. Both transcription and LLM rewrite will use your selected language.

- **Electron** + **React** + **TypeScript** — desktop shell and UI
- **@huggingface/transformers** — local Whisper inference
- **Ollama** — local LLM inference via API
- **Swift** — native macOS helper for Fn key listening, focus detection, and paste simulation
- **electron-vite** — build tooling
- **Hugeicons** — UI icons
Supported: German, English, French, Spanish, Italian, Portuguese, Dutch, Polish, Japanese, Chinese, Korean, and 90+ more via Whisper.

## Building for distribution
## Models

```bash
npm run package
```
| Purpose | Model | Size | Provider |
|---|---|---|---|
| Transcription (cloud) | Whisper Large v3 | Cloud | Groq (free) |
| Transcription (local) | whisper-base | ~150 MB | Local via HuggingFace |
| Text enhancement | qwen3.5:2b | ~2.7 GB | Local via Ollama |

Builds the Electron app, compiles the Swift helper, and packages everything into a `.dmg` and `.zip` in the `release/` directory.
### Alternative Cloud Providers

The app works with any OpenAI-compatible provider. Just change the Base URL and API key on the Models page:

| Provider | Base URL | Model |
|---|---|---|
| Groq (default) | `https://api.groq.com/openai` | `whisper-large-v3` |
| OpenAI | `https://api.openai.com` | `gpt-4o-mini-transcribe` |
| Lemonfox | `https://api.lemonfox.ai` | `whisper-1` |

## Project structure
### Alternative Text Models

Any Ollama model works. Recommendations by size:

| Model | Size | Quality | Speed |
|---|---|---|---|
| qwen3.5:2b (default) | 2.7 GB | Very good | Fast |
| qwen3:4b | 2.5 GB | Excellent | Fast |
| gemma3:4b | 3.3 GB | Excellent | Medium |
| qwen3.5:4b | 3.4 GB | Top tier | Medium |

## Tech Stack

- **Electron** + **React** + **TypeScript** for the desktop shell and UI
- **Groq API** (or any OpenAI-compatible provider) for cloud transcription
- **@huggingface/transformers** for local Whisper inference (offline fallback)
- **Ollama** for local LLM text enhancement
- **Swift** native macOS helper for Fn key listening, focus detection, and paste simulation
- **electron-vite** for build tooling
- **Electron safeStorage** for encrypted API key storage via macOS Keychain

## Project Structure

```
src/
main/ # Electron main process
dictation.ts # Transcription + rewrite pipeline
ollama.ts # Ollama API client + auto-launch
prompts.ts # Global rules + style + level prompt matrix
settings.ts # Settings persistence
windows.ts # Window creation and positioning
renderer/ # React UI
App.tsx # Sidebar layout, pages, setup wizard, overlay
styles.css # Complete styling
audio-recorder.ts # Web Audio recorder with level metering
preload/ # Electron preload bridge
shared/ # Shared types and constants
main/
api-key.ts # API key encryption (macOS Keychain)
cloud-transcription.ts # Cloud STT (Groq/OpenAI-compatible)
dictation.ts # Pipeline: transcribe -> rewrite -> paste
transcription.ts # Local Whisper inference (fallback)
ollama.ts # Ollama API client + auto-launch
prompts.ts # Prompt matrix (style x enhancement level)
settings.ts # Settings persistence
windows.ts # Window management
renderer/
App.tsx # UI: sidebar, pages, setup wizard, overlay
styles.css # Styling
audio-recorder.ts # Web Audio recorder with level metering
preload/ # Electron preload bridge
shared/ # Shared types and constants
swift/
OpenWhispHelper.swift # Native macOS helper
OpenWhispHelper.swift # Native macOS helper
```

## License
## Building for distribution

MIT
```bash
npm run package
```

Builds the Electron app, compiles the Swift helper, and packages everything into a `.dmg` and `.zip` in the `release/` directory.

---
## Credits

- Original [OpenWhisp](https://github.com/giusmarci/openwhisp) by [GiusMarci](https://x.com/GiusMarci) / [Raelume](https://raelume.ai)
- Enhanced version by [Fabian](https://github.com/nicremo)

## License

Made by [Raelume](https://raelume.ai)
MIT (same as the original)
14 changes: 14 additions & 0 deletions build/entitlements.mac.plist
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.cs.allow-jit</key>
<true/>
<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
<true/>
<key>com.apple.security.cs.disable-library-validation</key>
<true/>
<key>com.apple.security.device.audio-input</key>
<true/>
</dict>
</plist>
Binary file added build/icon.icns
Binary file not shown.
12 changes: 10 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@
{
"from": "build/native",
"to": "native"
},
{
"from": "build/icons",
"to": "icons"
}
],
"directories": {
Expand All @@ -35,6 +39,8 @@
"dmg",
"zip"
],
"entitlements": "build/entitlements.mac.plist",
"entitlementsInherit": "build/entitlements.mac.plist",
"extendInfo": {
"NSMicrophoneUsageDescription": "OpenWhisp needs microphone access to transcribe your speech locally."
}
Expand All @@ -44,15 +50,17 @@
"@hugeicons/core-free-icons": "^4.1.1",
"@hugeicons/react": "^1.1.6",
"@huggingface/transformers": "^4.0.1",
"react": "^19.2.5",
"react-dom": "^19.2.5"
},
"devDependencies": {
"@types/node": "^25.6.0",
"@types/react": "^19.2.14",
"@types/react-dom": "^19.2.3",
"@vitejs/plugin-react": "^5.2.0",
"electron": "^41.2.0",
"electron-builder": "^26.8.1",
"electron-vite": "^5.0.0",
"react": "^19.2.5",
"react-dom": "^19.2.5",
"typescript": "^6.0.2",
"vite": "^7.3.2"
}
Expand Down
40 changes: 40 additions & 0 deletions src/main/api-key.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import { safeStorage } from 'electron';

import type { AppSettings } from '../shared/types';

export function encryptApiKey(rawKey: string): string {
if (!safeStorage.isEncryptionAvailable()) {
throw new Error('Encryption is not available on this system. Cannot store API key securely.');
}

const encrypted = safeStorage.encryptString(rawKey);
return encrypted.toString('base64');
}

export function decryptApiKey(encrypted: string): string {
if (!encrypted) {
throw new Error('No encrypted API key to decrypt.');
}

if (!safeStorage.isEncryptionAvailable()) {
throw new Error('Encryption is not available on this system. Cannot read API key.');
}

return safeStorage.decryptString(Buffer.from(encrypted, 'base64'));
}

export function isApiKeySet(settings: AppSettings): boolean {
return settings.openaiApiKeyEncrypted.length > 0;
}

export function getApiKey(settings: AppSettings): string | null {
if (!isApiKeySet(settings)) {
return null;
}

try {
return decryptApiKey(settings.openaiApiKeyEncrypted);
} catch {
return null;
}
}
Loading