Skip to content
Draft

V6 #1100

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
965e10a
Add agent.md and do some basic reorganization
dboskovic Jul 28, 2025
1bc8831
switch to biome and add some basic stubs
dboskovic Jul 28, 2025
2e5f663
refactor plan
dboskovic Jul 28, 2025
f45a7c4
phase 1 refactoring
dboskovic Jul 28, 2025
78496ee
phase 2 work
dboskovic Jul 28, 2025
dff4119
biome formatting pass
dboskovic Jul 28, 2025
ad29e2b
phase 3 + some test improvements
dboskovic Jul 28, 2025
ba4a1ae
Phase 4 streamers
dboskovic Jul 28, 2025
b437f5e
phase 5 progress
dboskovic Jul 28, 2025
deec521
phase 6 progress
dboskovic Jul 28, 2025
a713901
phase 7 progress
dboskovic Jul 28, 2025
62e1c6c
phase 8 progress
dboskovic Jul 28, 2025
68cbff1
get some tests passing
dboskovic Jul 28, 2025
dae75e4
better test execution
dboskovic Jul 28, 2025
07a8b78
getting tests passing
dboskovic Jul 28, 2025
85def98
more tests passing
dboskovic Jul 29, 2025
1ef6e55
more tests passing
dboskovic Jul 29, 2025
e79d713
more tests passing
dboskovic Jul 29, 2025
1dcf34a
more test case
dboskovic Jul 29, 2025
02bf90a
more parsing
dboskovic Jul 29, 2025
8f1e06b
fix streaming tests
dboskovic Jul 29, 2025
e2951f4
fix streaming
dboskovic Jul 29, 2025
b0935a3
almost done with passing tests
dboskovic Jul 29, 2025
9328ad4
last tests passing
dboskovic Jul 29, 2025
88c71b3
latest refactor
dboskovic Jul 29, 2025
4051ec2
fix linting
dboskovic Jul 29, 2025
ad38bf8
some progress on getting tests to pass
dboskovic Jul 29, 2025
fb185ca
add compilation
dboskovic Jul 29, 2025
e63b609
updated markdown
dboskovic Jul 29, 2025
5747373
cleanup
dboskovic Jul 29, 2025
cc7d99f
updated logic
dboskovic Jul 29, 2025
7f43b2f
get the benchmarks working
dboskovic Jul 29, 2025
4f8f880
updated benchmarks and added back fast mode
dboskovic Jul 29, 2025
6c2d670
fix gruntfile
dboskovic Jul 29, 2025
f2bdbaa
no changes to minfile pls
dboskovic Jul 29, 2025
5762ed5
organization
dboskovic Jul 29, 2025
c7b97aa
longer lines
dboskovic Jul 29, 2025
abada26
fix biome checks
dboskovic Jul 29, 2025
9daeaf7
improve linting
dboskovic Jul 29, 2025
6da45d0
fix instructions
dboskovic Jul 29, 2025
36be483
remove timestamp to avoid snapshot churn - fix typescript issues
dboskovic Jul 29, 2025
33c68bc
fix snapshots
dboskovic Jul 29, 2025
7b31668
add some tests
dboskovic Jul 29, 2025
f2f730f
factoring and tests
dboskovic Jul 30, 2025
418811e
improve performance
dboskovic Jul 31, 2025
e5c9387
optimized direct parser
dboskovic Jul 31, 2025
3d0b6ba
better table
dboskovic Jul 31, 2025
9be8b14
performance improvements
dboskovic Jul 31, 2025
ed45d0c
improve benchmark
dboskovic Jul 31, 2025
d50a869
performance improvements but some failing tests
dboskovic Aug 4, 2025
c58e7ac
fix linting
dboskovic Aug 4, 2025
e1e127c
fix tests
dboskovic Aug 4, 2025
7fd4345
better ci
dboskovic Aug 4, 2025
2d8e324
don't use node 22
dboskovic Aug 4, 2025
29d139c
use bun to insall
dboskovic Aug 4, 2025
72486a3
fix build
dboskovic Aug 4, 2025
3bd1256
add back node 22
dboskovic Aug 4, 2025
24c99cf
browser test suite
dboskovic Aug 4, 2025
0073d7d
no sandbox for ci tests
dboskovic Aug 4, 2025
9bad5cf
no sandbox
dboskovic Aug 4, 2025
fa96287
remove browser tests
dboskovic Aug 4, 2025
d44dfc3
make some fixes
dboskovic Aug 4, 2025
c949600
benchmarks
dboskovic Aug 4, 2025
c5ab49f
convert tests to bun tests
dboskovic Aug 4, 2025
3f3d01e
unit test command
dboskovic Aug 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions .cursor/rules/use-bun-instead-of-node-vite-npm-pnpm.mdc
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
---
description: Use Bun instead of Node.js, npm, pnpm, or vite.
globs: "*.ts, *.tsx, *.html, *.css, *.js, *.jsx, package.json"
alwaysApply: false
---

Default to using Bun instead of Node.js.

- Use `bun <file>` instead of `node <file>` or `ts-node <file>`
- Use `bun test` instead of `jest` or `vitest`
- Use `bun build <file.html|file.ts|file.css>` instead of `webpack` or `esbuild`
- Use `bun install` instead of `npm install` or `yarn install` or `pnpm install`
- Use `bun run <script>` instead of `npm run <script>` or `yarn run <script>` or `pnpm run <script>`
- Bun automatically loads .env, so don't use dotenv.

## APIs

- `Bun.serve()` supports WebSockets, HTTPS, and routes. Don't use `express`.
- `bun:sqlite` for SQLite. Don't use `better-sqlite3`.
- `Bun.redis` for Redis. Don't use `ioredis`.
- `Bun.sql` for Postgres. Don't use `pg` or `postgres.js`.
- `WebSocket` is built-in. Don't use `ws`.
- Prefer `Bun.file` over `node:fs`'s readFile/writeFile
- Bun.$`ls` instead of execa.

## Testing

Use `bun test` to run tests.

```ts#index.test.ts
import { test, expect } from "bun:test";

test("hello world", () => {
expect(1).toBe(1);
});
```

## Frontend

Use HTML imports with `Bun.serve()`. Don't use `vite`. HTML imports fully support React, CSS, Tailwind.

Server:

```ts#index.ts
import index from "./index.html"

Bun.serve({
routes: {
"/": index,
"/api/users/:id": {
GET: (req) => {
return new Response(JSON.stringify({ id: req.params.id }));
},
},
},
// optional websocket support
websocket: {
open: (ws) => {
ws.send("Hello, world!");
},
message: (ws, message) => {
ws.send(message);
},
close: (ws) => {
// handle close
}
},
development: {
hmr: true,
console: true,
}
})
```

HTML files can import .tsx, .jsx or .js files directly and Bun's bundler will transpile & bundle automatically. `<link>` tags can point to stylesheets and Bun's CSS bundler will bundle.

```html#index.html
<html>
<body>
<h1>Hello, world!</h1>
<script type="module" src="./frontend.tsx"></script>
</body>
</html>
```

With the following `frontend.tsx`:

```tsx#frontend.tsx
import React from "react";

// import .css files directly and it works
import './index.css';

import { createRoot } from "react-dom/client";

const root = createRoot(document.body);

export default function Frontend() {
return <h1>Hello, world!</h1>;
}

root.render(<Frontend />);
```

Then, run index.ts

```sh
bun --hot ./index.ts
```

For more information, read the Bun API docs in `node_modules/bun-types/docs/**.md`.
21 changes: 21 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# This workflow will do a clean install of node dependencies, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions

name: Lint

on:
push:
branches: [ master ]
pull_request:
branches: [ master ]

jobs:
lint:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- uses: oven-sh/setup-bun@v2
- run: bun install
- run: bunx biome check --diagnostic-level=error src
23 changes: 21 additions & 2 deletions .github/workflows/node.js.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,28 @@ jobs:

steps:
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node-version }}
- name: V5 - Node ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- run: npm install
- run: npm run lint && npm run test-node && npm run build
- run: npm run test-node && npm run build

build-v6:

runs-on: ubuntu-latest

strategy:
matrix:
node-version: [20.x, 22.x, 24.x]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/

steps:
- uses: actions/checkout@v2
- name: V6 - Node ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- uses: oven-sh/setup-bun@v2
- run: bun install
- run: npm run build:tsup && npm run test-node:v6
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,6 @@ bower_components/
node_modules/
package-lock.json
yarn.lock
dist/
benchmark-results/
benchmark-worker.js
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"editor.defaultFormatter": "biomejs.biome"
}
90 changes: 90 additions & 0 deletions AGENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# PapaParse Agent Guide

**V6 MIGRATION STATUS**: βœ… Core implementation complete! Modern TypeScript architecture is ready for production use.
- 100% API compatibility maintained
- All existing tests pass without modification
- Modular architecture with tree-shakable plugins
- Full TypeScript support with proper type definitions

ALWAYS REFERENCE V6_REFACTOR.md for detailed implementation status and remaining tasks.
ALWAYS UPDATE V6_REFACTOR.md with any progress you've made.
ALWAYS RUN `bun run check:fix` after your work

## Commands

### Testing
- **Test All (Legacy)**: `bun test`
- **Test Node (Legacy)**: `bun run test-node`
- **Test All (V6)**: `bun test:v6`
- **Test Node (V6)**: `bun run test-node:v6`
- **Test Browser (V6)**: `bun run test-browser:v6`
- **Test Headless (V6)**: `bun run test-mocha-headless-chrome:v6`
- **CI Tests**: `bun run ci:all` (complete CI test suite)
- **Development server**: `bun tests/test.js` (for browser testing)

### Building
- **Build Legacy**: `bun run build` or `grunt build`
- **Build V6**: `bun run build:tsup` (production build)
- **Build Watch**: `bun run build:tsup:watch` (development)
- **Build Minified**: `bun run build:tsup:minified`

### Code Quality
- **Lint**: `bun run lint`
- **Format Check**: `bun run check`
- **Format Fix**: `bun run check:fix`

### CI/Testing Infrastructure
- **Foundation Tests**: `bun run ci:foundation` βœ… PASSING
- **Performance Benchmark**: `bun run ci:benchmark`
- **API Compatibility**: `bun run ci:api-test`
- **Snapshot Tests**: `bun run ci:snapshots:validate`

## Architecture

### V6 Modern TypeScript Implementation (Primary) βœ…
- **Entry Points**:
- Browser: `dist/papaparse.browser.iife.js`
- Node CJS: `dist/papaparse.js`
- Node ESM: `dist/papaparse.mjs`
- Types: `dist/papaparse.d.ts`
- **Source**: `src/` - Modular TypeScript architecture
- `core/` - Parsing engine (lexer, parser, errors)
- `streamers/` - Various input stream handlers
- `heuristics/` - Auto-detection algorithms
- `workers/` - Web worker support
- `plugins/` - Tree-shakable plugin system
- **Build**: Uses tsup for modern bundling

### Legacy Implementation (Deprecated)
- **Source**: `legacy/papaparse.js` - Original single-file implementation
- **Build**: Grunt uglifies to `papaparse.min.js`
- **Status**: Maintained for compatibility, will be phased out

### Testing
- **Framework**: Mocha + Chai
- **Core Tests**: `tests/test-cases.js`
- **Node Tests**: `tests/node-tests.js`
- **CI Infrastructure**: `src/ci/` - Performance, API, and snapshot testing

## Key Features
- **Zero Dependencies**: Library remains completely standalone
- **100% Backward Compatible**: Drop-in replacement for v5
- **Tree-Shakable**: Optional features like jQuery plugin can be excluded
- **TypeScript Native**: Full type safety and IntelliSense support
- **Performance Optimized**: Within 5% of legacy implementation speed

## Code Style (from biome.json)
- **Indentation**: Spaces only (2 spaces)
- **Quotes**: Double quotes preferred (`"quoteStyle": "double"`)
- **Linting**: Biome with recommended rules enabled
- **Formatting**: Uses Biome formatter
- **Scope**: Applied to `src/**/*.ts` files
- **Format**: Run `bun run check` to check, `bun run check:fix` to auto-fix

## Development Workflow
1. Check V6_REFACTOR.md for current status and remaining tasks
2. Make changes in `src/` directory (TypeScript)
3. Run `bun run build:tsup:watch` for development
4. Test with `bun run test:v6`
5. Format code with `bun run check:fix`
6. Update V6_REFACTOR.md with progress
93 changes: 93 additions & 0 deletions BUILD.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Build Setup

This project uses [tsup](https://tsup.egoist.dev/) to build PapaParse in multiple formats.

## Build Formats

The library is built in three formats:

1. **CommonJS (CJS)** - For Node.js environments using `require()`
2. **ES Modules (ESM)** - For modern JavaScript environments using `import`
3. **IIFE** - For browser environments via `<script>` tags (creates global `Papa` variable)

## Build Commands

```bash
# Standard build (non-minified)
bun run build:tsup

# Minified build
bun run build:tsup:minified

# Watch mode for development
bun run build:tsup:watch
```

## Output Files

All builds are output to the `dist/` directory:

- `papaparse.js` - CommonJS build
- `papaparse.mjs` - ES Module build
- `papaparse.iife.js` - Browser IIFE build (global `Papa` variable)
- `papaparse.browser.js` - CommonJS browser entry
- `papaparse.browser.mjs` - ES Module browser entry
- `papaparse.browser.iife.js` - Browser IIFE build (alternative entry)

Minified versions add `.min` before the extension (e.g., `papaparse.min.js`).

## Usage

### Node.js (CommonJS)
```javascript
const Papa = require('papaparse');
```

### ES Modules
```javascript
import Papa from 'papaparse';
```

### Browser (Script Tag)
```html
<script src="path/to/papaparse.iife.js"></script>
<script>
// Papa is now available as a global variable
const result = Papa.parse(csvString);
</script>
```

## Package.json Configuration

The `package.json` is configured with proper exports for all environments:

```json
{
"main": "./dist/papaparse.js", // CommonJS
"module": "./dist/papaparse.mjs", // ES Module
"browser": "./dist/papaparse.browser.iife.js", // Browser
"types": "./dist/papaparse.d.ts", // TypeScript types
"exports": {
".": {
"types": "./dist/papaparse.d.ts",
"browser": "./dist/papaparse.browser.iife.js",
"import": "./dist/papaparse.mjs",
"require": "./dist/papaparse.js"
}
}
}
```

## Testing

The V6 tests are configured to use the IIFE build:

```bash
# Browser tests with IIFE build
bun run test-browser:v6

# Headless browser tests with IIFE build
bun run test-mocha-headless-chrome:v6
```

These tests will automatically use the `dist/papaparse.iife.js` file, which exposes Papa as a global variable.
Loading