Skip to content

feat: import and export elements#4984

Draft
jabbadizzleCode wants to merge 14 commits into
v3from
import-export-elements
Draft

feat: import and export elements#4984
jabbadizzleCode wants to merge 14 commits into
v3from
import-export-elements

Conversation

@jabbadizzleCode

@jabbadizzleCode jabbadizzleCode commented Jan 12, 2026

Copy link
Copy Markdown
Collaborator

ClickUp Links

This was generated by AI during triage.

@coderabbitai

coderabbitai Bot commented Jan 12, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 67e35238-3f68-441e-adab-f5b34b37d519

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gitguardian

gitguardian Bot commented Jun 30, 2026

Copy link
Copy Markdown

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
26381906 Triggered Generic Password b035920 .agents/skills/agent-browser/templates/authenticated-session.sh View secret
1509424 Triggered Generic Password 220f6a6 .devcontainer/docker-compose.yml View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@greptile-apps

greptile-apps Bot commented Jun 30, 2026

Copy link
Copy Markdown

Confidence Score: 3/5

The core import/export logic is well-structured, but several defects flagged in the prior review round appear unaddressed — most critically the stale SAS URL cache in DownloadModal that permanently breaks repeated downloads after a network failure — and a few new quality concerns were identified in this pass.

The ownership checks on export operations are now properly implemented via exportPermissionFilter, the single-mutation import flow correctly places onClose() outside any loop, and the ZIP parser uses maxOutputLength to bound per-entry decompression. However, DownloadModal.tsx still caches the download package before createDownload succeeds and does not clear the cache on failure, leaving expired SAS URLs stuck in the ref until a full page reload. In addition, the entry→ref mapping in importElementPackageBuffer relies on PostgreSQL insertion-order IDs without an explicit guard, and skippedElements is an always-zero constant that pollutes metrics.

apps/frontend-manage/src/components/elements/manipulation/DownloadModal.tsx — stale SAS URL cache on failure; packages/graphql/src/services/elementImportExport.ts — entry-to-ref mapping by sort position and misleading skippedElements metric.

Important Files Changed

Filename Overview
packages/graphql/src/services/elementImportExport.ts Core import/export service — 2357 lines; implements rate-limiting, HMAC-signed import tokens, ownership-scoped package parsing, duplicate detection via fingerprints, and transactional element creation. Export uses exportPermissionFilter correctly; import validates blob ownership via assertUserPackageBlob and verifies SHA-256 hash before applying.
packages/graphql/src/lib/zip.ts Custom ZIP parser and writer. Validates paths (no traversal), checks entry count, accumulates uncompressed size from headers before decompressing, and uses inflateRawSync with maxOutputLength: uncompressedSize to cap per-entry decompression. CRC32 is verified post-decompress.
packages/graphql/src/services/packageStorage.ts Handles Azure Blob / local dev storage for import/export ZIP packages. Correctly scopes all blob paths under imports/{userId}/ or exports/{userId}/ and enforces this via assertUserPackageBlob. SAS tokens are short-lived (15 min). Local mode has no auth on the Express routes (intentional for dev/test only).
apps/frontend-manage/src/components/elements/manipulation/DownloadModal.tsx Download modal with SAS-URL caching via a parent-held ref. The cached entry is written before createDownload is awaited but not removed when it fails, meaning expired SAS URLs stay cached and cause repeated failures (flagged previously, still present).
apps/frontend-manage/src/components/elements/manipulation/UploadModal.tsx Previously flagged issues (wrong MIME type, contents[0] guard) appear resolved — the dropzone now accepts application/zip and application/x-zip-compressed, and the validation flow is ZIP-based rather than JSON-based.
apps/frontend-manage/src/components/elements/details/ImportedElementsOverviewTable.tsx Previously flagged onClose() inside a for-loop is resolved — the import now uses a single importElementPackage mutation rather than a per-element loop. onClose() is called once after the mutation succeeds.
packages/graphql/src/services/importExportFingerprints.ts Computes stable SHA-256 fingerprints for elements and answer collections using canonical JSON serialisation, enabling advisory duplicate detection across import/export. Lazy per-user backfill processes up to 500 items per validate call.
packages/graphql/src/services/mediaStorage.ts Handles staging and finalisation of imported media blobs in Azure, with deduplication via originalId. Cleanup of orphaned staged blobs is implemented. Stream reading is capped at MAX_IMPORT_EXPORT_MEDIA_BYTES.
apps/backend-docker/src/app.ts Adds unauthenticated /api/import-export-packages/:blobName PUT/GET routes activated only in local dev/test mode. Path traversal is guarded in getLocalPackagePath. Production uses signed Azure SAS URLs instead.
packages/prisma/src/prisma/schema/migrations/20260707120000_import_export_fingerprints/migration.sql Adds nullable importFingerprint TEXT columns to Element and AnswerCollection, with composite owner+fingerprint indexes for efficient duplicate lookup. No breaking schema changes.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant FE as Frontend
    participant GQL as GraphQL API
    participant Pkg as PackageStorage (Azure/Local)
    participant IE as elementImportExport
    participant DB as Database

    Note over FE,DB: Export Flow
    FE->>GQL: getElementExportPackagePreview(elementIds)
    GQL->>DB: findMany(elements + answerCollections, exportPermissionFilter)
    DB-->>GQL: elements + collections
    GQL-->>FE: preview (with warnings)
    FE->>GQL: getElementExportPackageLink(elementIds)
    GQL->>IE: createElementExportPackage
    IE->>DB: fetch elements + media
    IE->>IE: createZip(manifest + elements + media)
    IE->>Pkg: uploadPrivateImportExportBlob
    GQL-->>FE: downloadLink + filename + expiresAt

    Note over FE,DB: Import Flow
    FE->>GQL: prepareElementImportPackageUpload(filename)
    GQL->>Pkg: createUploadSasUrl imports userId uuid.zip
    GQL-->>FE: uploadSasURL + blobName
    FE->>Pkg: PUT file (direct to Azure/local)
    FE->>GQL: validateElementImportPackage(blobName)
    GQL->>Pkg: downloadElementImportPackage (assertUserPackageBlob)
    GQL->>IE: parseElementImportPackage (zip, schema, dep validation)
    GQL->>DB: backfillMissingImportFingerprintsForOwner
    GQL->>DB: findMany fingerprints for duplicate detection
    GQL-->>FE: importToken (HMAC) + preview elements + warnings
    FE->>GQL: importElementPackage(importToken, selectedRefs)
    GQL->>IE: verifyImportToken (HMAC + userId + expiry)
    GQL->>Pkg: downloadElementImportPackage
    GQL->>IE: "hashBuffer == token.sha256"
    GQL->>IE: stagePackageMediaFiles
    GQL->>DB: transaction create answerCollections + elements
    GQL-->>FE: importedElements count
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant FE as Frontend
    participant GQL as GraphQL API
    participant Pkg as PackageStorage (Azure/Local)
    participant IE as elementImportExport
    participant DB as Database

    Note over FE,DB: Export Flow
    FE->>GQL: getElementExportPackagePreview(elementIds)
    GQL->>DB: findMany(elements + answerCollections, exportPermissionFilter)
    DB-->>GQL: elements + collections
    GQL-->>FE: preview (with warnings)
    FE->>GQL: getElementExportPackageLink(elementIds)
    GQL->>IE: createElementExportPackage
    IE->>DB: fetch elements + media
    IE->>IE: createZip(manifest + elements + media)
    IE->>Pkg: uploadPrivateImportExportBlob
    GQL-->>FE: downloadLink + filename + expiresAt

    Note over FE,DB: Import Flow
    FE->>GQL: prepareElementImportPackageUpload(filename)
    GQL->>Pkg: createUploadSasUrl imports userId uuid.zip
    GQL-->>FE: uploadSasURL + blobName
    FE->>Pkg: PUT file (direct to Azure/local)
    FE->>GQL: validateElementImportPackage(blobName)
    GQL->>Pkg: downloadElementImportPackage (assertUserPackageBlob)
    GQL->>IE: parseElementImportPackage (zip, schema, dep validation)
    GQL->>DB: backfillMissingImportFingerprintsForOwner
    GQL->>DB: findMany fingerprints for duplicate detection
    GQL-->>FE: importToken (HMAC) + preview elements + warnings
    FE->>GQL: importElementPackage(importToken, selectedRefs)
    GQL->>IE: verifyImportToken (HMAC + userId + expiry)
    GQL->>Pkg: downloadElementImportPackage
    GQL->>IE: "hashBuffer == token.sha256"
    GQL->>IE: stagePackageMediaFiles
    GQL->>DB: transaction create answerCollections + elements
    GQL-->>FE: importedElements count
Loading

Reviews (7): Last reviewed commit: "Merge branch 'v3' of github.com:uzh-bf/k..." | Re-trigger Greptile

Comment thread packages/graphql/src/services/elements.ts Outdated
Comment thread packages/graphql/src/services/sharing.ts Outdated
Comment thread apps/frontend-manage/src/components/elements/manipulation/UploadModal.tsx Outdated
Comment thread packages/graphql/src/schema/query.ts
Comment thread packages/graphql/src/services/sharing.ts Outdated
Comment thread packages/graphql/src/lib/zip.ts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread apps/frontend-manage/src/components/elements/manipulation/DownloadModal.tsx Outdated
@greptile-apps

greptile-apps Bot commented Jul 14, 2026

Copy link
Copy Markdown

Too many files changed for review. (237 files found, 100 file limit)

Bypass the limit by tagging @greptile-apps to review.

) : error ? (
<UserNotification
type="error"
message={error || t('manage.elements.packagePreviewError')}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant