Skip to content

feat: context-aware activity suggestions — AI-powered "what next?" in the Activity Palette #555

Description

@chelslava

Vision

When a user selects a node on the canvas, the Activity Palette shows a "Suggested Next" section at the top with 3–5 activities that are statistically or semantically likely to follow the selected activity in an RPA flow.

This is a modern RPA trend (seen in Power Automate Copilot, UiPath Autopilot) and aligns with the existing AI infrastructure in RPAForge.

Approach A — Heuristic (no LLM, good first version)

Analyze existing diagram templates (packages/studio/src/templates/) and committed user diagrams to build a co-occurrence frequency table:

P(next = "Click Element" | current = "Navigate") = 0.78
P(next = "Get Element Text" | current = "Click Element") = 0.45

Store the table as a static JSON asset, updated by a build-time script. No API calls, instant suggestions.

Approach B — LLM-powered (extends existing AI infrastructure)

Reuse the ai:generateDiagram IPC infrastructure to ask the LLM for continuations:

Given this partial RPA diagram ending at [activity], what are the 5 most likely next steps?

Cache responses per (current_activity, context_hash) to avoid redundant API calls.

UI Integration

Activity Palette
┌─────────────────────────┐
│ 🔮 Suggested            │
│  ▸ Click Element        │
│  ▸ Get Element Text     │
│  ▸ Take Screenshot      │
├─────────────────────────┤
│ All Activities          │
│  ...existing list...    │
└─────────────────────────┘

The suggestion strip appears only when a canvas node is selected; collapses otherwise.

Relevant Files

  • packages/studio/src/components/Designer/ActivityPalette.tsx
  • packages/studio/src/stores/ (may need a new suggestionStore)
  • packages/studio/src/templates/index.ts (source for co-occurrence mining)
  • packages/studio/electron/ai/ (for Approach B)

Acceptance Criteria

  • Suggestions appear within 100 ms of node selection (heuristic approach) or with loading indicator (LLM approach)
  • Suggestions are dismissible / collapsible per user preference
  • Works without an AI provider configured (Approach A as baseline)
  • Suggestion quality metric (click-through rate) can be measured via debug logs

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions