feat(web): ask the experience survey after a delivered artifact, and add an "other" answer (#7117) (#7127)

* feat(web): add an "other" answer with a free-text field to the survey

The eight choices cover what users already tell us most often, but they cannot
cover what we have not heard yet — and the people whose problem is not on the
list are exactly the ones worth hearing from. Picking "Something else" opens a
text field instead of finishing the survey.

The cost is paid only by the people who need it: everyone whose answer is on
the list still finishes in one tap. This is the open-ended question we cut
earlier, put back behind a door rather than in everyone way.

Wire shape follows PostHog open-choice convention — the response recorded is
the text they typed. Picking "Something else" and typing nothing still reports
the choice itself, because "none of these fit" is an answer and dropping it
would turn those people into non-responders.

Skip disappears once the choice is made; it would only have been a second way
to submit.

* style(web): use a neutral focus border on the survey text field

The green ring competed with the brand fill on the selected chip directly
above it — two green things one under the other, neither reading as primary.
A neutral border marks focus without joining that fight, and the inset fill
still separates the field from the card.

* fix(web): keep the survey submit button on the footer right edge

Submit replaces Skip in the footer, but only Skip carried the margin-left:auto
that pushes it right — so the moment the text field opened, the primary action
collapsed back against the step counter and jumped from the right edge to the
left. Measured from the card edge, both now sit at the same 17px gap.

* feat(web): ask the experience survey after a delivered artifact, not an export

The post-export survey could only reach people who exported. Over 30 days
~13k users exported while ~37k produced an artifact, so roughly two thirds
of the people who got real work out of the product were never asked — and
the ones who tried it, got something, and left without exporting are the
ones worth hearing from most.

The card is now armed by a delivered design run (resolveDesignDeliveryOutcome
=== 'delivered'), from the second such delivery on. The first delivery
measures a first impression rather than an experience, and lands at the one
moment the user most wants to look at what they just got; waiting for the
second costs almost no reach because deliveries repeat (~13 per user per 30
days).

Because a delivery is usually mid-session rather than end-of-session, the
3s delay now bails out if the user starts writing their next prompt.
'beforeinput' rather than 'keydown' so IME composition counts as typing.
A dropped chance is taken by the next delivery, so the card lands after the
turn the user actually stopped on.

Export no longer arms anything; the three FileViewer call sites are gone.
Every survey event carries trigger: 'post_generation' so the two regimes
stay distinguishable in the events table.

* feat(web): arm the experience survey on the first delivered artifact

Drops the two-delivery threshold to one. Waiting for a second run bought a
more considered opinion at the cost of everyone who produces one artifact
and leaves — the users we understand least, and the ones this survey was
widened to reach in the first place.

Asking on the first delivery is affordable because the card is not an
interruption: the component still waits out the delay and still drops the
chance entirely if the user starts typing, so an early ask that lands badly
costs a dropped chance rather than a derailed session.

The delivery counter stays. It is the seam the policy is made of, and it is
what makes an unwritable store read as "not yet qualified" — a store that
cannot write cannot record a dismissal either, so without it the card would
come back after every run with no way to stop it.



* fix(web): retire the experience survey when it is shown, not when it is answered

One ask per user was the intent, but `retireSurvey()` only ran from `finish`
and `close`. Ignoring the card — the most common way to decline a prompt —
left the flag unset, so the card returned after the user's next artifact, and
the one after that. `exposedRef` did not cover this: it is a ref, so it resets
on the next page load.

Retiring on exposure makes the rule what it claims to be: a user sees this
card at most once, whatever they do with it. A chance dropped by the typing
bail-out is still retried, because a card the user never saw did not spend
their one ask.



* refactor(web): drop the typing bail-out from the experience survey

The card no longer cancels itself when the user starts typing during the
three-second delay. With one ask per user, a dropped chance is the whole ask
for anyone whose habit is to type straight into the next prompt — and that
habit belongs to the engaged users this survey most wants to hear from.

Removes the `beforeinput` listener and the IME reasoning that went with it.
The delay stays: it keeps the card from animating in on top of the artifact,
which was always its other job.



* fix(web): give the survey's free-text field an accessible name

Review finding from #7074, folded in with that branch. The textarea is
auto-focused the instant "Something else" is picked, so its accessible name
is the only thing telling a screen-reader user what the field is for — and it
had none. A placeholder is not a label and disappears on the first keystroke.

Reuses the placeholder string rather than adding a twentieth translation of
the same sentence, so the visible and announced names cannot drift.



* chore: ensure PR CI

---------





(cherry picked from commit 713caf0)

Co-authored-by: open-design-crew[bot] <299007234+open-design-crew[bot]@users.noreply.github.com>
Co-authored-by: elifive555555 <296440099+elifive555555@users.noreply.github.com>
Co-authored-by: Eli <qiongyu1999@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: open-design-crew[bot] <4197780+open-design-crew[bot]@users.noreply.github.com>

Author: 6 people

apps/web/src/analytics/events.ts

@@ -193,8 +193,10 @@ type Track = (
 import {
   EXPERIENCE_SURVEY_ID,
   EXPERIENCE_SURVEY_IMPROVEMENT_CHOICES,
+  EXPERIENCE_SURVEY_IMPROVEMENT_OTHER,
   EXPERIENCE_SURVEY_QUESTION_IDS,
   EXPERIENCE_SURVEY_QUESTION_TEXT,
+  EXPERIENCE_SURVEY_TRIGGER,
 } from './experience-survey-contract';
 
 function send<T extends object>(
@@ -1456,11 +1458,17 @@ export function trackWhatsNewPopupClick(
 // PostHog events rather than the v2 schema's own.
 
 export function trackExperienceSurveyShown(track: Track): void {
-  send(track, 'survey shown', { $survey_id: EXPERIENCE_SURVEY_ID });
+  send(track, 'survey shown', {
+    $survey_id: EXPERIENCE_SURVEY_ID,
+    trigger: EXPERIENCE_SURVEY_TRIGGER,
+  });
 }
 
 export function trackExperienceSurveyDismissed(track: Track): void {
-  send(track, 'survey dismissed', { $survey_id: EXPERIENCE_SURVEY_ID });
+  send(track, 'survey dismissed', {
+    $survey_id: EXPERIENCE_SURVEY_ID,
+    trigger: EXPERIENCE_SURVEY_TRIGGER,
+  });
 }
 
 /**
@@ -1470,7 +1478,7 @@ export function trackExperienceSurveyDismissed(track: Track): void {
  */
 export function trackExperienceSurveySent(
   track: Track,
-  answers: { recommendation: number; improvement?: number },
+  answers: { recommendation: number; improvement?: number; improvementOther?: string },
 ): void {
   const ids = EXPERIENCE_SURVEY_QUESTION_IDS;
   const text = EXPERIENCE_SURVEY_QUESTION_TEXT;
@@ -1483,13 +1491,23 @@ export function trackExperienceSurveySent(
   };
 
   add(ids.recommendation, text.recommendation, answers.recommendation);
-  if (typeof answers.improvement === 'number') {
+  if (typeof answers.improvementOther === 'string') {
+    // PostHog's open-choice convention: the response is what they typed. An
+    // empty field still reports the choice itself, so "none of these fit"
+    // survives instead of looking like the question was skipped.
+    add(
+      ids.improvement,
+      text.improvement,
+      answers.improvementOther.trim() || EXPERIENCE_SURVEY_IMPROVEMENT_OTHER,
+    );
+  } else if (typeof answers.improvement === 'number') {
     const choice = EXPERIENCE_SURVEY_IMPROVEMENT_CHOICES[answers.improvement];
     if (choice) add(ids.improvement, text.improvement, choice);
   }
 
   send(track, 'survey sent', {
     $survey_id: EXPERIENCE_SURVEY_ID,
+    trigger: EXPERIENCE_SURVEY_TRIGGER,
     $survey_questions: answered,
     ...responses,
   });

apps/web/src/analytics/experience-survey-contract.ts

@@ -17,6 +17,15 @@
 
 export const EXPERIENCE_SURVEY_ID = '01a00fd1-ed7e-0000-d38e-63bce21fb816';
 
+/**
+ * What armed the card. PostHog's survey analytics ignores extra properties, so
+ * this rides along purely for our own segmentation. It exists because the
+ * trigger moved from a successful export to a delivered artifact: without it,
+ * responses from the two regimes are indistinguishable in the events table and
+ * any before/after read of the score is guesswork.
+ */
+export const EXPERIENCE_SURVEY_TRIGGER = 'post_generation';
+
 export const EXPERIENCE_SURVEY_QUESTION_IDS = {
   recommendation: '146fefc0-9c11-4003-9869-1fd81be1650f',
   improvement: 'e487f41a-8111-4a87-8795-1358c9a11b55',
@@ -47,3 +56,13 @@ export const EXPERIENCE_SURVEY_IMPROVEMENT_CHOICES = [
   "Doesn't look good",
   'Breaks other things',
 ] as const;
+
+/**
+ * The escape hatch at the end of the choices. PostHog models this as the
+ * question's open choice: when a respondent picks it, the response recorded is
+ * the text they typed, not the word "Other". This constant is only what we
+ * report when they pick it and type nothing — "none of these fit" is itself an
+ * answer worth keeping, and dropping it would silently turn those people into
+ * non-responders.
+ */
+export const EXPERIENCE_SURVEY_IMPROVEMENT_OTHER = 'Other';

apps/web/src/components/ExperienceSurvey.module.css

@@ -262,13 +262,13 @@
   color: var(--text-faint, #bdbdbd);
 }
 
-/* Keep the inset fill on focus. Switching it to --bg made the field the same
-   white as the card behind it, so the input dissolved into the panel and only
-   a floating green outline was left. The ring carries the focus cue instead. */
+/* Keep the inset fill on focus — switching it to --bg made the field the same
+   white as the card behind it and the input dissolved into the panel. The
+   border alone carries the focus cue: neutral, because green here competed
+   with the brand fill on the selected chip right above it. */
 .textarea:focus {
   outline: none;
-  border-color: var(--brand-weak, #d0ffb5);
-  box-shadow: 0 0 0 3px var(--brand-soft, #ebffde);
+  border-color: var(--text-strong, #202020);
 }
 
 .foot {
@@ -312,6 +312,10 @@
 /* Sizing only — colour, hover and focus come from the shared Button primitive
    so this stays in step with every other primary action in the app. */
 .submit {
+  /* Same push as .skip. Submit replaces Skip in the footer, so without this it
+     collapsed back against the counter and the primary action jumped from the
+     right edge to the left the moment the field opened. */
+  margin-left: auto;
   font-size: var(--font-size-12, 12px);
   padding: 6px 14px;
   border-radius: 999px;

apps/web/src/components/ExperienceSurvey.tsx

@@ -1,6 +1,6 @@
-// Experience survey (NPS). Armed by a successful export, rendered globally
-// from App.tsx so it survives the project → home navigation, and retired
-// permanently the moment the user answers or closes it.
+// Experience survey (NPS). Armed by a delivered artifact — any design run that
+// actually produces one — rendered globally from App.tsx so it survives the
+// project → home navigation, and retired permanently the moment it is shown.
 //
 // Two questions. The score is the metric and costs one tap; the follow-up asks
 // what to fix first and can be skipped. Anything longer was cut deliberately —
@@ -9,12 +9,13 @@ import { useCallback, useEffect, useRef, useState } from 'react';
 import { createPortal } from 'react-dom';
 import { AnimatePresence, motion } from 'motion/react';
 import type { Variants } from 'motion/react';
+import { Button } from '@open-design/components';
 import { useT } from '../i18n';
 import styles from './ExperienceSurvey.module.css';
 import {
   SURVEY_DELAY_MS,
   isSurveyRetired,
-  onExportSucceeded,
+  onArtifactDelivered,
   retireSurvey,
 } from './experience-survey-trigger';
 
@@ -24,6 +25,11 @@ export interface ExperienceSurveyAnswers {
   recommendation: number;
   /** Index into the improvement options, in the order rendered. */
   improvement?: number;
+  /**
+   * Set when the respondent picked "Other". Empty string means they picked it
+   * and typed nothing, which still says none of the listed choices fit.
+   */
+  improvementOther?: string;
 }
 
 interface Props {
@@ -72,6 +78,15 @@ const IMPROVEMENT_KEYS = [
   'experienceSurvey.improvement.regression',
 ] as const;
 
+/**
+ * "Other" sits after the eight choices and behaves differently: instead of
+ * finishing the survey it opens a text field. The eight are what users already
+ * tell us most often, so most people never reach for this — but the ones who
+ * do are the ones whose problem we have not named yet, and they are worth the
+ * extra tap.
+ */
+const OTHER_INDEX = IMPROVEMENT_KEYS.length;
+
 type Step = 'recommendation' | 'improvement' | 'thanks';
 
 const STEP_ORDER: Step[] = ['recommendation', 'improvement'];
@@ -99,6 +114,7 @@ export function ExperienceSurvey({
   const [visible, setVisible] = useState(false);
   const [step, setStep] = useState<Step>('recommendation');
   const [picked, setPicked] = useState<number | null>(null);
+  const [otherText, setOtherText] = useState('');
   const answersRef = useRef<Partial<ExperienceSurveyAnswers>>({});
   const bodyRef = useRef<HTMLDivElement | null>(null);
   // Tallest step seen so far. The card is pinned to the bottom-right corner,
@@ -139,18 +155,28 @@ export function ExperienceSurvey({
     };
   }, []);
 
-  // Arm on export. The delay gives the user a beat to see their export land
-  // before anything else asks for attention.
+  // Arm on a delivered artifact. The delay gives the user a beat to look at
+  // what the run just produced before anything else asks for attention. Once
+  // armed, the card is shown — nothing the user does during the delay calls it
+  // off, because the survey is asked once per user and dropping a chance is
+  // how a user ends up never being asked at all.
   useEffect(() => {
     if (!metricsConsent) return;
     let armTimer: number | null = null;
     let modalWatcher: MutationObserver | null = null;
 
+    const clearArm = () => {
+      if (armTimer !== null) {
+        window.clearTimeout(armTimer);
+        armTimer = null;
+      }
+    };
+
     const reveal = () => {
       if (isSurveyRetired()) return;
       if (isModalOpen()) {
         // Stay armed and wait the dialog out rather than dropping the chance:
-        // the export already happened, and this is the only one we get.
+        // the artifact is already delivered and the dialog will close.
         modalWatcher?.disconnect();
         modalWatcher = new MutationObserver(() => {
           if (isModalOpen()) return;
@@ -167,24 +193,36 @@ export function ExperienceSurvey({
       setVisible(true);
     };
 
-    const unsubscribe = onExportSucceeded(() => {
+    const unsubscribe = onArtifactDelivered(() => {
       if (isSurveyRetired() || exposedRef.current || armTimer !== null) return;
+      // The delay is the only thing between the artifact landing and the card
+      // arriving. It is short on purpose: long enough that the two do not
+      // animate on top of each other, short enough that the card still reads
+      // as being about the run that just finished.
       armTimer = window.setTimeout(() => {
-        armTimer = null;
+        clearArm();
         reveal();
       }, SURVEY_DELAY_MS);
     });
 
     return () => {
       unsubscribe();
       modalWatcher?.disconnect();
-      if (armTimer !== null) window.clearTimeout(armTimer);
+      clearArm();
     };
   }, [metricsConsent]);
 
+  // Being shown is what spends the one ask this user gets. Retiring here
+  // rather than in `finish`/`close` is the difference between "asked once" and
+  // "asked until you engage": a user who reads the card and neither answers
+  // nor closes it — the most common way to respond to an unwanted prompt — was
+  // otherwise asked again after their next artifact, and again after the one
+  // after that. `exposedRef` alone could not prevent that; it is a ref, so it
+  // resets on the next page load.
   useEffect(() => {
     if (!visible || exposedRef.current) return;
     exposedRef.current = true;
+    retireSurvey();
     onExposure?.();
   }, [visible, onExposure]);
 
@@ -216,6 +254,17 @@ export function ExperienceSurvey({
     if (step !== 'thanks') onDismiss?.(answersRef.current);
   }, [onDismiss, step]);
 
+  /** Opens the free-text field rather than finishing the survey. */
+  const pickOther = useCallback(() => {
+    if (picked !== null) return;
+    setPicked(OTHER_INDEX);
+  }, [picked]);
+
+  const submitOther = useCallback(() => {
+    answersRef.current.improvementOther = otherText.trim();
+    finish();
+  }, [finish, otherText]);
+
   /** Lights the tapped choice, then advances — the tap needs a receipt. */
   const pick = useCallback(
     (value: number, apply: (value: number) => void, next: Step | 'finish') => {
@@ -303,12 +352,43 @@ export function ExperienceSurvey({
               {t(key)}
             </button>
           ))}
+          <button
+            type="button"
+            className={`${styles.option} ${picked === OTHER_INDEX ? styles.picked : ''}`}
+            onClick={pickOther}
+          >
+            {t('experienceSurvey.improvement.other')}
+          </button>
         </div>
+        {picked === OTHER_INDEX ? (
+          <textarea
+            className={styles.textarea}
+            value={otherText}
+            onChange={(event) => setOtherText(event.target.value)}
+            placeholder={t('experienceSurvey.otherPlaceholder')}
+            // The field is auto-focused, so its name is announced immediately
+            // and is the only thing telling a screen-reader user what to type.
+            // A placeholder cannot carry that: it is not a label, and it
+            // disappears on the first keystroke. Reusing the same string keeps
+            // the two in step without a 20-locale key for the same sentence.
+            aria-label={t('experienceSurvey.otherPlaceholder')}
+            autoFocus
+          />
+        ) : null}
         <div className={styles.foot}>
           {counter}
-          <button type="button" className={styles.skip} onClick={finish}>
-            {t('experienceSurvey.skip')}
-          </button>
+          {/* Once "Other" is picked the question is already answered, so Skip
+              would only be a second way to submit. Submit replaces it, and an
+              empty field still reports "none of these fit". */}
+          {picked === OTHER_INDEX ? (
+            <Button variant="primary" className={styles.submit} onClick={submitOther}>
+              {t('experienceSurvey.submit')}
+            </Button>
+          ) : (
+            <button type="button" className={styles.skip} onClick={finish}>
+              {t('experienceSurvey.skip')}
+            </button>
+          )}
         </div>
       </>
     );

apps/web/src/components/FileViewer.tsx

@@ -46,7 +46,6 @@ import {
   type PreviewTransportDocumentState,
   type PreviewTransportRecoverySignal,
 } from '../observability/iframe-error';
-import { notifyExportSucceeded } from './experience-survey-trigger';
 import {
   trackArtifactExportResult,
   trackArtifactDeployResult,
@@ -7527,7 +7526,6 @@ function HtmlViewer({
     const originPromise = resolveArtifactExportOrigin(context)
       .catch(() => unknownExportOrigin());
     const finish = async (result: 'success' | 'failed' | 'cancelled', errorCode?: string) => {
-      if (result === 'success') notifyExportSucceeded();
       const originProps = await originPromise;
       trackArtifactExportResult(
         analytics.track,
@@ -12939,7 +12937,6 @@ function HtmlViewer({
     const started = templateExportStartedRef.current || performance.now();
     const originPromise = templateExportOriginPromiseRef.current
       ?? resolveArtifactExportOrigin().catch(() => unknownExportOrigin());
-    if (result === 'success') notifyExportSucceeded();
     void originPromise.then((originProps) => {
       trackArtifactExportResult(
         analytics.track,
@@ -14121,7 +14118,6 @@ function HtmlViewer({
     const started = imageExportStartedRef.current || performance.now();
     const originPromise = imageExportOriginPromiseRef.current
       ?? resolveArtifactExportOrigin().catch(() => unknownExportOrigin());
-    if (result === 'success') notifyExportSucceeded();
     void originPromise.then((originProps) => {
       trackArtifactExportResult(
         analytics.track,

apps/web/src/components/ProjectView.tsx

@@ -137,6 +137,7 @@ import {
   resolveDesignDeliveryOutcome,
   type DesignDeliveryOutcome,
 } from '../runtime/design-delivery';
+import { notifyArtifactDelivered } from './experience-survey-trigger';
 import { RESUME_CONTINUE_PROMPT } from '../runtime/resume';
 import {
   amrBalanceGateScopeForWorkspaceContext,
@@ -7522,6 +7523,13 @@ export function ProjectView({
                 artifactPersistenceError,
               );
               latestAssistantMsg = finalized;
+              // Only the live completion path arms the experience survey. The
+              // reattach and artifact-recovery paths below also settle on
+              // `delivered`, but they do so while replaying a run that
+              // finished before this page load — "how was that?" about work
+              // the user cannot remember finishing is a worse question than
+              // one not asked.
+              if (deliveryOutcome === 'delivered') notifyArtifactDelivered();
               setMessages((curr) => {
                 const updated = curr.map((m) =>
                   m.id === assistantId