Skip to content

Commit c4c82eb

Browse files
authored
Agent publishing to the whole organization, trust levels, usage statistics (v2.4.0) (#153)
* feat(agents): organization-wide publishing, trust levels and usage statistics Any user can now publish their agents to the whole organization via a switch in the agent's "Access to Agent" section (publishing supersedes Entra-group sharing, which is deactivated while published). - Trust classification: Verified / Community badges on agent cards. Legacy admin-published agents count as Verified (lazily materialized on save). Classification actions (verify/downgrade/unpublish) are restricted to members of the Entra ID group in AGENT_VERIFIER_GROUP_ID (Graph checkMemberGroups, fail-closed, re-checked server-side). - Per-agent usage statistics (chats, interactions, tokens) as atomic Cosmos counters: AGENT_STATS docs under a sentinel partition, updated via Patch API increments from CreatePersonaChat and persistThread (threads now carry personaId). No time series; single-partition reads. - Home page shows favorites only, with "Browse all agents"; the Agents page gains sorting (most used default / recently updated / newest / name) and a trust filter. - Fix: UpsertPersona no longer overwrites createdAt on every edit (root cause of "last edited agent always on top"); adds updatedAt. * feat(models): GPT-5.6 Sol/Terra/Luna, rolling 7-day budget window; release v2.4.0 - Add GPT-5.6 family (Sol flagship / Terra balanced / Luna fast); Sol becomes the default model, Luna replaces GPT-5.4 Mini as the cost-cap downgrade target. GPT-5.3 Chat removed. - Budget window semantics: GetWeeklyUsage now computes an exact rolling 7-calendar-day window (-6 with inclusive compare); usage menu shows "Last 7 Days (rolling)" instead of a fixed Friday reset; downgrade message wording matches. - Release v2.4.0: changelog entries dated, package version bumped.
1 parent f966b7b commit c4c82eb

39 files changed

Lines changed: 1725 additions & 262 deletions

src/.env.example

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,9 @@ AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME=text-embedding-3-large
1414

1515
# New v1 API model deployments for Responses API
1616
AZURE_OPENAI_API_GPT52_DEPLOYMENT_NAME=gpt-5.2
17-
AZURE_OPENAI_API_GPT53_CHAT_DEPLOYMENT_NAME=gpt-5.3-chat
17+
AZURE_OPENAI_API_GPT56_SOL_DEPLOYMENT_NAME=gpt-5.6-sol
18+
AZURE_OPENAI_API_GPT56_TERRA_DEPLOYMENT_NAME=gpt-5.6-terra
19+
AZURE_OPENAI_API_GPT56_LUNA_DEPLOYMENT_NAME=gpt-5.6-luna
1820
AZURE_OPENAI_API_GPT55_DEPLOYMENT_NAME=gpt-5.5
1921
AZURE_OPENAI_API_GPT54_DEPLOYMENT_NAME=gpt-5.4
2022
AZURE_OPENAI_API_GPT54_MINI_DEPLOYMENT_NAME=gpt-5.4-mini
@@ -27,7 +29,7 @@ AZURE_OPENAI_GPT_IMAGE_DEPLOYMENT_NAME=gpt-image-1
2729
# Foundry-hosted low-cost models (DeepSeek/Kimi) used as automatic downgrade
2830
# targets. OpenAI-compatible Chat Completions on the Foundry host's /openai/v1
2931
# surface; Bearer API-key auth. Leave unset to disable Foundry downgrade
30-
# targets (gpt-5.4-mini still works as a hardCapEligible Azure fallback).
32+
# targets (gpt-5.6-luna still works as a hardCapEligible Azure fallback).
3133
FOUNDRY_OPENAI_BASE_URL=https://<resource>.services.ai.azure.com/openai/v1
3234
FOUNDRY_API_KEY=
3335
FOUNDRY_DEEPSEEK_DEPLOYMENT_NAME=DeepSeek-V4-Pro
@@ -66,6 +68,11 @@ AZURE_OPENAI_VISION_API_VERSION=2023-12-01-preview
6668
# Update your admin email addresses - comma separated
6769
ADMIN_EMAIL_ADDRESS=you@email.com,you2@email.com
6870

71+
# Agent trust management: object ID of the Entra ID group whose members may
72+
# verify/downgrade/unpublish organization-published agents. May be a
73+
# security group. Leave empty to disable trust actions for everyone.
74+
AGENT_VERIFIER_GROUP_ID=
75+
6976
# Identity provider is optional if you are running in development mode locally (npm run dev)
7077
AUTH_GITHUB_ID=
7178
AUTH_GITHUB_SECRET=

src/__tests__/INVENTORY.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -318,7 +318,7 @@ This is the largest feature folder with complex state management and streaming.
318318
319319
**Models:**
320320
- `chat-services/models.ts`
321-
- `ChatModel` type (gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.3-chat)
321+
- `ChatModel` type (gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, gpt-5.4, gpt-5.4-mini)
322322
- `MODEL_CONFIGS` record with pricing, context window, reasoning support
323323
- `ChatThreadModel`, `ChatMessageModel`, `ChatDocumentModel`, `AttachedFileModel`
324324
- `UserPrompt`, `AzureChatCompletion` response types
@@ -911,7 +911,7 @@ This folder contains shadcn/ui and custom UI primitives. Most are pure presentat
911911
912912
**Completions API** (main LLM)
913913
- Used in: chat-api-response.ts, openai-responses-stream.ts
914-
- Models: gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.3-chat
914+
- Models: gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, gpt-5.4, gpt-5.4-mini
915915
- Test strategy: Mock streaming response
916916
917917
**Embeddings API**

src/__tests__/e2e-fakes/cosmos.ts

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,10 @@ class InMemoryContainer {
6060
this.docs[idx][key] = op.value;
6161
} else if (op.op === "remove") {
6262
delete this.docs[idx][key];
63+
} else if (op.op === "incr") {
64+
const current = this.docs[idx][key];
65+
this.docs[idx][key] =
66+
(typeof current === "number" ? current : 0) + (op.value ?? 0);
6367
}
6468
}
6569
return { resource: this.docs[idx] };

src/app/(authenticated)/api/chat/route.ts

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -472,6 +472,7 @@ export async function POST(req: Request) {
472472
modelConfig,
473473
fallbackInfo: fallbackInfo.fellBack ? fallbackInfo : undefined,
474474
reasoningDurationMs: computeReasoningDurationMs(),
475+
personaId: ctx.thread.isTemporary ? undefined : ctx.thread.personaId,
475476
});
476477
} catch (err) {
477478
logError("/api/chat onAbort persist failed", {
@@ -521,6 +522,7 @@ export async function POST(req: Request) {
521522
fallbackInfo: fallbackInfo.fellBack ? fallbackInfo : undefined,
522523
streamError: lastStreamError,
523524
reasoningDurationMs: computeReasoningDurationMs(),
525+
personaId: ctx.thread.isTemporary ? undefined : ctx.thread.personaId,
524526
});
525527
// Generate a thread title from the first user message. ctx.history
526528
// has just the user message we appended in loadThreadContext when
@@ -578,6 +580,9 @@ export async function POST(req: Request) {
578580
fallbackInfo: fallbackInfo.fellBack ? fallbackInfo : undefined,
579581
streamError: lastStreamError,
580582
reasoningDurationMs: computeReasoningDurationMs(),
583+
personaId: ctx.thread.isTemporary
584+
? undefined
585+
: ctx.thread.personaId,
581586
});
582587
} catch (retryErr) {
583588
logError("/api/chat persistAssistantFromFinishEvent retry failed", {

src/app/(authenticated)/chat/page.tsx

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,20 @@
11
import { userHashedId } from "@/features/auth-page/helpers";
22
import { ChatHome } from "@/features/chat-home-page/chat-home";
3+
import { GetAllAgentStats } from "@/features/common/services/agent-stats-service";
34
import { FindAllNewsArticles } from "@/features/common/services/news-service/news-service";
45
import { FindAllExtensionForCurrentUser } from "@/features/extensions-page/extension-services/extension-service";
56
import { GetUserFavoriteAgents } from "@/features/persona-page/persona-services/agent-favorite-service";
67
import { FindAllPersonaForCurrentUser } from "@/features/persona-page/persona-services/persona-service";
78
import { DisplayError } from "@/features/ui/error/display-error";
89

910
export default async function Home() {
10-
const [personaResponse, extensionResponse, newsResponse, favoriteAgentIds, currentUserId] = await Promise.all([
11+
const [personaResponse, extensionResponse, newsResponse, favoriteAgentIds, currentUserId, agentStats] = await Promise.all([
1112
FindAllPersonaForCurrentUser(),
1213
FindAllExtensionForCurrentUser(),
1314
FindAllNewsArticles(),
1415
GetUserFavoriteAgents(),
1516
userHashedId(),
17+
GetAllAgentStats(),
1618
]);
1719

1820
if (personaResponse.status !== "OK") {
@@ -34,6 +36,7 @@ export default async function Home() {
3436
news={newsResponse.response}
3537
favoriteAgentIds={favoriteAgentIds}
3638
currentUserId={currentUserId}
39+
agentStats={agentStats}
3740
/>
3841
);
3942
}
Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,28 @@
11
import { userHashedId } from "@/features/auth-page/helpers";
2+
import { GetAllAgentStats } from "@/features/common/services/agent-stats-service";
23
import { FindAllExtensionForCurrentUser } from "@/features/extensions-page/extension-services/extension-service";
34
import { ChatPersonaPage } from "@/features/persona-page/persona-page";
45
import { GetUserFavoriteAgents } from "@/features/persona-page/persona-services/agent-favorite-service";
6+
import { IsAgentVerifier } from "@/features/persona-page/persona-services/agent-trust-service";
57
import { FindAllPersonaForCurrentUser } from "@/features/persona-page/persona-services/persona-service";
68
import { DisplayError } from "@/features/ui/error/display-error";
79

810
export default async function Home() {
9-
const [personasResponse, extensionsResponse, favoriteIds, currentUserId] =
10-
await Promise.all([
11-
FindAllPersonaForCurrentUser(),
12-
FindAllExtensionForCurrentUser(),
13-
GetUserFavoriteAgents(),
14-
userHashedId(),
15-
]);
11+
const [
12+
personasResponse,
13+
extensionsResponse,
14+
favoriteIds,
15+
currentUserId,
16+
agentStats,
17+
isVerifier,
18+
] = await Promise.all([
19+
FindAllPersonaForCurrentUser(),
20+
FindAllExtensionForCurrentUser(),
21+
GetUserFavoriteAgents(),
22+
userHashedId(),
23+
GetAllAgentStats(),
24+
IsAgentVerifier(),
25+
]);
1626
if (personasResponse.status !== "OK") {
1727
return <DisplayError errors={personasResponse.errors} />;
1828
}
@@ -25,6 +35,8 @@ export default async function Home() {
2535
extensions={extensionsResponse.response}
2636
initialFavoriteIds={favoriteIds}
2737
currentUserId={currentUserId}
38+
agentStats={agentStats}
39+
isVerifier={isVerifier}
2840
/>
2941
);
3042
}

src/features/chat-home-page/chat-home.tsx

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
"use client";
22

3+
import { AgentStatsModel } from "@/features/common/services/agent-stats-models";
34
import { ExtensionModel } from "@/features/extensions-page/extension-services/models";
45
import { AgentList } from "@/features/persona-page/agent-list";
56
import { AddNewPersona } from "@/features/persona-page/add-new-persona";
@@ -21,6 +22,7 @@ interface ChatPersonaProps {
2122
news: NewsArticleModel[];
2223
favoriteAgentIds: string[];
2324
currentUserId: string;
25+
agentStats?: Record<string, AgentStatsModel>;
2426
}
2527

2628
const FeedbackButton = ({ feedBackLink }: { feedBackLink: string }) => (
@@ -100,7 +102,7 @@ const ArticlesSection = ({
100102
</div>
101103
);
102104

103-
export const ChatHome: FC<ChatPersonaProps> = ({ personas, extensions, news, favoriteAgentIds, currentUserId }) => {
105+
export const ChatHome: FC<ChatPersonaProps> = ({ personas, extensions, news, favoriteAgentIds, currentUserId, agentStats }) => {
104106
const [showChangelog, setShowChangelog] = useState<boolean>(false);
105107

106108
return (
@@ -134,6 +136,8 @@ export const ChatHome: FC<ChatPersonaProps> = ({ personas, extensions, news, fav
134136
personas={personas}
135137
initialFavoriteIds={favoriteAgentIds}
136138
currentUserId={currentUserId}
139+
agentStats={agentStats}
140+
favoritesOnly
137141
/>
138142
</>
139143
)}

src/features/chat-page/__tests__/chat-store-factory.test.ts

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -68,12 +68,12 @@ describe("createChatStore — isolation", () => {
6868
storeA.subscribe(listenerA);
6969
storeB.subscribe(listenerB);
7070

71-
storeA.getState().setSelectedModel("gpt-5.3-chat");
71+
storeA.getState().setSelectedModel("gpt-5.4");
7272

7373
expect(listenerA).toHaveBeenCalledOnce();
7474
expect(listenerB).not.toHaveBeenCalled();
75-
expect(storeA.getState().selectedModel).toBe("gpt-5.3-chat");
76-
expect(storeB.getState().selectedModel).not.toBe("gpt-5.3-chat");
75+
expect(storeA.getState().selectedModel).toBe("gpt-5.4");
76+
expect(storeB.getState().selectedModel).not.toBe("gpt-5.4");
7777
});
7878

7979
// Tool-call-history restoration was deleted with the dead Zustand subsystem

src/features/chat-page/chat-services/chat-api/model-selection.ts

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -149,7 +149,10 @@ export async function resolveModelAndLimits(
149149
reason: "cap",
150150
originalModel: selectedModel,
151151
fallbackModel: target,
152-
message: `${budget.window === "weekly" ? "Weekly" : "Daily"} cost budget reached. Using ${targetConfig.name} until it resets.`,
152+
message:
153+
budget.window === "weekly"
154+
? `Rolling 7-day cost budget reached. Using ${targetConfig.name} until your usage from the last 7 days drops below the cap.`
155+
: `Daily cost budget reached. Using ${targetConfig.name} until it resets.`,
153156
limitType: "cost",
154157
currentUsage: budget.currentUsd ?? 0,
155158
limit: budget.limitUsd ?? 0,

src/features/chat-page/chat-services/chat-api/persist-assistant.ts

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ import type {
2828
import { createIdGenerator } from "ai";
2929
import { userHashedId } from "@/features/auth-page/helpers";
3030
import { logError, logInfo, logWarn } from "@/features/common/services/logger";
31+
import { RecordAgentInteraction } from "@/features/common/services/agent-stats-service";
3132
import { IncrementUsage } from "@/features/common/services/usage-service";
3233
import {
3334
reportPromptTokens,
@@ -148,6 +149,8 @@ export interface PersistPayload {
148149
};
149150
/** Token usage from the top-level streamText finish callback */
150151
usage: UsagePayload;
152+
/** Agent (persona) the thread was started from — attributes per-agent stats. */
153+
personaId?: string;
151154
}
152155

153156
// ---------------------------------------------------------------------------
@@ -173,6 +176,7 @@ export async function persistThread({
173176
messages,
174177
modelConfig,
175178
usage,
179+
personaId,
176180
}: PersistPayload): Promise<void> {
177181
const userId = await userHashedId();
178182

@@ -336,6 +340,23 @@ export async function persistThread({
336340
})
337341
);
338342

343+
// Per-agent usage counters (atomic Cosmos Patch increments on the
344+
// agent-stats doc). Only threads started from an agent carry a personaId.
345+
// Note: aborted turns arrive here with zero usage and still count as one
346+
// interaction — accepted, the user did see a partial answer.
347+
if (personaId) {
348+
RecordAgentInteraction(personaId, {
349+
inputTokens,
350+
outputTokens,
351+
cachedTokens,
352+
}).catch((err: unknown) =>
353+
logError("Failed to record agent interaction", {
354+
personaId,
355+
error: err instanceof Error ? err.message : String(err),
356+
})
357+
);
358+
}
359+
339360
// Emit App Insights custom metrics (OpenTelemetry) consumed by the
340361
// "Bühler GPT Usage" monitoring workbook. These were dropped when the
341362
// legacy orchestrator stack was removed (commit 8f818d6); the streamText
@@ -447,6 +468,8 @@ export interface PersistAssistantFromFinishParams<TOOLS extends ToolSet> {
447468
* hidden behind boilerplate.
448469
*/
449470
streamError?: { message: string; name?: string };
471+
/** Agent (persona) the thread was started from — attributes per-agent stats. */
472+
personaId?: string;
450473
}
451474

452475
/**
@@ -463,6 +486,7 @@ export async function persistAssistantFromFinishEvent<TOOLS extends ToolSet>({
463486
messageId,
464487
streamError,
465488
reasoningDurationMs,
489+
personaId,
466490
}: PersistAssistantFromFinishParams<TOOLS>): Promise<void> {
467491
// Detect an empty finish — Azure content-filter trips, aborted streams
468492
// before any output, or a model error that resolves without text/tools.
@@ -582,5 +606,6 @@ export async function persistAssistantFromFinishEvent<TOOLS extends ToolSet>({
582606
outputTokens: usageDetails.outputTokens ?? 0,
583607
cachedTokens,
584608
},
609+
personaId,
585610
});
586611
}

0 commit comments

Comments
 (0)