Orchestration Origins

[Mookse]

Orchestration

@Acroop35 is working on creating an observer agent who will be able create a confidence matrix and margin tabulation between various available intelligences when spun up via callAvatar tool-call. callAvatar will be a generic MyLife bot function that bubbles up a request to the Member Avatar to orchestrate when the request is considered "out-of-bounds" for a bot or its team. This level of decision-making needs more refinement, but centers around instructions and metadata that identify in-scope and out-of-scope questions (ex.: Biographer in-scope: I want to talk about memories of my great-great-grandfather who fought in the civil war; out-of-scope: write me an essay about the civil war, or help me finish my resume.); out-of-scope requests trigger a callAvatar tool call with the original request and other metadata.
The avatar.mjs code would take the member request and route it to the Observer, passing relevant bot/agent-data from bot-agent.mjs. The Observer code would ingest this content, and return a confidence margin of relevant bots. If a high enough confidence margin, the Member Avatar would set the appropriate Team and Agent accordingly, otherwise create an interface input mechanic (clickable buttons) shipped to the frontend to ask the member to select from the congested ranking.

Note: @Mookse is currently thinking of a pattern: Hand-off, Pass, or Hail Mary; a hand-off is an intentional and innately aware call to another intelligence, either by survey functionality pattern outcome or direct request from a user (please send me to resume bot or name or team), Pass would be the observation pattern Alex is working on, and Hail Mary would be external agent or high-margin of error assessment of observation.

Observation Detailed Inputs

• Scope Definition: help discern differentiation amongst "similar" agents
  • positives: category of keys
  • negatives: out-of-scope
• Prompt Examples: Entities (single {string} or preferred few-shot {object}: { message, relevance, role, }; note: relevance is akin to scope, meaning we could place in negative examples of what the system does not entertain) that help isolate and create confidence in orchestration selection
• Skills: As in A2A, descriptions and other skill-aligned mechanics that can be reviewed by observer

To-Do

• Configure Defaults
• Test Harness
• Define Bot Data (not at the instance level, but at template level, with factory.mjs (BotFactory specifically) access to these novel properties (i.e., not defined directly in A2A skills, but ensure they are present)
  • skills: { description, examples, tags, }
  • buildKeywordRules: { negative, positive, }
• Establish pipeline in codebase
• Build observer.mjs
• Test several alternative models with current test dataset to understand strengths and weaknesses of lightweight models (speed, ranking, size, processing requirements)
• Determine if we can use few-shot example objects with a weighted directional relevance property in addition to role and message (open question, but given that we are just embedding, it might require that we create "x#" of embeddings and make a multilateral calculation on array outputs
  • Current solution/suggestion is to add keyword elements to both positive and negative connotations, so that system requires a low-margin on positive associations, and a high-margin on negative
• @Acroop35 to investigate transformer.js and see if the library provides methods similar to those used in their python test script
• Determine whether or not there are javascript-native comparable models = YES, there are many, search for transformers.js and possibly requiring feature extraction
  • Node Module: @huggingface/transformers
• Develop test.js version (stand-alone) recreating the python functionality
  • Use huggingface to find relevant models (feature extraction required?) for embedding information
  • recreate variant of numpy .dot() in js, either from functions available in transformers.js (@huggingface/transformers) or npm library or written independently