Skip to content

[feature] Anaconda AI support#200

Merged
AlbertDeFusco merged 24 commits intomainfrom
groq-proxy
Sep 18, 2025
Merged

[feature] Anaconda AI support#200
AlbertDeFusco merged 24 commits intomainfrom
groq-proxy

Conversation

@AlbertDeFusco
Copy link
Copy Markdown
Contributor

@AlbertDeFusco AlbertDeFusco commented Aug 27, 2025

Here in this PR a small-ish refactor of AI Clients is provided to streamline discovering models and running the agentic response loop.

Since many LLM services provide OpenAI-like API endpoints the RuntOpenAIClient can now be used as a base class as-needed. Important factors are

  • setting custom provider name in subclasses
  • setting env var prefixes in subclasses
  • setting default configuration parameters, which can be overridden with a config dictionary

More could be done to have ollama and openai-like clients share a common ancestor for essential members like .discoverAiModels, .generateAgenticResponse and others that are essential to external operation, but I'll leave that for another PR.

Finally, the RUNT_API_KEY (for anaconda idp) can now be used to request chat/completions from an Anaconda endpoint. This means that user need not set any other AI keys or have ollama running to use the AI agent.

Screenshot 2025-09-16 at 22 51 52

@AlbertDeFusco AlbertDeFusco changed the title [feature] Groq API proxy [feature] Anaconda AI support Sep 17, 2025
@AlbertDeFusco AlbertDeFusco marked this pull request as ready for review September 17, 2025 02:46
@AlbertDeFusco
Copy link
Copy Markdown
Contributor Author

@rgbkrk , I've added reading the version from deno.json, but I'm concerned that it would only work when executing from a clone of the repo and not on the published package. Is it possible to include the deno.json in the published package? Or would there be a better approach.

5 | import pkgInfo from "../lib/deno.json" with { type: "json" };
  |                     ^^^^^^^^^^^^^^^^^^ the specifier
  | 
  = hint: replace the relative import with a bare specifier that imports from the other package by name and optionally an export (ex. "@runt/lib")

@AlbertDeFusco
Copy link
Copy Markdown
Contributor Author

AlbertDeFusco commented Sep 17, 2025

What do you think about adding a function here to write the version string to something like packages/lib/version.ts that gets updated when versions are bumped?

https://github.com/runtimed/runt/blob/87df1848f9f5c0afdacd8579e400e35b25db9a4e/scripts/bump-version.ts#L127

@AlbertDeFusco
Copy link
Copy Markdown
Contributor Author

I've added VERSION attribute to @runt/lib and the version.json is updated by the bumper script

Copy link
Copy Markdown
Contributor

@intentionally-left-nil intentionally-left-nil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@AlbertDeFusco AlbertDeFusco merged commit d3ad0f5 into main Sep 18, 2025
7 checks passed
@rgbkrk rgbkrk deleted the groq-proxy branch September 18, 2025 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants