Skip to content

KeyError: 'max_tokens' when running ontogpt list-models #543

Description

@jinzhang6688

When running ontogpt list-models, the CLI crashes with a KeyError: 'max_tokens' for certain providers (e.g., Amazon Bedrock, TwelveLabs embeddings). These models don’t expose a max_tokens field, but the CLI assumes it exists.


Steps to Reproduce

  1. Install OntoGPT (latest release).
  2. Run:
    ontogpt list-models
  3. Observe output: models are listed, then the command fails with:
    KeyError: 'max_tokens'
    

Traceback (excerpt)

File "ontogpt/cli.py", line 2228, in list_models
    max_tokens = models[model]["max_tokens"]
KeyError: 'max_tokens'

Expected Behavior
ontogpt list-models should list all available models without crashing, even if some providers don’t include max_tokens.


Current Code (before fix)

for model in models:
    primary_name = model
    provider = models[model]["litellm_provider"]

    if "mode" in models[model]:
        functionality = models[model]["mode"]
        if functionality not in ["chat", "completion", "embedding"]:
            continue
    else:
        continue

    max_tokens = models[model]["max_tokens"]

Proposed Fix
Use a safe lookup for max_tokens:

for model in models:
    model_info = models[model]
    primary_name = model
    provider = model_info.get("litellm_provider", "N/A")

    functionality = model_info.get("mode")
    if functionality not in ["chat", "completion", "embedding"]:
        continue

    max_tokens = model_info.get("max_tokens", "N/A")

Environment

  • Python 3.13
  • OntoGPT (latest pip release)
  • litellm (latest pip release)

Additional Notes
After applying the .get("max_tokens") patch locally, the command works correctly and lists all models. Suggest updating the CLI to handle missing fields gracefully.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions