Is there any way to access o200k_base.tiktoken by local file?

[aaa646123]

Git provider (optional)

None

System Info (optional)

Gitlab

Issues details

My company network not allow access 'https://openaipublic.blob.core.windows.net/encodings/o200k_base.tiktoken'.

So it would be occurred error when I run pr-agent now.

I just download the o200k_base.tiktoken file in another way.

Is there any way to let pr-agent use the local o200k_base.tiktoken file instead openai url?

[jomach]

yes. I use this on my pipeline

export TIKTOKEN_CACHE_DIR=/tmp/tiktok_cache
mkdir -p $TIKTOKEN_CACHE_DIR
# hashlib.sha1('https://openaipublic.blob.core.windows.net/encodings/o200k_base.tiktoken'.encode()).hexdigest()
cp o200k_base.tiktoken $TIKTOKEN_CACHE_DIR/fb374d419588a4632f3f557e76b4b70aebbca790