Release the Model Weights of Unitok-base

[an287-git]

Dear authors,
Thank you for your excellent work on UniTok.
I was wondering if you could kindly release the pre-trained model weights for unitok-base.
I attempted to train it from scratch on my datasets, but the performance was suboptimal.
Having access to the original weights would be very helpful for further research.

[machuofan]

Hi there,

Thank you for your interest in our work. We are sorry that the UniTok-Base model was used solely for ablation studies in our work and was not pretrained on billion-scale data. For your case, you might consider distilling the UniTok-Large model into a base model using your own dataset.