gpt-oss-20b gbs64 rcp logs (#869)

* add small llm pretraining

address review from previous PR

Credit co-authors for prior squash

Co-authored-by: ZixianWangAMD <zixiwang@amd.com>
Co-authored-by: Michal Marcinkiewicz <michalm@nvidia.com>
Co-authored-by: Lukasz Pierscieniewski <l.pierscieniewski@gmail.com>

* Update pretrain_llama31.py

disable async save and save intermediate checkpoint

* update README instruction, minor change to callback and set default target log perplexity to be 3.3 for consistency purposes

* set LR for GB32

* update README with download instructions from https://training.mlcommons-storage.org/index.html

* update link to  https://github.com/mlcommons/r2-downloader

* Revert "update link to  https://github.com/mlcommons/r2-downloader"

This reverts commit fed1bb4.

* initial commit for gpt-oss-20b

* update README

* update NV dockerfile and readme

* set random seed

* update license for amd + clean up rocm dockerfile

* revisit the target log perplexity after establishing rcp

* remove target  metric

* remove HF_TOKEN

* Update run_and_time.sh

* Create run.sub

* Update run_and_time.sh

* Update README.md

* Update run.sub

* Update run.sub

* Update run.sub

* Update Dockerfile.nvidia

* Update run.sub

* update configs to match 8b, addressed pr comments

* expose adam_eps with env var

* update more configs based on hf + nvidia configs

* remove yarn patch and add primus evaluator patch

* update megatron validation consumed samples

* remove utilities, not needed

* set master port to 29500, consistent with src/train.py

* remove hardcoded company name

* Revise evaluation metrics and training parameters in README

Updated README.md to clarify evaluation metrics and training parameters.

* update target metric and approximate runtime

* upload gbs16 logs

* rm gpt-oss-20b folder

* remove extra file

* update approximate runtime

* upload gbs32 rcp logs

* upload gbs64 rcp logs

---------

Co-authored-by: ZixianWangAMD <zixiwang@amd.com>
Co-authored-by: Michal Marcinkiewicz <michalm@nvidia.com>
Co-authored-by: Lukasz Pierscieniewski <l.pierscieniewski@gmail.com>
Co-authored-by: Michał Marcinkiewicz <43240942+mmarcinkiewicz@users.noreply.github.com>
Co-authored-by: Shriya Rishab <spalsamudram@nvidia.com>

Author: 5 people