included llama3.1 8b small llm training scripts

[ZixianWangAMD]

No description provided.

[github-actions]

MLCommons CLA bot:
Thank you very much for your submission, we really appreciate it. Before we can accept your contribution, we ask that you sign the MLCommons CLA (Apache 2). Please use this [Google form] (https://forms.gle/Ew1KkBVpyeJDuRw67) to initiate authorization. If you are from an MLCommons member organization, we will request that you be added to the CLA. If you are not from a member organization, we will email you a CLA to sign. For any questions, please contact support@mlcommons.org.
0 out of 1 committers have signed the MLCommons CLA.
❌ @zixian Wang
Zixian Wang seems not to be a GitHub user. You need a GitHub account after you become MLCommons member. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You can retrigger this bot by commenting recheck in this Pull Request}

[mmarcinkiewicz]

@ZixianWangAMD may I ask for a dockerfile that I can use to test it on H100? Or at least a hand how to modify the existing dockerfile?

[ShriyaRishab]

Based on Training WG feedback, can you please change the folder name from small_language_model_pretraining to small_llm_pretraining since that is the agreed upon long name for this benchmark?
The short name (which will be used everywhere in the code + logging) will be llama3.1_8b

[mmarcinkiewicz]

max_steps should be fixed at 1.2M
warmup_steps should be parametrizable from the config

[chenyuxyz]

will there be a new pre-tokenized dataset to download? this still points the dataset for 405B

[ShriyaRishab]

Yes, this is being uploaded after which we will modify the instructions.

[ShriyaRishab]

@ZixianWangAMD - can you please sign the CLA?

[ethanself]

@ZixianWangAMD - can you please sign the CLA?

Zixian has signed the CLA with user ZixianWangAMD, but the check is failing due to an incorrect Git Config. Commits were made locally with "Zixian Wang". GitHub usernames cannot have spaces.

I did notice that Zixian appears to have multiple GitHub accounts.

To set correct global configuration:
git config --global user.name "Your Correct Name Here"
git config --global user.email "your.email@example.com"

OR

Set local configuration for a specific repo:
git config user.name "Project Specific Name"
git config user.email "project.email@example.com"

Once config is fixed, a rebase of the local repo will need to be done to fix the associated author of commits.

[ShriyaRishab]

Close as duplicate of #814