From bcdb28d6fd4912dc0776e32b22a358b442af1fb6 Mon Sep 17 00:00:00 2001 From: Kaito Sugimoto Date: Sat, 28 Oct 2023 00:03:19 +0900 Subject: [PATCH] tiny fix --- README_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README_en.md b/README_en.md index 2f006c1..bf3476b 100644 --- a/README_en.md +++ b/README_en.md @@ -44,7 +44,7 @@ Please point out any errors on the [issues page](https://github.com/llm-jp/aweso | | Architecture | Training Data | Developer | License | |:---|:---:|:---:|:---:|:---:| | [LLM-jp-13B](https://www.nii.ac.jp/en/news/release/2023/1020.html) | GPT
([1.3b-v1.0](https://huggingface.co/llm-jp/llm-jp-1.3b-v1.0), [**13b**-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-v1.0), [**13b**-instruct-full-jaster-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-full-jaster-v1.0), [**13b**-instruct-full-jaster-dolly-oasst-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0), [**13b**-instruct-full-dolly-oasst-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-full-dolly-oasst-v1.0), [**13b**-instruct-lora-jaster-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-lora-jaster-v1.0), [**13b**-instruct-lora-jaster-dolly-oasst-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-lora-jaster-dolly-oasst-v1.0), [**13b**-instruct-lora-dolly-oasst-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-lora-dolly-oasst-v1.0)) | Pre-training: [llm-jp-corpus](https://github.com/llm-jp/llm-jp-corpus) (Wikipedia, Japanese mC4, The Pile, Stack) (**300B** tokens)
Instruction Tuning (SFT or LoRA): jaster, Dolly Dataset, OASST1 | LLM-jp | Apache 2.0 | -| [Stockmark-13b](https://stockmark.co.jp/news/20231027) | GPT
([**13b**](https://huggingface.co/stockmark/stockmark-13b)) | Japanese Wikipedia、Japanese CC-100、Japanese mC4、Japanese CommonCrawl、Japanese Patent、Stockmark Web Corpus
(**220B** tokens) | Stockmark | MIT | +| [Stockmark-13b](https://stockmark.co.jp/news/20231027) | GPT
([**13b**](https://huggingface.co/stockmark/stockmark-13b)) | Japanese Wikipedia, Japanese CC-100, Japanese mC4, Japanese CommonCrawl, Japanese Patent, Stockmark Web Corpus
(**220B** tokens) | Stockmark | MIT | | [PLaMo-13B](https://www.preferred.jp/en/news/pr20230928/) | Llama[^1]
([**13b**](https://huggingface.co/pfnet/plamo-13b)) | C4, Project Gutenberg, RedPajama, Japanese Wikipedia, Japanese mC4
(**1.5T** tokens) | Preferred Networks | Apache 2.0 | | [Weblab-10B](https://www.t.u-tokyo.ac.jp/press/pr2023-08-18-001) | GPT-NeoX
([**10b**](https://huggingface.co/matsuo-lab/weblab-10b), [**10b**-instruction-sft](https://huggingface.co/matsuo-lab/weblab-10b-instruction-sft)) | Japanese mC4, The Pile
(**600B** tokens)
SFT: Alpaca, FLAN | University of Tokyo Matsuo Lab | CC BY‑NC 4.0 | | [Japanese StableLM Alpha](https://stability.ai/blog/stability-ai-new-jplm-japanese-language-model-stablelm) | GPT-NeoX
([base-alpha-**7b**](https://huggingface.co/stabilityai/japanese-stablelm-base-alpha-7b), [instruct-alpha-**7b**](https://huggingface.co/stabilityai/japanese-stablelm-instruct-alpha-7b), [instruct-alpha-**7b**-v2](https://huggingface.co/stabilityai/japanese-stablelm-instruct-alpha-7b-v2)) | Wikipedia, Japanese CC‑100, Japanese mC4, Japanese OSCAR, RedPajama, private datasets[^2]
(**750B** tokens)
SFT: Dolly, HH‑RLHF, wikinews, Alpaca (discarded in v2) | Stability AI | base: Apache 2.0
instruct (v1): [Research license](https://huggingface.co/stabilityai/japanese-stablelm-instruct-alpha-7b/tree/main)
instruct (v2): Apache 2.0 |