lewtun HF staff commited on
Commit
ff6de3a
1 Parent(s): 728fcde

End of training

Browse files
README.md ADDED
@@ -0,0 +1,162 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ base_model: Qwen/Qwen2-0.5B
5
+ tags:
6
+ - trl
7
+ - reward-trainer
8
+ - generated_from_trainer
9
+ metrics:
10
+ - accuracy
11
+ model-index:
12
+ - name: reward-model
13
+ results: []
14
+ ---
15
+
16
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
+ should probably proofread and complete it, then remove this comment. -->
18
+
19
+ # reward-model
20
+
21
+ This model is a fine-tuned version of [Qwen/Qwen2-0.5B](https://huggingface.co/Qwen/Qwen2-0.5B) on an unknown dataset.
22
+ It achieves the following results on the evaluation set:
23
+ - Loss: 0.5182
24
+ - Accuracy: 0.73
25
+
26
+ ## Model description
27
+
28
+ More information needed
29
+
30
+ ## Intended uses & limitations
31
+
32
+ More information needed
33
+
34
+ ## Training and evaluation data
35
+
36
+ More information needed
37
+
38
+ ## Training procedure
39
+
40
+ ### Training hyperparameters
41
+
42
+ The following hyperparameters were used during training:
43
+ - learning_rate: 1e-05
44
+ - train_batch_size: 8
45
+ - eval_batch_size: 8
46
+ - seed: 42
47
+ - distributed_type: multi-GPU
48
+ - num_devices: 8
49
+ - total_train_batch_size: 64
50
+ - total_eval_batch_size: 64
51
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
52
+ - lr_scheduler_type: linear
53
+ - num_epochs: 1.0
54
+
55
+ ### Training results
56
+
57
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
58
+ |:-------------:|:------:|:----:|:---------------:|:--------:|
59
+ | 0.7794 | 0.0103 | 10 | 0.6609 | 0.603 |
60
+ | 0.6484 | 0.0206 | 20 | 0.6517 | 0.639 |
61
+ | 0.6565 | 0.0310 | 30 | 0.6356 | 0.626 |
62
+ | 0.6522 | 0.0413 | 40 | 0.6274 | 0.649 |
63
+ | 0.6354 | 0.0516 | 50 | 0.6114 | 0.665 |
64
+ | 0.6337 | 0.0619 | 60 | 0.6120 | 0.674 |
65
+ | 0.6133 | 0.0722 | 70 | 0.5871 | 0.677 |
66
+ | 0.6075 | 0.0826 | 80 | 0.5823 | 0.685 |
67
+ | 0.5795 | 0.0929 | 90 | 0.5812 | 0.686 |
68
+ | 0.585 | 0.1032 | 100 | 0.5759 | 0.691 |
69
+ | 0.5664 | 0.1135 | 110 | 0.5713 | 0.687 |
70
+ | 0.5772 | 0.1238 | 120 | 0.5619 | 0.692 |
71
+ | 0.5762 | 0.1342 | 130 | 0.5701 | 0.701 |
72
+ | 0.5726 | 0.1445 | 140 | 0.5550 | 0.707 |
73
+ | 0.5889 | 0.1548 | 150 | 0.5864 | 0.678 |
74
+ | 0.5758 | 0.1651 | 160 | 0.5623 | 0.702 |
75
+ | 0.5721 | 0.1754 | 170 | 0.5510 | 0.708 |
76
+ | 0.5595 | 0.1858 | 180 | 0.5458 | 0.707 |
77
+ | 0.5601 | 0.1961 | 190 | 0.5531 | 0.709 |
78
+ | 0.5646 | 0.2064 | 200 | 0.5473 | 0.702 |
79
+ | 0.5467 | 0.2167 | 210 | 0.5586 | 0.704 |
80
+ | 0.5558 | 0.2270 | 220 | 0.5539 | 0.7 |
81
+ | 0.5457 | 0.2374 | 230 | 0.5744 | 0.686 |
82
+ | 0.5848 | 0.2477 | 240 | 0.5571 | 0.712 |
83
+ | 0.5478 | 0.2580 | 250 | 0.5432 | 0.701 |
84
+ | 0.5461 | 0.2683 | 260 | 0.5447 | 0.708 |
85
+ | 0.5607 | 0.2786 | 270 | 0.5433 | 0.711 |
86
+ | 0.5886 | 0.2890 | 280 | 0.5378 | 0.716 |
87
+ | 0.5616 | 0.2993 | 290 | 0.5354 | 0.724 |
88
+ | 0.5426 | 0.3096 | 300 | 0.5327 | 0.722 |
89
+ | 0.5553 | 0.3199 | 310 | 0.5358 | 0.728 |
90
+ | 0.5335 | 0.3302 | 320 | 0.5259 | 0.732 |
91
+ | 0.5354 | 0.3406 | 330 | 0.5277 | 0.725 |
92
+ | 0.5437 | 0.3509 | 340 | 0.5344 | 0.727 |
93
+ | 0.5317 | 0.3612 | 350 | 0.5372 | 0.718 |
94
+ | 0.5609 | 0.3715 | 360 | 0.5422 | 0.715 |
95
+ | 0.5522 | 0.3818 | 370 | 0.5349 | 0.721 |
96
+ | 0.5714 | 0.3922 | 380 | 0.5329 | 0.728 |
97
+ | 0.5099 | 0.4025 | 390 | 0.5296 | 0.719 |
98
+ | 0.5148 | 0.4128 | 400 | 0.5337 | 0.721 |
99
+ | 0.5366 | 0.4231 | 410 | 0.5308 | 0.733 |
100
+ | 0.5432 | 0.4334 | 420 | 0.5270 | 0.733 |
101
+ | 0.5431 | 0.4438 | 430 | 0.5318 | 0.725 |
102
+ | 0.5531 | 0.4541 | 440 | 0.5364 | 0.728 |
103
+ | 0.5585 | 0.4644 | 450 | 0.5436 | 0.731 |
104
+ | 0.5275 | 0.4747 | 460 | 0.5348 | 0.721 |
105
+ | 0.553 | 0.4850 | 470 | 0.5315 | 0.728 |
106
+ | 0.5152 | 0.4954 | 480 | 0.5309 | 0.721 |
107
+ | 0.5405 | 0.5057 | 490 | 0.5332 | 0.721 |
108
+ | 0.5089 | 0.5160 | 500 | 0.5339 | 0.727 |
109
+ | 0.5334 | 0.5263 | 510 | 0.5321 | 0.725 |
110
+ | 0.535 | 0.5366 | 520 | 0.5342 | 0.719 |
111
+ | 0.5228 | 0.5470 | 530 | 0.5327 | 0.725 |
112
+ | 0.5384 | 0.5573 | 540 | 0.5316 | 0.713 |
113
+ | 0.5306 | 0.5676 | 550 | 0.5274 | 0.728 |
114
+ | 0.5183 | 0.5779 | 560 | 0.5230 | 0.724 |
115
+ | 0.5205 | 0.5882 | 570 | 0.5259 | 0.723 |
116
+ | 0.5152 | 0.5986 | 580 | 0.5282 | 0.718 |
117
+ | 0.5266 | 0.6089 | 590 | 0.5236 | 0.716 |
118
+ | 0.5141 | 0.6192 | 600 | 0.5223 | 0.721 |
119
+ | 0.5527 | 0.6295 | 610 | 0.5169 | 0.727 |
120
+ | 0.5037 | 0.6398 | 620 | 0.5172 | 0.733 |
121
+ | 0.5305 | 0.6502 | 630 | 0.5174 | 0.733 |
122
+ | 0.5277 | 0.6605 | 640 | 0.5217 | 0.724 |
123
+ | 0.5157 | 0.6708 | 650 | 0.5191 | 0.725 |
124
+ | 0.4997 | 0.6811 | 660 | 0.5216 | 0.723 |
125
+ | 0.5298 | 0.6914 | 670 | 0.5262 | 0.716 |
126
+ | 0.509 | 0.7018 | 680 | 0.5245 | 0.723 |
127
+ | 0.5216 | 0.7121 | 690 | 0.5199 | 0.727 |
128
+ | 0.5012 | 0.7224 | 700 | 0.5175 | 0.724 |
129
+ | 0.544 | 0.7327 | 710 | 0.5200 | 0.722 |
130
+ | 0.513 | 0.7430 | 720 | 0.5202 | 0.723 |
131
+ | 0.5153 | 0.7534 | 730 | 0.5220 | 0.721 |
132
+ | 0.5065 | 0.7637 | 740 | 0.5232 | 0.721 |
133
+ | 0.5291 | 0.7740 | 750 | 0.5230 | 0.721 |
134
+ | 0.5506 | 0.7843 | 760 | 0.5203 | 0.717 |
135
+ | 0.517 | 0.7946 | 770 | 0.5211 | 0.719 |
136
+ | 0.5102 | 0.8050 | 780 | 0.5212 | 0.718 |
137
+ | 0.4861 | 0.8153 | 790 | 0.5225 | 0.715 |
138
+ | 0.5407 | 0.8256 | 800 | 0.5236 | 0.718 |
139
+ | 0.5067 | 0.8359 | 810 | 0.5254 | 0.718 |
140
+ | 0.5207 | 0.8462 | 820 | 0.5269 | 0.718 |
141
+ | 0.5134 | 0.8566 | 830 | 0.5256 | 0.723 |
142
+ | 0.5116 | 0.8669 | 840 | 0.5241 | 0.724 |
143
+ | 0.5256 | 0.8772 | 850 | 0.5235 | 0.723 |
144
+ | 0.5233 | 0.8875 | 860 | 0.5222 | 0.727 |
145
+ | 0.4895 | 0.8978 | 870 | 0.5194 | 0.727 |
146
+ | 0.4877 | 0.9082 | 880 | 0.5185 | 0.725 |
147
+ | 0.5299 | 0.9185 | 890 | 0.5194 | 0.725 |
148
+ | 0.4815 | 0.9288 | 900 | 0.5196 | 0.725 |
149
+ | 0.477 | 0.9391 | 910 | 0.5204 | 0.728 |
150
+ | 0.5368 | 0.9494 | 920 | 0.5199 | 0.73 |
151
+ | 0.5133 | 0.9598 | 930 | 0.5184 | 0.733 |
152
+ | 0.4974 | 0.9701 | 940 | 0.5183 | 0.731 |
153
+ | 0.4948 | 0.9804 | 950 | 0.5183 | 0.728 |
154
+ | 0.5217 | 0.9907 | 960 | 0.5182 | 0.73 |
155
+
156
+
157
+ ### Framework versions
158
+
159
+ - Transformers 4.44.2
160
+ - Pytorch 2.4.0+cu121
161
+ - Datasets 2.21.0
162
+ - Tokenizers 0.19.1
added_tokens.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "<|endoftext|>": 151643,
3
+ "<|im_end|>": 151645,
4
+ "<|im_start|>": 151644
5
+ }
config.json ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "Qwen/Qwen2-0.5B",
3
+ "architectures": [
4
+ "Qwen2ForSequenceClassification"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 151643,
8
+ "eos_token_id": 151643,
9
+ "hidden_act": "silu",
10
+ "hidden_size": 896,
11
+ "id2label": {
12
+ "0": "LABEL_0"
13
+ },
14
+ "initializer_range": 0.02,
15
+ "intermediate_size": 4864,
16
+ "label2id": {
17
+ "LABEL_0": 0
18
+ },
19
+ "max_position_embeddings": 131072,
20
+ "max_window_layers": 24,
21
+ "model_type": "qwen2",
22
+ "num_attention_heads": 14,
23
+ "num_hidden_layers": 24,
24
+ "num_key_value_heads": 2,
25
+ "pad_token_id": 151643,
26
+ "rms_norm_eps": 1e-06,
27
+ "rope_theta": 1000000.0,
28
+ "sliding_window": null,
29
+ "tie_word_embeddings": true,
30
+ "torch_dtype": "float32",
31
+ "transformers_version": "4.44.2",
32
+ "use_cache": true,
33
+ "use_sliding_window": false,
34
+ "vocab_size": 151936
35
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d59b1043a328078221fdec071b7c48249b6d2577f39e3277398f29d4893d3ece
3
+ size 1976167144
special_tokens_map.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ "<|im_start|>",
4
+ "<|im_end|>"
5
+ ],
6
+ "eos_token": {
7
+ "content": "<|endoftext|>",
8
+ "lstrip": false,
9
+ "normalized": false,
10
+ "rstrip": false,
11
+ "single_word": false
12
+ },
13
+ "pad_token": {
14
+ "content": "<|endoftext|>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false
19
+ }
20
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "151643": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "151644": {
13
+ "content": "<|im_start|>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "151645": {
21
+ "content": "<|im_end|>",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ }
28
+ },
29
+ "additional_special_tokens": [
30
+ "<|im_start|>",
31
+ "<|im_end|>"
32
+ ],
33
+ "bos_token": null,
34
+ "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
35
+ "clean_up_tokenization_spaces": false,
36
+ "eos_token": "<|endoftext|>",
37
+ "errors": "replace",
38
+ "model_max_length": 32768,
39
+ "pad_token": "<|endoftext|>",
40
+ "split_special_tokens": false,
41
+ "tokenizer_class": "Qwen2Tokenizer",
42
+ "unk_token": null
43
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6666915e34b0d49c1e937d28940b9d8ebbbf3ffaf7ad86d949fca19c463a46cc
3
+ size 5240
vocab.json ADDED
The diff for this file is too large to render. See raw diff