bart-pubhealth-expanded

This model is a fine-tuned version of facebook/bart-large on the clupubhealth dataset. It achieves the following results on the evaluation set:

• Loss: 2.3926
• Rouge1: 29.8528
• Rouge2: 10.8495
• Rougel: 23.3682
• Rougelsum: 23.7565
• Gen Len: 19.85

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.7469	0.26	500	2.0845	30.9611	10.7145	23.9719	24.1042	19.905
2.5524	0.51	1000	2.0628	32.0352	11.8898	24.9032	25.1368	19.895
2.429	0.77	1500	2.0787	32.2632	12.0353	25.1245	25.3728	19.895
2.2234	1.03	2000	2.1178	30.6437	11.5713	24.9071	25.1126	19.955
2.1249	1.29	2500	2.1183	31.6095	11.6573	25.0593	25.2063	19.87
2.0302	1.54	3000	2.1319	30.7417	11.4924	24.6388	24.8722	19.895
1.9761	1.8	3500	2.1850	31.6709	11.3036	24.4853	24.7571	19.87
1.8279	2.06	4000	2.2092	31.5778	11.59	24.7599	24.9956	19.825
1.8083	2.32	4500	2.1781	31.0441	10.7513	24.0656	24.3112	19.89
1.7527	2.57	5000	2.2155	31.1191	11.4519	24.4673	24.7157	19.81
1.723	2.83	5500	2.2024	31.9787	12.3158	24.9863	25.2597	19.94
1.5975	3.09	6000	2.2567	31.236	10.9733	24.1302	24.3433	19.9
1.5933	3.35	6500	2.2425	31.022	11.0249	24.1257	24.3555	19.92
1.5792	3.6	7000	2.2428	29.8844	10.3622	23.0802	23.4003	19.96
1.5718	3.86	7500	2.2367	31.2369	11.3854	24.8528	25.1287	19.815
1.4467	4.12	8000	2.2988	30.4903	10.4057	23.9914	24.239	19.715
1.4458	4.37	8500	2.2738	31.4345	11.2989	24.4239	24.6047	19.75
1.4342	4.63	9000	2.3092	28.8421	10.5744	23.0084	23.1741	19.855
1.4416	4.89	9500	2.2747	31.7111	11.5903	24.3422	24.6867	19.945
1.3437	5.15	10000	2.3203	31.11	11.0	24.6098	24.7362	19.81
1.3525	5.4	10500	2.3085	29.414	10.3412	23.3134	23.6552	19.935
1.3533	5.66	11000	2.3123	31.321	11.2686	23.9922	24.336	19.77
1.3248	5.92	11500	2.2916	30.8841	10.779	23.9407	24.0865	19.845
1.2617	6.18	12000	2.3530	29.7167	10.3162	23.4805	23.724	19.93
1.2846	6.43	12500	2.3712	28.3334	9.8425	22.1151	22.2951	19.92
1.2472	6.69	13000	2.3378	29.563	10.0033	23.1863	23.5065	19.865
1.2934	6.95	13500	2.3262	29.137	10.1232	22.9234	23.3799	19.855
1.2136	7.21	14000	2.3640	29.753	10.4865	23.4892	23.8778	19.885
1.2096	7.46	14500	2.3654	29.512	10.3891	23.0427	23.3684	19.88
1.211	7.72	15000	2.3491	30.9014	10.9117	24.127	24.3518	19.785
1.1954	7.98	15500	2.3626	29.0622	10.5405	22.7407	22.9454	19.84
1.1756	8.23	16000	2.3759	29.5277	10.2961	22.7888	23.1239	19.88
1.1516	8.49	16500	2.3772	29.3161	10.1894	23.0404	23.486	19.885
1.1604	8.75	17000	2.3710	29.6161	10.3543	22.8748	23.1849	19.905
1.1639	9.01	17500	2.3889	30.2817	10.8654	23.6438	23.8639	19.895
1.12	9.26	18000	2.3968	28.8747	9.8686	22.2775	22.6541	19.895
1.1136	9.52	18500	2.3950	30.1197	10.8992	23.2575	23.5732	19.86
1.1437	9.78	19000	2.3926	29.8528	10.8495	23.3682	23.7565	19.85

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.7.1
• Tokenizers 0.13.2