adalaw/Llama2-7B-GSM8K-MFT

Introduction

The model is trained with Masked thought Fine-Tuning (MFT), a simple variant of standard Supervised Fine-Tuning (SFT). You can refer to our code and paper below.

Links

• Code: https://github.com/ChangyuChen347/MaskedThought
• Paper: https://arxiv.org/abs/2403.02178

Results

We test it with the scripts provided in our code.

Model	GSM8K
adalaw/Llama2-7B-GSM8K-SFT	42.8
adalaw/Llama2-7B-GSM8K-MFT	47.3