MathQA: Paper study & Fine-Tuning

Project dedicated to the study of the scientific paper "MathQA: Towards Interpretable MathWord Problem Solving with Operation-Based Formalisms", with the fine-tuning of Mistral 7B customized on the MathQA dataset.

The work required input/output prompt formatting, 4-bit quantization (NF4) to reduce memory usage, the use of the PEFT library with the LoRA method to update only some components of the model, training for over 3,700 epochs with regular checkpoints and saves, and testing followed by publication on the Hugging Face Hub.