Efficient Finetuning of LLMs for Domain-Specific Code Generation

Abstract

This work investigates fine-tuning Large Language Models (LLMs) on domain-specific code generation, aiming to enhance their performance in test code generation. The work consists of both small size code dataset and large size code dataset. It explores fine-tuning techniques for different open source large language models (e.g., Llama, Phi). The primary focus of this work was to meticulously prepare the dataset for the task and to explore efficient fine-tuning techniques that would enable the adaptation of our large dataset while operating within the constraints of limited computational resources. This involved curating and structuring the dataset to ensure its relevance and effectiveness, as well as identifying optimization strategies to enhance the fine-tuning process. By carefully balancing model performance with resource efficiency, this work aimed to achieve optimal results despite computational limitations. The primary focus of this work was to explore efficient fine-tuning techniques. Finding the best model and best set of hyperparameters for the datasets we were using was the main objective. The project also focus on the impact of the size and complexity of the dataset in finetuning task. The project seeks to identify effective fine-tuning methods for both the cases. Ultimately, its goal is to deepen our understanding of fine-tuning LLMs and improve their applicability to specific tasks and domains. We also experiment with Retrieval-Augmented Generation (RAG) and Reinforcement Learning, to enhance model performance by improving response accuracy. By integrating all these techniques and methodologies, the project aims to contribute valuable insights into scalable and efficient model training, ultimately enhancing the robustness and practicality of LLMs for real-world applications.