Project Team
Students
Aditya Kendre
Computer Science
Penn State Harrisburg
Faculty Mentors
Hien Nguyen
Penn State Harrisburg
Computer Science and Engineering
Suman Saha
Penn State University Park
Computer Science and Engineering
Project
https://sites.psu.edu/mcreu/files/formidable/2/2023-07-25/LLM-3.pdf
Project Video
Project Abstract
Large Language Models (LLMs) have brought significant advancements in Natural Language Processing (NLP), enabling breakthroughs in tasks like translation and text generation. However, their immense size and computational requirements pose challenges for deployment in resource-constrained environments. To address this, knowledge distillation has emerged as a technique to transfer knowledge from larger models to smaller ones. In this paper, we demonstrate the effectiveness of fine-tuning by training the Pythia-1B model on an instruction database. This allows the smaller model to benefit from the advanced language capabilities and reasoning abilities. Furthermore, we emphasize the deployability of the proposed model by choosing a well-calibrated smaller model, ensuring optimal computation-to-performance ratio, particularly for resource-constrained devices like smartphones. By deploying the LLM closer to the consumer, we observe decreased latency, enhanced privacy, and the ability to function effectively in offline or low-connectivity scenarios. This work contributes to making LLM technology more accessible and practical, with implications for improving user experience and privacy in real-world applications.
Evaluate this Project
Use this form link to provide feedback to the presenters, and add your project evaluation for award(s) consideration.