Table of Links
Abstract and 1. Introduction
2 Related works
3 Methodology and 3.1 Causal language model as a classification model
3.2 Functional token
3.3 Dataset collection
3.4 Model development and training
4 Experiments and 4.1 Android function calls
4.2 Extension to Vehicle, Yelp, and DoorDash function sets
4.3 Full and partial training datasets and 4.4 Full training and LoRA training
4.5 Parallel and nested function call and 4.6 Weighted loss function for special tokens
5 Discussion and future works and References
Appendix
A.1 Android function examples
A.2 Vehicle function examples
Deployment of on-device language models Due to memory limitations and lower inference speeds, deploying larger models on edge devices like PCs or smartphones is challenging. Nonetheless, efforts to deploy smaller-scale Large Language Models (LLMs) to edge devices are underway. Open-source models of manageable sizes, such as Gemma-2B, Gemma-7B, StableCode-3B [31], and Llama-7B [47], have been introduced. To enhance these models’ inference speed on devices, research initiatives like Llama cpp [24] have been developed. The MLC LLM framework [46] allows the operation of 7B language models on mobile phones and other edge devices, demonstrating compatibility across various hardware, including AMD, NVIDIA, Apple, and Intel GPUs.
Function calling in language models Rapid advancements have been observed in the function-calling capabilities of smaller-scale models. Projects such as NexusRaven [42], Toolformer [37], ToolAlpaca [44], Gorrilla [30], ToolLlama [32] and Taskmatrix [20] have demonstrated that 7B and 13B models can call external APIs with efficacy comparable to GPT-4. The pioneering Octopus v1 project even enabled a 2B model to perform on par with GPT-4. This body of work utilizes a RAG-based method for function calling, where the model retrieves relevant functions from a large pool based on the user’s query, then generates a response using these functions as context.
Fine-tuning and adaptors of language models Fine-tuning language models has become a prevalent practice, with various efforts dedicated to this endeavor. LoRA [14] is often the method of choice for training models under GPU resource constraints. We use both full model training and LoRA training in our work, and compare their performance. A notable benefit of LoRA is its facilitation of extended functionalities in models, suggesting its potential to adapt our current framework for a broad range of applications.
Authors:
(1) Wei Chen, Stanford University, with equal contribution and a corresponding author {weichen6}@stanford.edu;
(2) Zhiyuan Li, Stanford University and a corresponding author {zhiyuan8}@stanford.edu.
