Fine-tuning large language models (LLMs) with backpropagation — even for a subset of parameters such as LoRA — can be much more memory-consuming than inference and is often deemed impractical for resource-constrained mobile devices. Alternative methods, such as zeroth-order optimization (ZO), can greatly reduce the memory footprint but come at the cost of significantly slower model convergence (10× to 100× more steps than backpropagation). We propose a memory-efficient implementation of backpropagation (MeBP) on mobile devices that provides better trade-off between memory usage and compute… Read More
Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices
Fine-tuning large language models (LLMs) with backpropagation — even for a subset of parameters such as LoRA — can be much more memory-consuming than inference and is often deemed impractical for resource-constrained mobile devices. Alternative methods, such as zeroth-order optimization (ZO), can greatly reduce the memory footprint but come at the cost of significantly slower model convergence (10× to 100× more steps than backpropagation). We propose a memory-efficient implementation of backpropagation (MeBP) on mobile devices that provides better trade-off between memory usage and compute…
Related Posts
Revolutionizing MLOps: Enhanced BigQuery ML UI for Seamless Model Creation and Management
Exciting news for BigQuery ML (BQML) users.
Build an Inference Cache to Save Costs in High-Traffic LLM Apps
Large language models (LLMs) are widely used in applications like chatbots, customer support, code assistants,…
The Model Selection Showdown: 6 Considerations for Choosing the Best Model
Selecting the right model is one of the most critical decisions in any machine learning…