Convert Models to GGUF Format
This guide explains the steps to convert models published on HuggingFace to GGUF format. The GGUF format offers the possibility to run models efficiently on our computers. For this, we can use tools like Llama.cpp or Ollama. Download Llama.cpp To convert to GGUF format locally on our machine, we first need to download llama.cpp to be able to use its conversion tools. # Clone the llama.cpp repository git clone https://github.com/ggml-org/llama.cpp.git Prepare the environment We will use uv to prepare the environment and install necessary dependencies. If you haven’t installed it, you will need to install it. ...