Llm

This guide explains the steps to convert models published on HuggingFace to GGUF format. The GGUF format offers the possibility to run models efficiently on our computers. For this, we can use tools like Llama.cpp or Ollama. Download Llama.cpp To convert to GGUF format locally on our machine, we first need to download llama.cpp to be able to use its conversion tools. # Clone the llama.cpp repository git clone https://github.com/ggml-org/llama.cpp.git Prepare the environment We will use uv to prepare the environment and install necessary dependencies. If you haven’t installed it, you will need to install it. ...

Recently, the HiTZ center published the Latxa VL models. These models are based on Qwen3-VL models and possess vision capabilities. Currently, two sizes are available, both very small: models with 2B and 4B parameters. The tests presented below were performed with the 4B model. Models that know Basque Generally, models of such small size tend to only have the ability to answer decently in English. These Latxa VL models, however, have been trained to perform in Basque. ...

Orai has just released a language model called Kimu. Based on Gemma 2, they have created models with 2B and 9B parameters. They have successfully injected the knowledge required to speak and understand Basque into the base model. I have converted both Kimu models to the GGUF format and published them on Hugging Face. This makes it possible to use the Kimu model with applications like Llama.cpp or Ollama. ...

In recent months, Local LLMs have significantly improved, and some of them perform surprisingly well with Basque language. Among them, I want to highlight Tülu 3 70B, which shows good results in Basque when using the quantized version (q4_K_M). Until the Latxa instruction model becomes available, this is probably the best option for having conversations or generating text in Basque. ...