1 min read

LazyVim/Neovim and CodeLlama how-to

Quick how-to on using huggingface/llm.nvim to use CodeLlama autocompletion inside Neovim.

Screen capture of CodeLlama autocompletion


Context: HuggingFace has revamped an older plugin to introduce https://github.com/huggingface/llm.nvim plugin for Neovim (see corresponding plugin for VSCode).

Why Neovim: I love VSCode and I use it when I'm focused on a specific project. However, when I'm jumping in and out of multiple code repositories and mostly searching/reading, I prefer the combination of Neovim + LazyVim + LazyGit.

Why LazyVim: I use LazyVim as the configuration for Neovim, since it works out-of-the-box and drastically reduces my yak shaving.

Why CodeLlama: Llama 2 has become the new defacto LLM to get started with. Given that CodeLlama is a specialized model of Llama2, I was eager to try it for autocompletion inside Neovim.

Why HuggingFace plugin: It is easy to get started with using HuggingFace servers for model inference, but you have the option to run your own server.


If you are using Neovim and not using LazyVim, jump to steps in https://github.com/huggingface/llm.nvim#package-manager instead.

Here is my quick how-to to set up llm.nvim with LazyVim:

Step 1 - Configure LazyVim

Create ~/.config/nvim/lua/plugins/llm.lua with the following code:

-- https://github.com/huggingface/llm.nvim#readme
return {
    opts = {
      -- https://github.com/huggingface/llm.nvim#models
      tokens_to_clear = { "<EOT>" },
      fim = {
        enabled = true,
        prefix = "<PRE> ",
        middle = " <MID>",
        suffix = " <SUF>",
      model = "codellama/CodeLlama-13b-hf",
      context_window = 4096,
      tokenizer = {
        repository = "codellama/CodeLlama-13b-hf",
      accept_keymap = "<c-g>",
      dismiss_keymap = "<c-x>",

Content of ~/.config/nvim/lua/plugins/llm.lua

Use ctrl-g to accept the completion and ctrl-x to dismiss the completion (because I could not get <tab> to be accepted by LazyVim, which is the suggestion in the llm.nvim documentation).

Step 2 - Login to HuggingFace

First, generate a new token at https://huggingface.co/settings/tokens (use the name huggingface/llm.nvim )

Second, login in the terminal:

# https://huggingface.co/docs/huggingface_hub/quick-start
pip install -U huggingface_hub

# https://huggingface.co/docs/huggingface_hub/quick-start#login
huggingface-cli login

Step 3 - Done!

Open Neovim, it's ready for llm.nvim & CodeLlama autocompletion.


Because of rate limiting and pricing options, decided to stick with GitHub Copilot for now instead, which LazyVim supports out of the box.