high ram and cpu usage with tailwindcss

[ghost]

FAQ

• I have checked the FAQ and it didn't resolve my problem.

Announcement

• I have checked Breaking change announcement.

Minimal reproducible full config

call plug#begin()
Plug 'williamboman/nvim-lsp-installer'
Plug 'neovim/nvim-lspconfig'
Plug 'hrsh7th/cmp-nvim-lsp'
Plug 'hrsh7th/cmp-buffer'
Plug 'hrsh7th/cmp-path'
Plug 'hrsh7th/cmp-cmdline'
Plug 'hrsh7th/nvim-cmp'
Plug 'hrsh7th/cmp-vsnip'
Plug 'hrsh7th/vim-vsnip'
call plug#end()

local on_attach = function(client, bufnr)
  -- Enable completion triggered by <c-x><c-o>
  vim.api.nvim_buf_set_option(bufnr, 'omnifunc', 'v:lua.vim.lsp.omnifunc')

  -- Mappings.
  -- See `:help vim.lsp.*` for documentation on any of the below functions
  vim.api.nvim_buf_set_keymap(bufnr, 'n', 'gD', '<cmd>lua vim.lsp.buf.declaration()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', 'gd', '<cmd>lua vim.lsp.buf.definition()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', 'K', '<cmd>lua vim.lsp.buf.hover()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', 'gi', '<cmd>lua vim.lsp.buf.implementation()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<C-k>', '<cmd>lua vim.lsp.buf.signature_help()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<space>wa', '<cmd>lua vim.lsp.buf.add_workspace_folder()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<space>wr', '<cmd>lua vim.lsp.buf.remove_workspace_folder()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<space>wl', '<cmd>lua print(vim.inspect(vim.lsp.buf.list_workspace_folders()))<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<space>D', '<cmd>lua vim.lsp.buf.type_definition()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<space>rn', '<cmd>lua vim.lsp.buf.rename()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<space>ca', '<cmd>lua vim.lsp.buf.code_action()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', 'gr', '<cmd>lua vim.lsp.buf.references()<CR>', opts)
  vim.api.nvim_buf_set_keymap(bufnr, 'n', '<space>f', '<cmd>lua vim.lsp.buf.formatting()<CR>', opts)
end

local cmp = require'cmp'

cmp.setup({
    snippet = {
      -- REQUIRED - you must specify a snippet engine
      expand = function(args)
        vim.fn["vsnip#anonymous"](args.body) -- For `vsnip` users.
        -- require('luasnip').lsp_expand(args.body) -- For `luasnip` users.
        -- require('snippy').expand_snippet(args.body) -- For `snippy` users.
        -- vim.fn["UltiSnips#Anon"](args.body) -- For `ultisnips` users.
      end,
    },
    window = {
      -- completion = cmp.config.window.bordered(),
      -- documentation = cmp.config.window.bordered(),
    },
    mapping = cmp.mapping.preset.insert({
        ['<C-b>'] = cmp.mapping.scroll_docs(-4),
        ['<C-f>'] = cmp.mapping.scroll_docs(4),
        ['<C-Space>'] = cmp.mapping.complete(),
        ['<C-e>'] = cmp.mapping.abort(),
        ['<CR>'] = cmp.mapping.confirm({ select = true }), -- Accept currently selected item. Set `select` to `false` to only confirm explicitly selected items.
      }),
    sources = cmp.config.sources({
        { name = 'nvim_lsp' },
        { name = 'vsnip' }, -- For vsnip users.
        -- { name = 'luasnip' }, -- For luasnip users.
        -- { name = 'ultisnips' }, -- For ultisnips users.
        -- { name = 'snippy' }, -- For snippy users.
      }, {
        { name = 'buffer' },
      })
  })

-- Set configuration for specific filetype.
cmp.setup.filetype('gitcommit', {
    sources = cmp.config.sources({
        { name = 'cmp_git' }, -- You can specify the `cmp_git` source if you were installed it.
      }, {
        { name = 'buffer' },
      })
  })

-- Use buffer source for `/` (if you enabled `native_menu`, this won't work anymore).
cmp.setup.cmdline('/', {
    mapping = cmp.mapping.preset.cmdline(),
    sources = {
      { name = 'buffer' }
    }
  })

-- Use cmdline & path source for ':' (if you enabled `native_menu`, this won't work anymore).
cmp.setup.cmdline(':', {
    mapping = cmp.mapping.preset.cmdline(),
    sources = cmp.config.sources({
        { name = 'path' }
      }, {
        { name = 'cmdline' }
      })
  })


-- Setup lspconfig.
local capabilities = require('cmp_nvim_lsp').update_capabilities(vim.lsp.protocol.make_client_capabilities())

-- Use a loop to conveniently call 'setup' on multiple servers and
-- map buffer local keybindings when the language server attaches
local servers = {'tailwindcss'}
for _, lsp in pairs(servers) do
  require('lspconfig')[lsp].setup {
    capabilities = capabilities,
    on_attach = on_attach,
  }
end

Description

Neovim 0.7.0. Linux debian.
While using tailwindcss in class(className) - ram is out of control and cpu is high.

Steps to reproduce

Just install via LspInstall tailwindcss and start using it. In React or something.

Expected behavior

Low ram and cpu, without lags.

Actual behavior

Lags and ram usage big.

Additional context

No response

[Shougo]

It is not nvim-cmp issue.
It is just LSP server issue

[ghost]

It is not nvim-cmp issue. It is just LSP server issue

Thanks for the answer! I mean ram usage big not in LSP process, but in nvim itself, like 700 mb and higher with nvim-cmp. If i use it with coc-nvim - all is fine, memory is low. Maybe i something don't understand.

  "languageserver": {
    "tailwind-lsp": {
      "command": "tailwindcss-language-server",
      "args": ["--stdio"],
      "filetypes": ["javascriptreact", "typescriptreact"],
      "rootPatterns": ["tailwind.config.js", "tailwind.config.ts"],
      "settings": {
        "tailwindCSS": {
          "validate": true,
          "lint": {
            "cssConflict": "warning",
            "invalidApply": "error",
            "invalidScreen": "error",
            "invalidVariant": "error",
            "invalidConfigPath": "error",
            "invalidTailwindDirective": "error",
            "recommendedVariantOrder": "warning"
          },
          "classAttributes": ["class", "className", "classList", "ngClass"],
          "experimental": {}
        }
      }
    }
  },

[Shougo]

If so, it should be reported in nvim-lspconfig instead.
Because nvim-cmp just use neovim LSP interface.

[ghost]

I think there is some leak in nvim-cmp itself or cmp-nvim-lsp. Configuration of tailwindcss in nvim-lspconfig is correct. With coc-nvim tailwindcss works fine.

Ram usage 1.3g with tailwindcss - just an example:

Screencast.2022-06-26.09.22.07.mp4

[ghost]

If so, it should be reported in nvim-lspconfig instead. Because nvim-cmp just use neovim LSP interface.

Maybe some problems in LSP implementaion of neovim?

[Shougo]

coc-nvim does not use neovim LSP.
If it is nvim-cmp problem, you need to compare with other nvim-lsp plugins.

[siblanco]

I can also reproduce this issue and I've posted about it here aswell: neovim/neovim#19118 (comment)

When disabling nvim-cmp I have no issues at all.

Let me add: I've tested it with omnifunc (vim.api.nvim_buf_set_option(bufnr, 'omnifunc', 'v:lua.vim.lsp.omnifunc')) - no laggs. Memory below 250 mb.

[stoickeyboard]

@username664 Can you reopen this? Or did you find a solution? People still seem to be having a problem with this including myself.

[Shougo]

It is closed. Because it is not nvim-cmp problem.

[stoickeyboard]

@Shougo ah okay my apologies. I figured @siblanco's comment suggested different

[siblanco]

@Shougo how is this not a nvim-cmp problem? Please read this neovim/neovim#19118 (comment)

I can only reproduce it with nvim-cmp :(

[Shougo]

Please test other LSP completion plugins(It must use nvim-lsp).
If it is reproduced only nvim-cmp, it is nvim-cmp problem.

[Shougo]

Hm... I will try it with ddc.vim. Please create the minimal init.vim and the reproduce-able source code.

[Shougo]

neovim/neovim#19118 (comment)

Just install tailwindcss language server and try to use it some time. You will see that memory growing high (1.3GB process of nvim) and not handled properly.
i have tried nvim-cmp and ddc.vim.
In another editors and in coc-nvim - memory stable and not growing.
And because of it - i think some features of impl. of nvim native lSP is broken, don't know how to dig more into that.

It is really nvim-cmp problem?

[siblanco]

I didnt test ddc.vim, only coq_nvim, and no problems there. If you still need reproducable repos, let me know.

[Shougo]

I don't reproduce it with ddc.vim.
I will test it with nvim-cmp.
Please create the minimal init.vim.

[Shougo]

I have tested nvim-cmp with tailwindcss.
I have found lags but I don't reproduce the memory usage.

Note: The lags is not found in ddc.vim.

set rtp+=~/src/nvim-cmp
set rtp+=~/src/cmp-buffer
set rtp+=~/src/cmp-nvim-lsp
set rtp+=~/src/vim-vsnip
set rtp+=~/src/cmp-vsnip
set rtp+=~/src/nvim-lspconfig
set rtp+=~/src/cmp-cmdline

set title
set titlestring="%F %r"

lua <<EOF
  -- Setup nvim-cmp.
  local cmp = require'cmp'

  cmp.setup({
    snippet = {
      expand = function(args)
        print(args)
        vim.fn["vsnip#anonymous"](args.body)
      end,
    },
    mapping = {
      ['<CR>'] = cmp.mapping.confirm {
        behavior = cmp.ConfirmBehavior.Replace,
        select = true,
      },
      ['<tab>'] = cmp.mapping(cmp.mapping.confirm({ select = true }), { 'i', 's', 'c' }),
      ['<C-p>'] = cmp.mapping.select_prev_item(),
      ['<C-n>'] = cmp.mapping.select_next_item(),
    },
    sources = {
      { name = 'nvim_lsp' },
      { name = 'buffer' },
    },
    completion = {
    completeopt = 'menu,menuone,noselect'
    }
  })
  cmp.setup.cmdline(':', {
    mapping = cmp.mapping.preset.cmdline(),
    sources = {
      { name = 'cmdline' }
    }
  })
EOF

lua << EOF
-- Add additional capabilities supported by nvim-cmp
local capabilities = vim.lsp.protocol.make_client_capabilities()
capabilities = require('cmp_nvim_lsp').update_capabilities(capabilities)

--require'lspconfig'.sumneko_lua.setup{}
--require'lspconfig'.cssls.setup{}
--require'lspconfig'.pylsp.setup{}
require'lspconfig'.tailwindcss.setup{}
require'lspconfig'.gopls.setup{
  capabilities = capabilities,
  init_options = {
    usePlaceholders = true,
  },
}
EOF

runtime! after/plugin/*.lua

[Shougo]

It seems reproduced.

[Shougo]

nvim-cmp calls LSP completion when every key press and the items seems cached.
So it is laggy.

[Shougo]

It seems nvim-cmp problem instead of nvim-lsp.

@hrsh7th Can you test it?

[siblanco]

@Shougo thanks for your effort man, j just logged in to create the repo for you to see you are done already. Cheers, hope we find a quick fix

[hrsh7th]

I'd investigate this.

1. LuaJIT does not free memory space after garbage collection.

   • So if nvim-cmp consumes the 1GB memory space before working gabage collection, LuaJIT always ensure 1GB memory after garbage collection.
   • LuaJIT re-use garbage collected memory space so this behavior will be stop after ensured memory space is enough.

2. tailwindcss server always returns the 9k items

   • So nvim-cmp is slowdown because this works on the UI thread.
   • I found the small improvements in nvim-cmp. I'll push it.

[hrsh7th]

If you want to limit the allocation of memory space as much as possible, you can run collectgarbage('collect') on the InsertLeave event.
Note that this solution has a negative performance impact in some cases.

[hrsh7th]

I'd pushed the performance improvements in e1f3177

[schardev]

The Yioneko fork makes Tailwind usable. Hopefully someone smart can backport the algorithm back here, it is a night and day difference.

No need to backport anything. You can apply yioneko's changes as a patch to the master branch without any merge conflicts (for now). Though I do hope that hrsh7th can look up his commit and maybe hopefully improve on that whenever he gets time.

[lunacd]

This is probably slightly tangential, but how can we profile runtime performance of a plugin? Like this plugin: https://github.com/tweekmonster/startuptime.vim, but not just for startup time. Because I feel like trying to optimize without knowing what is taking time is pretty difficult.

[lunacd]

Nvm I found https://github.com/nvim-lua/plenary.nvim#plenaryprofile. Sorry for the noise.

[ghost]

Probably, the following snippet can be helped us.

cmp.setup {
  sources = {
    {
      name = 'nvim_lsp',
      max_item_count = 200,
    }
  }
}

This improves it for me in my particular setup. Thank you!

[dan-myles]

Just jumping in on this issue to say that using yioneko's fork has completely fixed my stuttering issues!

[siblanco]

I've applied the mentioned patch (https://github.com/yioneko/nvim-cmp/commit/ca812e4c0d677f4cebd94bbe7a875412dc8e90d2.patch). It does not really help, it still is stuttering for me.

[folke]

@hrsh7th I just did some testing and the extreme memory usage is caused by this line

nvim-cmp/lua/cmp/source.lua

Line 126 in d153771

self.cache:set({ 'get_entries', tostring(self.revision), ctx.cursor_before_line }, entries)

I don't really understand what that code does that calculates target_entries. Is this just an optimization so that you can filter entries that are already filtered?

In the case of tailwind, on my config, I get 11000 completion items per completion.

If I'm in a tsx file, inside a class="" string, and just type a couple of spaces, then memory usage increases very quickly to over 1GB.

If I remove the line that adds the entries to the cache, memory usage stays stable.

If I understand the code correctly, it's probably best to remove the whole target_entries calculation and just use self.entries?

[hrsh7th]

If the buffer source provides 2k items, the target_entries is still useful (for CPU).
The target_entries is memoizing the filtered results per each user input.

0. input=``, target_entries=2k
1. input=f, target_entries=500
2. input=fo, target_entries=100

If we remove the target_entries related stuff, the filtering process will be the following.

0. input=``, target_entries=2k
1. input=f, target_entries=2k
2. input=fo, target_entries=2k

I can provide the option for this.

[folke]

I'm working on a fix. It's not needed to process all the items since by default only 200 are returned?
So it should be enough to always filter self.entries but stop after 200 matches and no longer cache those results.
Especially since there's also the match_cache

[folke]

@hrsh7th #1574

A draft for now. I'm looking into further optimizing the entry matching.

With my changes, nvim-cmp is faster and memory usage is now stable, but I'll see if I can make some additional optims.

[hrsh7th]

I was reviewing the code and noticed that nvim-cmp was no longer returning correct results.

Originally, items should be cut after sorting by score. But already nvim-cmp is doing pre-filtering.

Also, be aware that your fix raises CPU load and reduces memory load.

google translated

[folke]

get_entries didn't do any sorting as far as I can see? It returns the first 200 items that match the filters. My changes do the same thing, no?

[hrsh7th]

Yes. Your fix does not affect the sorting and filtering results.
I mean the nvim-cmp is already doesn't return correct results.
So I don't mean your fix break the completion results.

But I notice your fix fixes memory issue but CPU issue does not solve (or make worse).

[hrsh7th]

I think that understanding and incorporating @yioneko's solution will be the fundamental solution (introducing a time budget to the filtering process)

But I have a lot of tasks, so I can't investigate right away.

It's also possible that yioneko's solution won't help either.

[folke]

@yioneko solution doesn't fix the memory issue. But I do agree that async processing will also be needed.

My solution does fix the memory issue and by printing the amount of processed entries before and after my changes, on average, less items are processed for filtering, so it does make it faster than before and does use less CPU.

Did you find the opposite?

[hrsh7th]

First, I think target_entries is

1. Increasing memory pressure because it's new cache data.
2. Reducing CPU pressure because it's reducing item count for filtering on each user input.

Your fix seems to be the following to me.

1. Decreasing memory pressure
2. Not sure about CPU pressure

• Your fix is increasing the item count for filtering process because it removes the target_entries cache.
• You fix is doing early return so it can be helped for CPU in this point.

[hrsh7th]

TBH, I'm confusing with this problem.

1. Some people say that yioneko's solution worked, some say it doesn't.
2. I didn't met the stutter with current nvim-cmp...

So contrary to my opinion, folke's solution may work.

I'll try native omnifunc with tailwindcss language server.

[folke]

@hrsh7th turns out that get_entries was called about 6 times for every completion.

I just fixed that as well, so that calculating the filtered items will only be done when the context changes. This makes a really big difference.

Stuttering in Tailwind is pretty much non-existent now for me.

[hrsh7th]

Very good news! The get_entries is most heavy process in nvim-cmp.
Thank you. I'll review it.

[bluz71]

TBH, I'm confusing with this problem.

1. Some people say that yioneko's solution worked, some say it doesn't.
2. I didn't met the stutter with current nvim-cmp...

In my case 100% @yioneko's solution solved my stutter issue completely. I never looked at RAM usage since I have so much RAM to begin with, but the stutter was very noticeable. With @yioneko's the stutter went away.

So contrary to my opinion, folke's solution may work.

From the little I have read of the conversation, @folke's solution is about avoiding unnecessary work by aborting early once internal nvim-cmp threshold's are exceeded.

I could well believe that @folke's solution would help with both excessive RAM and stutter. Hmm, as I am typing this, maybe it is better that I just try @folke's PR and report back rather than speak theoretically.

To-be-continued.

[hrsh7th]

IMO, the de-duplication of get_entries will fix most of the problems. folke thanks for finding this!

[samnj]

@yioneko's solution improved my sutter issue but didn't completely solve it. Classic @folke by the way. Thank you.

[bluz71]

My results:

• Current nvim-cmp stutters noticeably
• @folke's patch removes about 80% of the stutter, but I still notice a tiny amount of pause/lag; whilst fixing RAM issue
• @yioneko's patch removes almost 100% of the stutter from base-line nvim-cmp, but does not address RAM issue at all

Both @folke's (smartly avoid unnecessary work) and @yioneko's (async interrupt) patches would together appear to be the ultimate solution.

[ghost]

solved the issue: #1574

[ghost]

Thanks for a quick turn around on this one! Much appreciated!

[folke]

For anyone that wants to try, there's a new PR that makes cmp async #1583