fix(snippets): filter out incompatible builtin snippets

[soifou]

Skip early incompatible builtin snippets using LSP grammar.

Closes #2028

[soifou]

This does not skip only "friendly snippets" but all snippets that fail to parse (mainly nested placeholders). We could try to add more check and try to fix some of them (which I did for options), this method is very naive though.

mini.nvim process snippet nodes char by char which is way more powerful but also complex. Don't know if we want to go on this path.

[saghen]

Perhaps it'd make sense to only skip friendly snippets that fail to parse and log an error when user snippets fail to parse? Otherwise we'd be silently ignoring issues with the user's config. Also it'd be great if you could double check the performance as iirc snippet loading had significant overhead already. Cache likely makes it a non-issue though

[soifou]

Sounds good! I'll do some additional testing and get back to you when I'll be on the right track.

[soifou]

Pushed an update. I tried to fix snippets that were not coming from the user configuration (most likely from friendly-snippets) and added a message when parsing fails for user snippets; otherwise, they are silently discarded. I also attempted to filter out unsupported snippets using a regex as a lightweight pre-check before applying the LSP grammar, but my testing was inconclusive.

As for the numbers, around 170 friendly-snippets were fixed. There is some overhead from the gsub operations and LSP parsing, but performance seems fine to me. Terraform has over 600 snippets, and it loads correctly in nvim even on my older machine.

Below are the current results (fastest to slowest) using the provided benchmark methods.

Current logic - Cold run (load registry + trigger completion per filetype)

Run 1: 0.00519 seconds, snippets: 9, filetype: kubernetes
Run 1: 0.00525 seconds, snippets: 11, filetype: loremipsum
Run 1: 0.00541 seconds, snippets: 12, filetype: zig
Run 1: 0.00545 seconds, snippets: 9, filetype: nushell
Run 1: 0.00562 seconds, snippets: 24, filetype: purescript
Run 1: 0.00566 seconds, snippets: 9, filetype: rmarkdown
Run 1: 0.00572 seconds, snippets: 9, filetype: PowerShell
Run 1: 0.00583 seconds, snippets: 22, filetype: reason
Run 1: 0.00589 seconds, snippets: 21, filetype: scala
Run 1: 0.00589 seconds, snippets: 40, filetype: haskell
Run 1: 0.00590 seconds, snippets: 14, filetype: make
Run 1: 0.00593 seconds, snippets: 32, filetype: org
Run 1: 0.00597 seconds, snippets: 29, filetype: swift
Run 1: 0.00604 seconds, snippets: 19, filetype: nix
Run 1: 0.00609 seconds, snippets: 19, filetype: fennel
Run 1: 0.00610 seconds, snippets: 18, filetype: editorconfig
Run 1: 0.00624 seconds, snippets: 9, filetype: erb
Run 1: 0.00629 seconds, snippets: 24, filetype: gitcommit
Run 1: 0.00633 seconds, snippets: 36, filetype: cmake
Run 1: 0.00648 seconds, snippets: 24, filetype: dart
Run 1: 0.00656 seconds, snippets: 49, filetype: plantuml
Run 1: 0.00657 seconds, snippets: 16, filetype: eelixir
Run 1: 0.00664 seconds, snippets: 16, filetype: erlang
Run 1: 0.00669 seconds, snippets: 47, filetype: sql
Run 1: 0.00673 seconds, snippets: 26, filetype: license
Run 1: 0.00675 seconds, snippets: 35, filetype: mint
Run 1: 0.00677 seconds, snippets: 48, filetype: gleam
Run 1: 0.00678 seconds, snippets: 28, filetype: beancount
Run 1: 0.00679 seconds, snippets: 33, filetype: perl
Run 1: 0.00682 seconds, snippets: 87, filetype: julia
Run 1: 0.00684 seconds, snippets: 40, filetype: rst
Run 1: 0.00689 seconds, snippets: 36, filetype: fsh
Run 1: 0.00690 seconds, snippets: 18, filetype: global
Run 1: 0.00694 seconds, snippets: 62, filetype: norg
Run 1: 0.00698 seconds, snippets: 9, filetype: latex
Run 1: 0.00708 seconds, snippets: 45, filetype: elixir
Run 1: 0.00726 seconds, snippets: 26, filetype: verilog
Run 1: 0.00743 seconds, snippets: 34, filetype: r
Run 1: 0.00765 seconds, snippets: 34, filetype: gdscript
Run 1: 0.00772 seconds, snippets: 110, filetype: glsl
Run 1: 0.00777 seconds, snippets: 59, filetype: kivy
Run 1: 0.00794 seconds, snippets: 80, filetype: markdown
Run 1: 0.00803 seconds, snippets: 128, filetype: asciidoc
Run 1: 0.00819 seconds, snippets: 86, filetype: tcl
Run 1: 0.00832 seconds, snippets: 57, filetype: vhdl
Run 1: 0.00839 seconds, snippets: 76, filetype: rescript
Run 1: 0.00843 seconds, snippets: 79, filetype: objc
Run 1: 0.00853 seconds, snippets: 84, filetype: quarto
Run 1: 0.00869 seconds, snippets: 127, filetype: go
Run 1: 0.00871 seconds, snippets: 54, filetype: solidity
Run 1: 0.00879 seconds, snippets: 93, filetype: fortran
Run 1: 0.00935 seconds, snippets: 137, filetype: liquid
Run 1: 0.00951 seconds, snippets: 165, filetype: css
Run 1: 0.00958 seconds, snippets: 257, filetype: html
Run 1: 0.01108 seconds, snippets: 193, filetype: systemverilog
Run 1: 0.01470 seconds, snippets: 304, filetype: svelte
Run 1: 0.01981 seconds, snippets: 646, filetype: terraform

New logic WITH FIXES - Cold run (load registry + trigger completion per filetype)

Run 1: 0.00514 seconds, snippets: 9, filetype: PowerShell
Run 1: 0.00541 seconds, snippets: 12, filetype: zig
Run 1: 0.00559 seconds, snippets: 9, filetype: rmarkdown
Run 1: 0.00578 seconds, snippets: 24, filetype: purescript
Run 1: 0.00588 seconds, snippets: 9, filetype: nushell
Run 1: 0.00617 seconds, snippets: 9, filetype: kubernetes
Run 1: 0.00650 seconds, snippets: 18, filetype: global
Run 1: 0.00654 seconds, snippets: 12, filetype: make
Run 1: 0.00654 seconds, snippets: 34, filetype: gdscript
Run 1: 0.00659 seconds, snippets: 48, filetype: gleam
Run 1: 0.00664 seconds, snippets: 9, filetype: latex
Run 1: 0.00668 seconds, snippets: 34, filetype: r
Run 1: 0.00671 seconds, snippets: 21, filetype: scala
Run 1: 0.00678 seconds, snippets: 29, filetype: swift
Run 1: 0.00687 seconds, snippets: 30, filetype: org
Run 1: 0.00696 seconds, snippets: 19, filetype: reason
Run 1: 0.00700 seconds, snippets: 11, filetype: loremipsum
Run 1: 0.00701 seconds, snippets: 35, filetype: mint
Run 1: 0.00713 seconds, snippets: 19, filetype: fennel
Run 1: 0.00713 seconds, snippets: 62, filetype: norg
Run 1: 0.00715 seconds, snippets: 19, filetype: nix
Run 1: 0.00716 seconds, snippets: 49, filetype: plantuml
Run 1: 0.00723 seconds, snippets: 9, filetype: erb
Run 1: 0.00740 seconds, snippets: 24, filetype: gitcommit
Run 1: 0.00757 seconds, snippets: 26, filetype: haskell
Run 1: 0.00758 seconds, snippets: 33, filetype: perl
Run 1: 0.00762 seconds, snippets: 40, filetype: rst
Run 1: 0.00771 seconds, snippets: 16, filetype: erlang
Run 1: 0.00774 seconds, snippets: 26, filetype: verilog
Run 1: 0.00780 seconds, snippets: 57, filetype: vhdl
Run 1: 0.00794 seconds, snippets: 47, filetype: sql
Run 1: 0.00816 seconds, snippets: 59, filetype: kivy
Run 1: 0.00835 seconds, snippets: 73, filetype: rescript
Run 1: 0.00848 seconds, snippets: 16, filetype: eelixir
Run 1: 0.00902 seconds, snippets: 18, filetype: fsh
Run 1: 0.00922 seconds, snippets: 65, filetype: fortran
Run 1: 0.00927 seconds, snippets: 18, filetype: editorconfig
Run 1: 0.00930 seconds, snippets: 84, filetype: quarto
Run 1: 0.00955 seconds, snippets: 110, filetype: glsl
Run 1: 0.00958 seconds, snippets: 87, filetype: julia
Run 1: 0.00977 seconds, snippets: 79, filetype: objc
Run 1: 0.00979 seconds, snippets: 127, filetype: asciidoc
Run 1: 0.00994 seconds, snippets: 45, filetype: elixir
Run 1: 0.01006 seconds, snippets: 24, filetype: dart
Run 1: 0.01057 seconds, snippets: 127, filetype: go
Run 1: 0.01059 seconds, snippets: 34, filetype: cmake
Run 1: 0.01108 seconds, snippets: 80, filetype: markdown
Run 1: 0.01129 seconds, snippets: 28, filetype: beancount
Run 1: 0.01135 seconds, snippets: 26, filetype: license
Run 1: 0.01173 seconds, snippets: 86, filetype: tcl
Run 1: 0.01304 seconds, snippets: 257, filetype: html
Run 1: 0.01408 seconds, snippets: 137, filetype: liquid
Run 1: 0.01431 seconds, snippets: 165, filetype: css
Run 1: 0.01495 seconds, snippets: 189, filetype: systemverilog
Run 1: 0.01514 seconds, snippets: 303, filetype: svelte
Run 1: 0.02958 seconds, snippets: 54, filetype: solidity
Run 1: 0.06219 seconds, snippets: 616, filetype: terraform

New logic NO FIX - Cold run (load registry + trigger completion per filetype)

Run 1: 0.00496 seconds, snippets: 9, filetype: erb
Run 1: 0.00524 seconds, snippets: 16, filetype: eelixir
Run 1: 0.00536 seconds, snippets: 9, filetype: kubernetes
Run 1: 0.00551 seconds, snippets: 12, filetype: zig
Run 1: 0.00560 seconds, snippets: 9, filetype: PowerShell
Run 1: 0.00578 seconds, snippets: 24, filetype: dart
Run 1: 0.00583 seconds, snippets: 12, filetype: make
Run 1: 0.00591 seconds, snippets: 9, filetype: nushell
Run 1: 0.00595 seconds, snippets: 16, filetype: erlang
Run 1: 0.00595 seconds, snippets: 9, filetype: rmarkdown
Run 1: 0.00609 seconds, snippets: 24, filetype: purescript
Run 1: 0.00617 seconds, snippets: 18, filetype: global
Run 1: 0.00620 seconds, snippets: 19, filetype: reason
Run 1: 0.00622 seconds, snippets: 11, filetype: loremipsum
Run 1: 0.00638 seconds, snippets: 18, filetype: editorconfig
Run 1: 0.00639 seconds, snippets: 36, filetype: cmake
Run 1: 0.00667 seconds, snippets: 9, filetype: latex
Run 1: 0.00668 seconds, snippets: 24, filetype: gitcommit
Run 1: 0.00677 seconds, snippets: 18, filetype: fsh
Run 1: 0.00677 seconds, snippets: 21, filetype: scala
Run 1: 0.00678 seconds, snippets: 40, filetype: rst
Run 1: 0.00679 seconds, snippets: 34, filetype: r
Run 1: 0.00681 seconds, snippets: 45, filetype: elixir
Run 1: 0.00687 seconds, snippets: 19, filetype: fennel
Run 1: 0.00692 seconds, snippets: 59, filetype: kivy
Run 1: 0.00701 seconds, snippets: 34, filetype: gdscript
Run 1: 0.00702 seconds, snippets: 60, filetype: norg
Run 1: 0.00705 seconds, snippets: 29, filetype: org
Run 1: 0.00706 seconds, snippets: 73, filetype: rescript
Run 1: 0.00711 seconds, snippets: 33, filetype: perl
Run 1: 0.00714 seconds, snippets: 28, filetype: swift
Run 1: 0.00719 seconds, snippets: 57, filetype: vhdl
Run 1: 0.00721 seconds, snippets: 19, filetype: nix
Run 1: 0.00721 seconds, snippets: 49, filetype: plantuml
Run 1: 0.00723 seconds, snippets: 26, filetype: haskell
Run 1: 0.00737 seconds, snippets: 48, filetype: gleam
Run 1: 0.00768 seconds, snippets: 47, filetype: sql
Run 1: 0.00773 seconds, snippets: 35, filetype: mint
Run 1: 0.00779 seconds, snippets: 28, filetype: beancount
Run 1: 0.00781 seconds, snippets: 26, filetype: verilog
Run 1: 0.00794 seconds, snippets: 85, filetype: tcl
Run 1: 0.00801 seconds, snippets: 65, filetype: fortran
Run 1: 0.00824 seconds, snippets: 87, filetype: julia
Run 1: 0.00848 seconds, snippets: 110, filetype: glsl
Run 1: 0.00848 seconds, snippets: 127, filetype: asciidoc
Run 1: 0.00860 seconds, snippets: 165, filetype: css
Run 1: 0.00863 seconds, snippets: 80, filetype: markdown
Run 1: 0.00875 seconds, snippets: 84, filetype: quarto
Run 1: 0.00895 seconds, snippets: 127, filetype: go
Run 1: 0.00939 seconds, snippets: 26, filetype: license
Run 1: 0.00990 seconds, snippets: 257, filetype: html
Run 1: 0.00994 seconds, snippets: 79, filetype: objc
Run 1: 0.01032 seconds, snippets: 54, filetype: solidity
Run 1: 0.01035 seconds, snippets: 137, filetype: liquid
Run 1: 0.01062 seconds, snippets: 189, filetype: systemverilog
Run 1: 0.01303 seconds, snippets: 298, filetype: svelte
Run 1: 0.02322 seconds, snippets: 607, filetype: terraform

Benchmark source

-- nvim --cmd "set rtp+=~/Development/nvim/blink.cmp" --cmd "set rtp+=~/.local/share/nvim/lazy/friendly-snippets" -l bench/snippet.lua

-- ┏━┓┏━┓┏━╸┏━┓
-- ┣━┫┣┳┛┃╺┓┗━┓
-- ╹ ╹╹┗╸┗━┛┗━┛
local iterations = tonumber((vim and vim.v and vim.v.argv and vim.v.argv[8]) or 1) or 1
local ft_snippet = (vim and vim.v and vim.v.argv and vim.v.argv[9]) or 'markdown'
print('Running benchmark for ' .. iterations .. ' iterations - ' .. ft_snippet)

-- ┏━┓┏━┓╺┳╸┏━┓
-- ┃ ┃┣━┛ ┃ ┗━┓
-- ┗━┛╹   ╹ ┗━┛
---@type blink.cmp.SnippetsOpts
local config = {
  friendly_snippets = true,
  get_filetype = function() return ft_snippet end,
  search_paths = {
    vim.fn.stdpath('config') .. '/snippets/vscode',
  },
}

-- ┏┓ ┏━╸┏┓╻┏━╸╻ ╻
-- ┣┻┓┣╸ ┃┗┫┃  ┣━┫
-- ┗━┛┗━╸╹ ╹┗━╸╹ ╹
local snip = require('blink.cmp.sources.snippets.default')
local context = require('blink.cmp.completion.trigger.context').new({
  id = 0,
  mode = 'default',
  providers = { 'snippets' },
  line = '',
  initial_trigger_kind = 'manual',
  initial_trigger_character = '',
  trigger_kind = 'manual',
  initial_selected_item_idx = 1,
})
local builtin_snippets = snip.new(config)

local function benchmark(iterations)
  iterations = iterations or 10
  local total_time = 0

  for i = 1, iterations do
    local start_time = os.clock()

    local completion_items = 0
    builtin_snippets:get_completions(context, function(res)
      completion_items = #res.items
    end)

    local elapsed = os.clock() - start_time
    print(string.format('Run %d: %.5f seconds, snippets: %d, filetype: %s', i, elapsed, completion_items, ft_snippet))
    total_time = total_time + elapsed
  end

  local avg_time = total_time / iterations
  print(string.format('Average time over %d runs: %.5f seconds, filetype %s', iterations, avg_time, ft_snippet))
  return avg_time
end

benchmark(iterations)

#!/usr/bin/env sh

# Usage:
# time ./snip > /dev/null 2>&1
# ./snip 2>&1 | grep 'Run 1:' | sort -k8

fs_path=~/.local/share/nvim/lazy/friendly-snippets

for ft in $(find $fs_path/snippets -maxdepth 1 -name '*.json' -exec basename {} \; | sed 's/.json//' | sort); do
nvim --cmd "set rtp+=~/Development/nvim/blink.cmp" --cmd "set rtp+=$fs_path" -l bench/snippet.lua 1 $ft
done