Thoughts on Ark / ark.nvim

[wurli]

Hey @jalvesaq, @PMassicotte :)

You might have seen already, but I recently started work on ark.nvim, a plugin which in theory could eventually replicate a lot of the functionality of R.nvim using the Ark Jupyter kernel as a back-end.

The obvious question is, would it be better to just bring Ark into R.nvim? My feeling is no, I don't think Ark would be a good fit for this plugin. The reasons for this are outlined below, copied from a response I wrote to a question from an R.nvim user. I'll probably add something like this to the ark.nvim README if you both agree with my thinking.

Currently ark.nvim is at 'minimum viable product' stage, and is really just a proof of concept. Whether or not I'll manage to develop it into something more serious, time will tell! However, my plan for the project is to use Rust to create a framework for Neovim to talk to Jupyter kernels, then use this to power ark.nvim.

Very grateful to hear any thoughts from either of you about the project, and especially how you think it could/should relate to R.nvim! I don't see it becoming a viable alternative to R.nvim any time soon, but best case scenario, eventually it could do.

Copied from my response to a question on Reddit:

Personally I don't think Ark is a good fit for R.nvim; here's why. The real 'killer app' of R.nvim is {nvimcom}, an R package that powers the plugin and allows R to communicate with the Neovim session. This also works with cmp-r, a source for cmp-nvim which provides completions which respond to the R environment. For other LSP features, R.nvim recommends the {languageserver} project.

Ark, on the other hand, is a Jupyter kernel which wraps R. This architecture means that all communication with R needs to go via Jupyter, which would mean {nvimcom} would not be usable with the project. The Jupyter kernel then spins up an LSP server which, similarly to cmp-r, provides autocompletions which respond to the R environment. This LSP server would completely replace both cmp-r and {languageserver}.

In theory, Ark can also be used to power a variables pane, which would replace R.nvim's global environment window. Ark also has some other nice stuff, like a DAP server, which currently has no R.nvim equivalent.

So, if R.nvim did adopt Ark, it would actually be easier to do a full rewrite of the plugin anyway. It would also likely necessitate a move to a new back-end architecture, e.g. Python or Rust. This is why I decided to just start a standalone project, which, if it goes anywhere, will definitely take a while to become nearly as usable as R.nvim.

Another important consideration is that Jakson, the main author and maintainer of R.nvim, isn't a Rust user. Ark is built in Rust, and this would be a big barrier to development. Jakson recently commented that it would be better not to integrate the Air formatter with R.nvim, since there would be some big risks and costs associated with bringing a Rust dependency, or a compiled binary, into R.nvim. Instead, he agreed that users should just use Air as an LSP server to complement R.nvim. I think his concerns would apply even more so if Ark were brought in as a dependency to the project.

[PMassicotte]

That is nice to see another project. In fact, just starting the kernel :lua require("ark").start_lsp() makes lsp completion available in R.nvim

Besides that, I think that air will eventually be a full lsp that could be added to whatever IDE people want to use. What do you think?

[jalvesaq]

That's great! In my opinion, we can create options to disable in R.nvim what works better in ark.nvim. Features clearly better in ark.nvim (faster, using less computer resources, more reliable, etc), can be removed from R.nvim.

[wurli]

Thanks for your comments, both!

Besides that, I think that air will eventually be a full lsp that could be added to whatever IDE people want to use. What do you think?

Yeah, I believe that's the plan. In fact, from what I've seen in GitHub issues, lots of features will eventually be migrated to Air from Ark – but not the ability for the LSP to respond to changes in the R environment. I suppose Air will probably become the de-facto LSP server for R in most editors, but I also think there's room for a more specialised plugin which wraps Ark to get more advanced LSP features, plus potentially a variables pane, DAP server, etc.

That's great! In my opinion, we can create options to disable in R.nvim what works better in ark.nvim. Features clearly better in ark.nvim (faster, using less computer resources, more reliable, etc), can be removed from R.nvim.

I think this could make sense in the long term! In that case I'll continue working on the plugin in my spare time, and once I feel confident in the general architecture perhaps we could discuss how to optimise the two plugins to play nicely with each other :)

Thanks!

[PMassicotte]

but not the ability for the LSP to respond to changes in the R environment.

Can you explain what you mean here? I am not sure to understand :)

[wurli]

Can you explain what you mean here? I am not sure to understand :)

No problem :) For example take the following code:

x <- iris
x$

With Ark, similarly to cmp-r, the cursor just after the $ should trigger autocomplete with the column names from the iris dataset - but only if you've already run the first line of code; the creation of x would be the change in the R environment in this case. I believe the plan for Air is for it to purely 'static' analysis based on the text files in your project, so the LSP server would have no way of knowing what the columns of x are.

[PMassicotte]

Excellent explanation, tx!