Thoughts on future of PHP completion

[AJenbo]

Describe your idea in details

I’ve been a very happy user of CodeLite for years. Its PHP parser has generally been much faster than other tools I’ve tried. Even though its completion and features were not always industry-leading, they were rarely incorrect, which is more than I can say for some other IDEs such as PhpStorm.

Recently, a coworker who uses Neovim approached me about building a better PHP LSP in Rust than what is currently available (C#, PHP, and JS implementations). Over the past few weeks, I have been working on that:

https://github.com/AJenbo/phpantom_lsp
(CodeLite takes about 900MB RAM (about half is the PHP index) and 20 sec for the project used in the comparison table)

The results have been very promising. I have switched to using it instead of CodeLite’s built-in PHP parser. In my testing, PHPantom provides faster and more accurate completion and go-to-definition than other available PHP tools, with minimal memory usage even on large projects.

Given this, I am wondering whether it still makes sense to continue investing effort in the built-in PHP parsing and completion engine in CodeLite. Would there be interest in integrating PHPantom directly into CodeLite?

If so, I would be happy to focus my efforts on improving LSP support further. This approach could also simplify the PHP plugin considerably, potentially reducing it to mainly the Xdebug integration.

[eranif]

Sorry for the delayed response.

Recently, a coworker who uses Neovim approached me about building a better PHP LSP in Rust than what is currently available (C#, PHP, and JS implementations). Over the past few weeks, I have been working on that:

In general, I am all about removing custom parsing code from CodeLite and move it to LSP. I am not sure if you used CodeLite with Rust or C++ - but it completely relies on rust-analyzer / clangd. I tried to do the same move for PHP but I couldn't find a good enough LSP server for PHP, so I was left with the one that I have written couple of years ago.

Given this, I am wondering whether it still makes sense to continue investing effort in the built-in PHP parsing and completion engine in CodeLite. Would there be interest in integrating PHPantom directly into CodeLite?

I would vote for taking the LSP route.

P.S. I have been using Rust for over 6 years now, and it is an amazing language. If only it had a native GUI library (spoiler: it does not), I would use it for all my future projects :)

I think Rust is an excellent choice (mainly when combined with tokio)

[eranif]

BTW, have you tried it with CodeLite already? if it follows the LSP protocol, it should work out of the box.

[AJenbo]

Sorry for the delayed response.

No worries, I have had plenty of things to do :)

I am not sure if you used CodeLite with Rust or C++

I do use CodeLite with C++, since 2018

I couldn't find a good enough LSP server for PHP

Yep, I hit that same problem.

BTW, have you tried it with CodeLite already?

That was the first thing I got it running in :) It was also the easiest for getting debug info from the server.

I think PHPantom is already good enough that switching to it would be an upgrade compared to what's in CodeLite now. Do you have a checklist of what you would want from it, and how should we bundle it?

[eranif]

I do use CodeLite with C++, since 2018

I think you will be surprised how good it is with Rust as well :)

I think PHPantom is already good enough that switching to it would be an upgrade compared to what's in CodeLite now. Do you have a checklist of what you would want from it, and how should we bundle it?

Usually, as with other LSPs we don't ship them directly, but we add support for auto-detection.
The idea is that you install it and CodeLite should pick it automatically (it does that on startup and by clicking the "Scan" button).

Having said that, we do make exceptions for Windows (where building something is usually harder than other platforms), so for example, we do package clangd.exe & clang-format.exe with Windows, I guess that we can do the same for the PHP LSP.

Example: this is how rust-analyzer is being detected by CodeLite:
https://github.com/eranif/codelite/blob/master/Plugin/LSP/detectors/LSPRustAnalyzerDetector.cpp

In addition, we should definitely add support in the documentation section:
https://github.com/eranif/codelite/blob/master/docs/docs/plugins/lsp.md

This is how it looks like on the website:
https://docs.codelite.org/plugins/lsp/#installing-language-servers

The main features that I would expect from any decent LSP are:

• textDocument/definition
• textDocument/implementation (in some LSPs these are identical, i.e. it makes sense for C++/C not so much for Java for example)
• textDocument/semanticTokens/full - this one is important as it controls the color of various words in the editor
• textDocument/completion
• textDocument/signatureHelp
• textDocument/hover
• textDocument/documentSymbol - current file's symbols

An advanced features (nice to have, but usually are more complicated to implement):

• textDocument/references
• textDocument/codeAction
• textDocument/rename
• workspace/symbol - workspace wide symbols.

[AJenbo]

I guess that we can do the same for the PHP LSP.

It would make sens since the component we are replacing was bunded so there is a real chance users might not know what to do and see it more as a loss of functionality.

I do make prebuilds for various platforms so it could also offer to download it if needed.

The main features that I would expect from any decent LSP are:

What's missing:

• textDocument/semanticTokens/full (Implemented in main)
• textDocument/documentSymbol (Implemented in main)

An advanced features (nice to have, but usually are more complicated to implement):

What's missing:

• workspace/symbol (Implemented in main)

[AJenbo]

All 3 are now implemented and will be part of the next release.

[eranif]

Nice. I added support for CodeLite, which includes:

• I changed the PHP syntax highlighting so it will also support variables / methods & classes for PHP. To get this change, you need to: Settings -> Colours & Fonts -> Defaults (this is done this way for backward compatibility).
• Added auto-detection for phphantom_lsp executable - if it is in the PATH, CodeLite will pick it up: by: Plugins -> Language Servers -> Settings and Scan button. OR if it is a new installation, CodeLite will also pick it up without any user intervention.
• Documentation was added

It works nicely, I updated my local build and tried the new features you added:

• textDocument/semanticTokens/full worked out of the box (you do need to run the first step I mentioned to get the full colouring)
• textDocument/documentSymbol - this one did not work for me, this is due to the fact that CodeLite only supported SymbolInformation[] result and not DocumentSymbol[] (which you used). I fixed that and it is now working as expected.
• workspace/symbol - does not work for me, I suspect a similar issue.

Code completion / Find references / Find Symbol => are working with no apparent issues.

This addition, basically makes the PHPWorkspace class redundant. As we can now a simple "File System Workspace" with LSP enabled. The only missing part of integrating X-debug into the Debug tab of the "File System Workspace".

Once this is done (or a reasonable workaround is presented) - I will deprecate that class and provide a migration code from the PHP workspace -> "File System Workspace".

Outline View

Colouring - notice that some "variables" are coloured as class (e.g. $msg), I am not sure where the bug is (CodeLite or the LSP).

[eranif]

EDIT: I found and fixed the workspace symbols not working with this commit: 824b4a7.

So basically, I can confirm that all 3 issues are fixed.

[AJenbo]

PHPWorkspace class redundant. As we can now a simple "File System Workspace"

This also solves some of the performance issued I had reported: #2598 probably also this one #3577

[AJenbo]

textDocument/semanticTokens/full

There was also a report from the Neovim users that this needs a bit of adjusting.

[eranif]

Everything should be working for you now. Please give it a try and let me know if something is broken for you...

[AJenbo]

Here is the current status in terms of replacing CodeLite PHP features.

It now also covers php-cs-fixer intergration, phpcbf intergration, phpstan intergration, docblock generation, method extraction, it has build in syntax error detection, use statment clean up, rename symbol.

It doesn't cover PHPMD yet, I'm waiting until 3.0 is released (it's PHP 8.1+ so it won't work for very old project, but unlike 2.x it will work for PHP 8.4+).

It doesn't cover PHPCS (it does cover it's fixer/formatter phpcbf) for errors, but might in the future.

It doesn't and won't support PSR-0 class renaming for entier folders. PSR-0 was old when we added this integration in CodeLite, but it does handle renaming of files when renaming classes, it will support namespace renaming, and it does suggest class name based on file name. So I don't think there is a meaning full loss here.

I'm undecided weather it will support "Convert to instance variable", but this does seem to be completely broken in CodeLite (my fork of Refactor Browser more specifically).

I don't see a way for it to support Xdebug debugging so that feature will have to stay as part of CodeLite's own PHP support. Still this should be a massive reduction in CodeLite's code-base and feature improvement at the same time.

Let me know if I missed anything. Once PHPMD, PHPCS, and "Convert to instance variable" is dealt with I'll start creating PRs for cleaning up CodeLite's own code.