Windows long path support

[HertzDevil]

By default, filenames on Windows cannot be longer than 256 characters, excluding the drive name. There are two ways to overcome this since Windows 10 version 1607:

• Replace path with "\\\\?\\#{File.expand_path(path)}" at the Win32 API boundary;
• Set the Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled registry key to 1 and link a manifest containing the longPathAware element into every program.

MSVC can only embed one manifest into an executable, so if the user provides their own manifest, they have to merge their own manifest with the one generated by Crystal, or Crystal's CLI has to support manifests specifically. There is also no guarantee that the registry key can be set by a user. Therefore, the first option is probably more suitable for the standard library, as all we need to do is trying the extended name if the original name fails:

module Crystal::System::Dir
  def self.create(path : String, mode : Int32) : Nil
    if LibC._wmkdir(System.to_wstr(path)) == -1
      if LibC._wmkdir(System.to_wstr("\\\\?\\#{File.expand_path(path)}")) == -1
        raise ::File::Error.from_errno("Unable to create directory", file: path)
      end
    end
  end
end

The second call can most likely be skipped too if path already begins with something like \\.\. (The standard library does not publicly state that those names are usable in the file APIs, but in most cases those String paths are passed unmodified all the way to the Win32 API functions.)

Although not mentioned in the linked article, it seems C runtime functions also support long paths in this manner. It might be better to switch to their Win32 equivalents whenever possible though (CreateDirectoryW in this example).

[stakach]

I think changing the path at the win32 boundary makes the most sense
Check if size > 256 at the boundary and then could check for windown >= 10 and run the success path else raise an error

assuming you want to support win8 / 7 etc I can't imagine there is a huge demand for anything less than 10

[HertzDevil]

path.size > 256 is not a necessary condition here, because at this point path could still be a relative path and the current working directory could have been near the MAX_PATH limit.

[stakach]

reading the docs, the "\\\\?\\#{File.expand_path(path)}" at the boundary should work in older versions of windows too
the registry key and manifest are only required if you want long paths using the common Win32 file and directory functions
versus the Unicode versions that permit an extended-length path

[HertzDevil]

Note that contrary to the docs, SetCurrentDirectoryW does not work for long paths, and when you are inside a long directory in the command prompt, you can't start any processes at all.

[stakach]

the command prompt has issues with long paths. The cmd.exe process does not support long paths, even with the manifest file - need to use powershell

I think the only workaround is to use paths with the \\?\ prefix

also SetCurrentDirectoryW, GetCurrentDirectoryW, GetFullPathNameW won't work with long paths unless the \\?\ prefix is provided

[HertzDevil]

.NET does its own thing, but the same error happens as soon as it tries to do non-.NET things:

What I'm saying is long paths aren't supported here even with \\?\.

For that matter the command prompt does do its own thing too; that's why each drive has its "own" working directory and also why internal variables like %=C:% exist.

[stakach]

I think that's unrelated as when you try to start a new process in a directory with a path that's too long, the new process will inherit the terminals current directory, which would cause problems if the path is longer than 260 characters.

[mominshaikhdevs]

CreateProcessW does not support long paths at all (both \\?\ and manifest files).

Those APIs need to support long paths in future Windows versions.

[stakach]

Fairly certain that the application can be at a long path location, that is lpApplicationName can be a long path.
It's only the lpCurrentDirectory param that does not support long paths and it defaults to the application name path (which might be where the confusion is happening?)

See details here: https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessw

MS has a pretty messed up API

[TwoLeaves]

Is this issue the cause of the error I'm seeing when trying to compile a Crystal program on Windows? And is there a workaround? The error is as below (experienced when compiling natively and cross-compiling from Linux):

In /usr/lib/crystal/crystal/system/win32/process.cr:210:48

 210 | make_env_block(env, clear_env), chdir.try { |str| System.to_wstr(str) },
                                             ^--
Error: argument 'lpCurrentDirectory' of 'LibC#CreateProcessW' must be Pointer(UInt16), not (Pointer(UInt16) | Nil)