Projects using hpack (not .cabal) cannot be built

[srid]

Seems to be a regression from #57 #42

       > No cabal.project file or cabal file matching the default glob './*.cabal' was found.
       > Please create a package description file <pkgname>.cabal or a cabal.project file referencing the packages you want to build.

Turns out that the use of buildFromCabalSdist is the culprit:

haskell-flake/haskell-project.nix

Lines 36 to 50 in 34641d4

	let
	fromSdist = self.buildFromCabalSdist or (builtins.trace "Your version of Nixpkgs does not support hs.buildFromCabalSdist yet." (pkg: pkg));
	filterSrc = name: src: lib.cleanSourceWith { inherit src name; filter = path: type: true; };
	in
	lib.mapAttrs
	(name: value:
	let
	# callCabal2nix does not need a filtered source. It will
	# only pick out the cabal and/or hpack file.
	pkgProto = self.callCabal2nix name value.root { };
	pkgFiltered = pkgs.haskell.lib.overrideSrc pkgProto {
	src = filterSrc name value.root;
	};
	in
	fromSdist pkgFiltered)

[srid]

@roberth I failed to catch this during the PR review, but do you know what the purpose of cleanSourceWith here, is? Because it is not really doing any filtering.

haskell-flake/haskell-project.nix

Line 38 in 34641d4

filterSrc = name: src: lib.cleanSourceWith { inherit src name; filter = path: type: true; };

Also, from your original PR #57:

This prevents unnecessary rebuilds of packages that haven't really changed, except for a hash that included irrelevant sources.

Does it really do this? Because, with filtering disabled I can confirm that the foo package of https://github.com/srid/haskell-multi-nix does not rebuild when either the top-level README.md or the Haskell source inside of ./bar changes.

Or do you have an example of a specific scenario under which this kind of filtering prevents rebuilds?

[roberth]

I think it's supposed to cut the src directory from any parent directory contents, and it could serve as an extension point to develop a filter that filters contents within the directory too.

Maybe there's something different about the situation where I used it, such as using a path value instead of string or vice versa. I did set source overrides using paths at somepoint instead of using the default behavior, which is self.outPath string-based iirc.

[srid]

What would happen if we revert #57 ? Is there a particular scenario where #57 is useful? Just trying to see why we need it -- and if so, at what level -- especially given this bug (#85).

I can see buildFromCabalSdist being useful in including only files recognized by cabal, but that doesn't work with hpack.

[roberth]

It probably regresses the rebuilds of packages whose sources are specified with a path value, such as

packages.foo.root = ./haskell/foo;

I have not confirmed this.

can see buildFromCabalSdist being useful in including only files recognized by cabal, but that doesn't work with hpack

That invocation existed before #57. Did you refer to the intended PR?

[ParetoOptimalDev]

Does it really do this? Because, with filtering disabled I can confirm that the foo package of https://github.com/srid/haskell-multi-nix does not rebuild when either the top-level README.md or the Haskell source inside of ./bar changes.

I've wondered this for a while. I know that there seems to be schism in practice between people who use things like gitignore.nix and those that just use the source alone.

My memory of browsing search results was that those who use things like gitignore.nix are in the minority and I thought the change @roberth made would be a good way to give people the desired behavior.

I can't tell you how to reproduce it, but I feel like I've ran into this rebuild issue as well. That's not very actionable though except perhaps to encourage a little more thought or nudge me to go try and reproduce it again if I have time.

[srid]

Did you refer to the intended PR?

Oh, you are right. This issue was introduced in #42 (not 57). "This catches incomplete release tarballs before they get out." - well, that's useful feature to have. So, how do we support projects with package.yaml (but no .cabal file) ...

[srid]

Workaround:

{
          basePackages = pkgs.haskellPackages.extend (self: super: {
            # Workaround for https://github.com/srid/haskell-flake/issues/85
            buildFromCabalSdist = x: x;
          });
}

how do we support projects with package.yaml (but no .cabal file) ...

Perhaps a project-level option acting as source filter:

{
   haskellProjects.default = {
     sourceFilter = hp: src: src;  # Default uses hp.buildFromCabalSdist
     packages = { .. };
   };
}

But does such an option (sourceFilter) have valuable use cases, beyond working around this bug?

Perhaps we should fix buildFromCabalSdist itself to run hpack just like cabal2nix does.