Project Refactor, Act 2

[oscarlevin]

I've started to wire up the new refactored project with the rest of the CLI. I published a branch refactor2 with my first steps (basically just getting pretext support working).

Let's continue the discussion from #458 here as we work on this.

In terms of workflow, I suggest anyone who wants to contribute to this part make a pull request to the refactor2 branch. I will merge those in, and when it is all working, we can do a pull request to main.

[bjones1]

I'm working on replacing output in the project file with output_dir and output_file per our discussion, but running into some problems: In the current build.py, we don't pass an output filename to the latex builder at all: the latex function in build.py calls core.latex with the parameter out_file (second to last argument) as None:

            core.latex(
                ptxfile,
                pub_file.as_posix(),
                stringparams,
                custom_xsl and custom_xsl.as_posix(),
                None,  # <================ This is out_file.
                output.as_posix(),  # <========= This is dest_dir.
            )

Do we want to stay with this -- the CLI only builds Latex to a provided directory, not to a specific file -- or allow specifying an output filename? (The same pattern seem used in most of the builders -- would the same answer apply?)

[oscarlevin]

I think it would be nice to give users the option, as long as core allows it (which I think it does). This will be especially nice for the multiple-version-deploy that we are rolling out.

[bjones1]

OK, I'm moving forward with this.

[bjones1]

@oscarlevin, it looks like refactor.py is now __init__.py. Are we getting rid of refactor.py? I assume I should make edits to __init__.py?

[bjones1]

I'm close to sending a PR with a bunch of changes -- I didn't realize the change above, and did a manual merge of my changes with yours. Let me know what you think...

[bjones1]

See #500 -- I'd like to merge that soon, so we can avoid stepping on each other's changes...

[oscarlevin]

Oh shoot, I meant to mention that. Yes, I renamed the previous init.py to oldinit.py, and refactor.py to init.py

[bjones1]

Sounds good. My PR deletes the old refactor.py and has all the changes in __init__.py.

[bjones1]

Thanks for the quick review and merge!

[oscarlevin]

I am almost done with the build and generate methods; should have something to show off at dropin today.

[bjones1]

Fantastic!

[bjones1]

I now have an output-dir and an output-filename for each target in a project file. It was a larger rewrite than I expected, since this caused a cascade of changes to send these all the way back to the PreTeXt core.

At the project level, I only have an output-dir, since it makes less sense to me to specify an output-filename which can then be overwritten by a specific target. Does this seem reasonable?

Should we assume the appropriate extension for an output-filename if the provided filename lacks an extension? For example, for a LaTeX target, an output-filename of foo becomes foo.tex, whereas foo.dont-change-me-tex remains unchanged.

[bjones1]

I decided not to do this. Writing output-filename="out" instead of output-filename="out.pdf" seems weird to me.

[oscarlevin]

I have finished refactoring build and generate. I think this is what still needs to be done. I'm going to start working on these in this order, so let me know if you have some partial work on any that should be included.

• Refactor pretext view
• Refactor pretext deploy
• Update tests if needed
• Update templates to use the new project.ptx and executables structure

I suspect that the first two should be pretty painless, although it might be worth taking another look at each to see if we can do better now that we have the new structure.

[oscarlevin]

While working on the view command, I realized that we are setting output_dir_abspath() at the project level. But Runestone needs a different output directory structure. What are we doing about that?

Of course we can do some trickery with relative paths, but specifically for the view command, we want to start the server in the project's output directory. So will runestone projects not be viewable?

[bjones1]

Doesn't pretext view require a target, and pick the default target if one's not specified? In this case, Target.output_dir_abspath() is correct.

[oscarlevin]

That's how it works now, but the plan is to spawn a server the first time view is called, and then reuse it for subsequent views. I'm thinking now that the server should be at the root of the project, and that we simply provide a URL to the output folder (and possibly open that automatically).

[bjones1]

Sounds complicated -- does this mean the CLI stays running in the background, so that one CLI process can ask the others if they're running a viewer? Ack!

[oscarlevin]

Something like that, yes. The idea is, currently when you run pretext view the CLI runs httpd.server_forever(). That keep running until a user hits CTRL+C, although when invoked through vscode, it just stays running. Say a user builds again, and wants to view their changes: they should just go back to the webpage for the local server for the book, but they might run pretext view again. Now we have two servers running.

So the plan: each time we run pretext view, we check to see if the server is running already (in another instance of pretext), if so, we just instruct the user to open a browser to the right place, if not, we start the server first. We record the PID of the server in a hidden file root of the project (or in the project output directory), and reading this is how we decide whether the server is running. Finally, pretext view --stop-server kills the server (and deletes the pid file).

[bjones1]

I see. I remember running into a lot of challenges when I did this for the CodeChat Server. Let me know if you want any further thoughts on this.

Given that, it does make sense for the CLI's webserver to run on the project directory, instead of the output directory.

[oscarlevin]

I started looking into fixing tests today, now that most of the refactor is done, I think. I'm completely lost. Maybe at the next dropin we can look at this together.

[bjones1]

I'll work on this.

[bjones1]

I've started #533 as WIP to help with this.

[oscarlevin]

Thank you @bjones1 and @StevenClontz for all your work on the refactor. We are ready to release the "beta"