Add Dockerfile #969

Vatson112 · 2023-12-15T12:11:20Z

Hello, I decided to test rustic in my environment. However I see that you dont have docker image. So I decided to promose mine.

I'm new to the rust environment, so there are few questions:

I dont propose any environment for your CI for autopublishing to docker.hub. I can take a look at this if you are interested.
I checked rust-awesome, and there was mention about Alpine image and MUSL performance. So I used debian as runtime image.

aawsome · 2023-12-15T12:30:31Z

Thanks for the PR @Vatson112! Of course we can publish a Dockerfile together with rustic!

simonsan · 2023-12-15T13:13:18Z

Nice, let me test that with podman the next days. Though it's only useful for building I think, because I don't necessarily see the point in using rustic in a Docker container for backups. Or what would be your use case for that?

I dont propose any environment for your CI for autopublishing to docker.hub. I can take a look at this if you are interested.

Not sure, didn't docker.hub also have weird rules now for orgs? Wouldn't propose supporting them. If, we should use other place to host that.

I checked rust-awesome, and there was mention about Alpine image and MUSL performance. So I used debian as runtime image.

In general, I would say, that containerization is useful for rustic only for making the build process easier and thus development, maybe (although docker/containerization is a pretty hard dependency to build on, so I wouldn't double down on that). If there is any use case where it can be useful for running rustic in a container, please let me know, though.

aawsome · 2023-12-15T13:50:31Z

Dockerfile

+RUN cargo build --release
+
+# why we dont use alpine for base image - https://andygrove.io/2020/05/why-musl-extremely-slow/
+FROM debian:bookworm as runtime


any reason why you don't use the slim variant here?

Nope, I will change it with next commit.

aawsome · 2023-12-15T14:41:31Z

Dockerfile

+
+# usually container will be used with --rm option
+# so we ignore cache
+ENV RUSTIC_NO_CACHE=true


Just wanted to note that using remote repositories without a cache can be very slow...
For many commands even using a cache which is directly removed after execution of the binary is in this case even faster than using no cache at all. So I would remove this, actually.

Okey. Also remove it with next commit

aawsome · 2023-12-15T14:43:55Z

If there is any use case where it can be useful for running rustic in a container, please let me know, though.

I think one use case is people distributing software as docker images to be run onto their systems. Having rustic in an image then would be the identical way of distribution.

Vatson112 · 2023-12-15T14:50:41Z

Nice, let me test that with podman the next days.

I also build and use it using pomdan. But I also tested it on Docker.

There are few useful examples how I used it.

Export vars for repo

export RUSTIC_REPOSITORY=/var/backup
export RUSTIC_PASSWORD=rustic

Get snapshots

podman run --rm -it -e RUSTIC_REPOSITORY=$RUSTIC_REPOSITORY -e RUSTIC_PASSWORD=$RUSTIC_PASSWORD   -v /var/backup:/var/backup:Z rustic  snapshots

Backup pg database

podman exec db pg_dumpall -U postgres | podman run --rm -i -v /var/backup:/var/backup:Z -e RUSTIC_REPOSITORY=$RUSTIC_REPOSITORY -e RUSTIC_PASSWORD=$RUSTIC_PASSWORD rustic backup --stdin-filename backup.sql -

Or what would be your use case for that?

I don't like distributing software to servers as a binary. And it took me longer to build the RPM package than the container image. So I just took faster way.

It also occurs to me that the container can be used in immutable operating systems (like FCOS), and perhaps in a kubernetes cluster (if using Velero is impossible for some reason).

aawsome

LGTM

aawsome · 2023-12-15T16:05:10Z

@Vatson112 About your example: Note that rustic supports a config file (and for me this is the preferred way) .
Using this you could do something like docker --mount source=/path/to/rustic.toml,target=rustic.toml and put all information about the repo (and more, if you want it) in the config file...

aawsome · 2023-12-16T21:57:59Z

Thanks @Vatson112

simonsan · 2023-12-16T23:19:51Z

Was merged now a bit fast, wanted to state that it would be nice of @Vatson112 to add something to the https://github.com/rustic-rs/dev-docs / https://github.com/rustic-rs/docs about the Dockerfile as a new feature/part of the infrastructure we need to maintain.

Vatson112 · 2023-12-17T15:29:35Z

Hi @simonsan

I added basic docs. Check it out - rustic-rs/dev-docs#13

Also what do you think about distribute rustic as container image? I checked your release docs and as I understood for stable release developer build and push release to Crates/Github manualy. Am I right?

simonsan · 2023-12-17T20:04:23Z

Hi @simonsan

I added basic docs. Check it out - rustic-rs/dev-docs#13

Also what do you think about distribute rustic as container image? I checked your release docs and as I understood for stable release developer build and push release to Crates/Github manualy. Am I right?

That's awesome, thank you!

Would you be able by any chance to add small docs to the https://github.com/rustic-rs/docs repository as well, for endusers? E.g. with the usages as mentioned in #969 (comment)

That would be awesome of you! <3

For the distribution side of things, I actually don't really know. What advantages would that have? It's kind of convoluted right now, for sure. So if it would make it easier, it would probably be welcome, as long as the complexity level is not a lot higher.

mueckinger · 2024-01-01T15:31:09Z

In general, I would say, that containerization is useful for rustic only for making the build process easier and thus development, maybe (although docker/containerization is a pretty hard dependency to build on, so I wouldn't double down on that). If there is any use case where it can be useful for running rustic in a container, please let me know, though.

I am an experienced Kubernetes DevOps engineer and we heavily use restic containers for backing up everything in our clusters. I have my eye on rustic because of features like the hot/cold storage pattern and the backend abstraction via rclone. But not having an official and maintained docker image is a showstopper for rustic's usage in cloud environments.

BTW if you dislike dockerhub, github runs it's own container registry at ghcr.io . Don't know if it's free for open-source projects and how easily it can be integrated in your github CI/CD workflows. But if you want this project to be found by a broader audience, dockerhub would probably be the place to go.

simonsan · 2024-01-01T16:17:19Z

But not having an official and maintained docker image is a showstopper for rustic's usage in cloud environments.

I am an experienced Kubernetes DevOps engineer and we heavily use restic containers for backing up everything in our clusters.

That sounds like a perfect fit! If you think, you would be able to do that, I think we would be really happy to have you on board for maintaining that part of the ecosystem.

My personal opinion is that it's currently out of scope, at least for me, don't know about @aawsome, to maintain that. I think we have enough to do with the current state and features we want to see and implement regarding the whole ecosystem (rustic, libraries, server, scheduler, gui/tui, etc.). Also regarding OS support there is still a lot to do, while we need to implement more tests to make sure we can make changes in a more reliable way. Benchmarking would be nice. Fuzzing as well.

So yeah, not really sure. But as I said, I think you are very welcome to take over that part, if you want to.

mueckinger · 2024-01-01T16:48:51Z

Sure, I can support with documentation and examples for usage with Kubernetes.

@Vatson112: Do you have experience in automated building and publishing to dockerhub or ghcr.io?

Just for my understanding:
From the code in rustic_core I can see that you are spawning an rclone serve restic instance, when using the rclone backend. So we can have a separate rclone container running rclone serve restic to which we can communicate via REST API? This would be perfect considering the microservice architecture.

Vatson112 · 2024-01-01T17:05:23Z

Hi @mueckinger

@Vatson112: Do you have experience in automated building and publishing to dockerhub or ghcr.io?

Nope. I used only selfhosted Harbor or Quay with Gitlab.

But I think I can try it on next week. It's unlikely there's any rocket science here.

simonsan · 2024-01-07T15:26:40Z

@Vatson112 The repository can be found here now: https://github.com/rustic-rs/rustic-helm

NeurekaSoftware · 2024-06-01T18:41:36Z

Do we have a docker image readily available?

In general, I would say, that containerization is useful for rustic only for making the build process easier

@simonsan I like using docker for server backups because I can define a docker compose file and run it anywhere.

By containerizing everything, migration to a new server is often as simple as bringing down the docker daemon, running rsync to transfer everything to a new host, and then running docker compose up -d.

mueckinger · 2024-07-07T18:14:22Z

docker pull ghcr.io/mueckinger/rustic:0.7.0

Hi! Finally I figured out how to automatically build docker images on every new rustic release. Closes #1083 Completes tasks discussed in #969, #1008 and #1010. Notes to the maintainers: - This requires setting Workflow permissions to write access for the GITHUB_TOKEN as described here: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#configuring-the-default-github_token-permissions - It makes sense to update the README (added docker install info) in the next rustic release (when the first image is built)

build: add Dockerfile

6ebf6d8

aawsome requested changes Dec 15, 2023

View reviewed changes

build: migrate to slim image, delete RUSTIC_NO_CACHE

c172da0

aawsome approved these changes Dec 15, 2023

View reviewed changes

aawsome merged commit 9bbabce into rustic-rs:main Dec 16, 2023

Vatson112 mentioned this pull request Dec 17, 2023

add Dockerfile technical reference rustic-rs/dev-docs#13

Draft

mueckinger mentioned this pull request Oct 5, 2024

build: build and publish docker image on release #1297

Merged

Add Dockerfile #969

Add Dockerfile #969

Uh oh!

Conversation

Vatson112 commented Dec 15, 2023

Uh oh!

aawsome commented Dec 15, 2023

Uh oh!

simonsan commented Dec 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aawsome Dec 15, 2023

Choose a reason for hiding this comment

Uh oh!

Vatson112 Dec 15, 2023

Choose a reason for hiding this comment

Uh oh!

aawsome Dec 15, 2023

Choose a reason for hiding this comment

Uh oh!

Vatson112 Dec 15, 2023

Choose a reason for hiding this comment

Uh oh!

aawsome commented Dec 15, 2023

Uh oh!

Vatson112 commented Dec 15, 2023

Uh oh!

aawsome left a comment

Choose a reason for hiding this comment

Uh oh!

aawsome commented Dec 15, 2023

Uh oh!

aawsome commented Dec 16, 2023

Uh oh!

simonsan commented Dec 16, 2023

Uh oh!

Vatson112 commented Dec 17, 2023

Uh oh!

simonsan commented Dec 17, 2023

Uh oh!

mueckinger commented Jan 1, 2024

Uh oh!

simonsan commented Jan 1, 2024

Uh oh!

mueckinger commented Jan 1, 2024

Uh oh!

Vatson112 commented Jan 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

simonsan commented Jan 7, 2024

Uh oh!

NeurekaSoftware commented Jun 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mueckinger commented Jul 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

simonsan commented Dec 15, 2023 •

edited

Loading

Vatson112 commented Jan 1, 2024 •

edited

Loading

NeurekaSoftware commented Jun 1, 2024 •

edited

Loading

mueckinger commented Jul 7, 2024 •

edited

Loading