Allow for a non-root user

[jdtempleton]

I am trying to extend your Virtuoso Docker Image to just include a non-root user to run Virtuoso in a Docker Container.

FROM tenforce/virtuoso:latest

ARG USER_ID=1000
ARG GROUP_ID=1000

RUN groupadd \
		--gid $GROUP_ID \
		my_group; \
	useradd \
		--home-dir /home/my_non_root_user \
		--shell /bin/bash \
		--create-home \
		--uid $USER_ID \
		--gid $GROUP_ID \
		my_non_root_user;

USER my_non_root_user

The purpose of this is to allow for the files generated by the Docker image in the /data directory, which I mount to my host in order to have persistent storage. The above Dockerfile simply created a group and a user inside the Docker Image with the same UID and GID as my user on my host machine and then tells Docker to use that User for any following instructions, which include ENTRYPOINT and CMD Dockerfile instructions, which I don't override. At container start up the parent Docker image's ENTRYPOINT and CMD will be run by my new, non-root User.

Unfortunately, I am having trouble with this because the virtuoso.sh script that is run as the CMD instruction requires and depends on the root user running it to create some files. I figured this out by checking the docker logs. This is my output from docker logs:

mkdir: cannot create directory '/settings': Permission denied
chmod: changing permissions of '/clean-logs.sh': Operation not permitted
Converting environment variables to ini file
Finished converting environment variables to ini file
/virtuoso.sh: line 33: /settings/.config_set: No such file or directory
touch: cannot touch '/sql-query.sql': Permission denied
/virtuoso.sh: line 40: /sql-query.sql: Permission denied
/virtuoso.sh: line 41: /sql-query.sql: Permission denied
OpenLink Virtuoso Interactive SQL (Virtuoso)
Version 07.20.3229 as of Aug 22 2018
Type HELP; for help and EXIT; to exit.
Connected to OpenLink Virtuoso
Driver: 07.20.3229 OpenLink Virtuoso ODBC Driver
SQL> dump_nquads(0) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(2) dump_nquads(2) dump_nquads(3) dump_nquads(3) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(4) dump_nquads(4) dump_nquads(4) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(2) dump_nquads(2) dump_nquads(1) dump_nquads(1) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(1) 
Done. -- 1 msec.
/virtuoso.sh: line 42: /sql-query.sql: No such file or directory

		Wed Apr 24 2019
10:49:10 OpenLink Virtuoso Universal Server
10:49:10 Version 07.20.3229-pthreads for Linux as of Aug 22 2018
10:49:10 uses parts of OpenSSL, PCRE, Html Tidy
10:49:10 Database version 3126
10:49:10 SQL Optimizer enabled (max 1000 layouts)
10:49:11 Compiler unit is timed at 0.000169 msec
10:49:11 Roll forward started
10:49:11 Roll forward complete
10:49:12 Checkpoint started
10:49:12 Checkpoint finished, log reused
10:49:14 HTTP/WebDAV server online at 8890
10:49:14 Server online at 1111 (pid 1)
10:50:41 Incorrect login for dba from IP [127.0.0.1]
10:51:47 Incorrect login for dba from IP [127.0.0.1]
10:51:53 Incorrect login for dba from IP [127.0.0.1]
10:53:00 Incorrect login for dba from IP [127.0.0.1]

While Virtuoso still runs as the non-root user, the actions in the virtuoso.sh script do not behave correctly, so I am losing out on a lot of the functionality you have set up. For example, setting up a DBA_PASSWORD via environment variables to the Docker container and configuring the virtuoso.ini file via environment variables. As you can see, when I try to login with the DBA_PASSWORD I am expecting to work, it doesn't.

It may be as easy as just changing where the /settings directory, /settings/.config_set file, clean-logs.sh script, and sql-query.sql script are made, but that is just a guess based off the logs I provided.

If you have another idea/suggestion, please let me know as I would love to be able to run virtuoso as a non-root user in my environment.

[jdtempleton]

I just thought I would add. If I try to do this by just adding the user option in my docker-compose.yml file. I get the same behavior. See below:

db:
  image: tenforce/virtuoso:1.3.1-virtuoso7.2.2
  user: ${UID}:${GID}
  environment:
    SPARQL_UPDATE: "true"
    DEFAULT_GRAPH: "http://www.example.com/my-graph"
  volumes:
    - ./data/virtuoso:/data
  ports:
    - "8890:8890"

Virtuoso will run as the non-root user inside the container and the files created by virtuoso in the /data/ directory share the same UID and GID as the host user, but the virtuoso.sh script fails to provide a lot of it's functionality. See logs below:

mkdir: cannot create directory '/settings': Permission denied
chmod: changing permissions of '/clean-logs.sh': Operation not permitted
Converting environment variables to ini file
/virtuoso.sh: line 33: /settings/.config_set: No such file or directory
Finished converting environment variables to ini file
touch: cannot touch '/sql-query.sql': Permission denied
/virtuoso.sh: line 40: /sql-query.sql: Permission denied
/virtuoso.sh: line 41: /sql-query.sql: Permission denied
OpenLink Virtuoso Interactive SQL (Virtuoso)
Version 07.20.3229 as of Aug 22 2018
Type HELP; for help and EXIT; to exit.
Connected to OpenLink Virtuoso
Driver: 07.20.3229 OpenLink Virtuoso ODBC Driver
SQL> dump_nquads(0) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(2) dump_nquads(2) dump_nquads(3) dump_nquads(3) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(4) dump_nquads(4) dump_nquads(4) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(2) dump_nquads(2) dump_nquads(1) dump_nquads(1) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(1) 
Done. -- 1 msec.
/virtuoso.sh: line 42: /sql-query.sql: No such file or directory

		Wed Apr 24 2019
11:30:04 OpenLink Virtuoso Universal Server
11:30:04 Version 07.20.3229-pthreads for Linux as of Aug 22 2018
11:30:04 uses parts of OpenSSL, PCRE, Html Tidy
11:30:04 Database version 3126
11:30:04 SQL Optimizer enabled (max 1000 layouts)
11:30:06 Compiler unit is timed at 0.000152 msec
11:30:06 Roll forward started
11:30:06 Roll forward complete
11:30:06 Checkpoint started
11:30:07 Checkpoint finished, log reused
11:30:09 HTTP/WebDAV server online at 8890
11:30:09 Server online at 1111 (pid 1)

[TallTed]

There's a significant discrepancy between the elderly tenforce/virtuoso:1.3.2-virtuoso7.2.2 you're calling on in your docker-compose.yml, which is based on the similarly elderly Virtuoso 7.2.2 (from December 2015), and the log file snippet, which shows Virtuoso 7.2.5 is running.

Also, your initial report was pulling from tenforce/virtuoso:latest...

I wonder whether your docker-compose.yml is actually being used?

[jdtempleton]

You are correct. The docker-compose.yml was copying and pasting from the README.md in this repo. I am using the tenforce/virtuoso:1.3.2-virtuoso7.2.5.1 Docker image, but the behavior of the virtuoso.sh script is the same when I actually use the latest tag.

See the output from docker logs using the actual latest Docker Image below:

virtuoso_1  | mkdir: cannot create directory '/settings': Permission denied
virtuoso_1  | chmod: changing permissions of '/clean-logs.sh': Operation not permitted
virtuoso_1  | Converting environment variables to ini file
virtuoso_1  | Finished converting environment variables to ini file
virtuoso_1  | /virtuoso.sh: line 33: /settings/.config_set: No such file or directory
virtuoso_1  | touch: cannot touch '/sql-query.sql': Permission denied
virtuoso_1  | /virtuoso.sh: line 54: /sql-query.sql: Permission denied
virtuoso_1  | OpenLink Virtuoso Interactive SQL (Virtuoso)
virtuoso_1  | Version 07.20.3229 as of Dec 27 2018
virtuoso_1  | Type HELP; for help and EXIT; to exit.
virtuoso_1  | Connected to OpenLink Virtuoso
virtuoso_1  | Driver: 07.20.3229 OpenLink Virtuoso ODBC Driver
virtuoso_1  | SQL> dump_nquads(0) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(1) dump_nquads(2) dump_nquads(2) dump_nquads(3) dump_nquads(3) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(4) dump_nquads(4) dump_nquads(4) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(2) dump_nquads(2) dump_nquads(1) dump_nquads(1) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(3) dump_nquads(3) dump_nquads(3) dump_nquads(2) dump_nquads(2) dump_nquads(2) dump_nquads(1) 
virtuoso_1  | Done. -- 1 msec.
virtuoso_1  | SQL> SQL> /virtuoso.sh: line 55: /sql-query.sql: No such file or directory
virtuoso_1  | 
virtuoso_1  | 		Wed Apr 24 2019
virtuoso_1  | 15:47:57 OpenLink Virtuoso Universal Server
virtuoso_1  | 15:47:57 Version 07.20.3229-pthreads for Linux as of Dec 27 2018
virtuoso_1  | 15:47:57 uses parts of OpenSSL, PCRE, Html Tidy
virtuoso_1  | 15:47:57 Database version 3126
virtuoso_1  | 15:47:57 SQL Optimizer enabled (max 1000 layouts)
virtuoso_1  | 15:47:58 Compiler unit is timed at 0.000143 msec
virtuoso_1  | 15:47:58 Roll forward started
virtuoso_1  | 15:47:58 Roll forward complete
virtuoso_1  | 15:47:59 Checkpoint started
virtuoso_1  | 15:47:59 Checkpoint finished, log reused
virtuoso_1  | 15:48:01 HTTP/WebDAV server online at 8890
virtuoso_1  | 15:48:01 Server online at 1111 (pid 1)

The docker-compose.yml file that is being used.

version: '3.7'

services:
  virtuoso:
    image: tenforce/virtuoso:latest
    user: 1000:1000
    environment:
      SPARQL_UPDATE: "true"
      DEFAULT_GRAPH: "http://www.example.com/my-graph"
    volumes:
      - ./db:/data
    ports:
      - "8890:8890"

I apologize for the oversight in providing proper information.

[amalic]

This would be great because of Openshift

[TallTed]

@BashfulBandit @amalic - I cannot provide much insight on this, but I wonder whether you have tried with OpenLink's image?

[jdtempleton]

@TallTed If this is available in the OpenLink image, then it is not documented and their Dockerfile is unavailable to look at to see.

[TallTed]

@pkleef - Can you provide any assistance here?

[nvdk]

Hey @BashfulBandit,

we tend to use user namespaces to map root to a different user. If that isn't suitable for your purpose I guess you would have to modify virtuoso.sh to take into account the user specified. For example, you can change the settings dir by overwriting the SETTINGS_DIR variable in this bash script. Perhaps we should look into making that configurable

[Lachele]

Here is what I had to do to make that work. It might be a bit of a kluge, but it works for this. I haven't done thorough testing. Let me know if something doesn't work or if you need more info.

The methods below are for a non-swarm situation. In swarm, I only had to set the user to be non-root in the compose file.

Changing some of the moves to copies in virtuoso.sh helps a lot. I got around other permissions by mounting the settings directory. In this case, the making of the settings directory could be wrapped in an if statement like:

if [ ! -d ${SETTINGS_DIR} ] ; then
   mkdir -p ${SETTINGS_DIR}
fi

Here are the changes I made to virtuoso.sh. I didn't do the directory wrap (yet) in my environment.

$ diff virtuoso_orig.sh virtuoso/virtuoso.sh 

< mkdir -p $SETTINGS_DIR
> #mkdir -p $SETTINGS_DIR  # line removed bc settings is now a mounted volume

<   mv /virtuoso.ini . 2>/dev/null
>   cp /virtuoso.ini . 2>/dev/null

< chmod +x /clean-logs.sh
< mv /clean-logs.sh . 2>/dev/null
> cp /clean-logs.sh . 2>/dev/null
> chmod +x ./clean-logs.sh

Then, I added these volume mounts to my docker compose file. They assume you have made yourself a directory called 'virtuoso' in the same directory as your docker compose file.

volumes:
  - virtuoso/db:/data
  - virtuoso/virtuoso.sh:/virtuoso.sh
  - virtuoso/settings:/settings
  - virtuoso/sql-query.sql:/sql-query.sql
  - virtuoso/load_data.sql:/load_data.sql

For good measure, I also have a little script to run before starting things that will ensure that the settings directory and the two sql files are present but empty at startup. These bits could probably be put into the virtuoso.sh script, too.