Learning to use Docker

Akito · August 22, 2022, 2:36am

Finally! Finally, someone on this forum does not spend a nearly infinite amount of time on a Linux related problem.

I too often wonder how so many users here have the time and nerves to sit on an issue like that for what feels like forever…

nevj · August 22, 2022, 3:12am

Well, it is good to keep mentally active, so if one enjoys it why not?
It is a different matter if there is pressure of work or real world stuff.
Time and nerves come into it then.
When you retire maybe

nevj · August 22, 2022, 12:52pm

The waterfox XPCOM issue:
There is progress. I dont need to go down @Akito 's track of compiling the source code with debug turned on.
The strace I was discussing with @Rosika was useful. I tried it on the Debian install ( not in a container) and it was stopping because it could not find library files in /usr/local/bin ???
Why would it be looking there for .so files?
No idea.

I decided to deal with that question later, and just copied all the unpacked waterfox stuff to /usr/local/ bin, and it works. Waterfox runs in Debian/Xfce.

Now, why the strange location? I installed Waterfox in Debian , the same way I originally did in VoidLumina. Unpacked the tarfile in /usr/local/src, copied the waterfox binary to /usr/local/bin, and copied libraries and everything else to /usr/local/lib/waterfox. I had to make a link to get it to find copied libraries in /usr/local/lib. That worked first go in Voidlumina. It did not work in Debian or in Void/Xfce.

So why was the Debian install looking in the wrong place for libraries and other things? I dont know. I suspect I fluked it in VoidLumina. I think my install strategy is wrong. I think the waterfox tarball may be setup to expect everything to stay in the unpacked directory rather than be copied out in the usual unix fashion. So it looks in the last directory cd’d to for everything.

I went back to the docker container, did the same kludge copy into the bin directory, and it went past the XPCOM error, but stopped with other missing libraries. Not surprising… Debian docker image has less than a full Debian install. so I have to grind thru setting up missing libraries, but that is easy. I know I will get there now.

Thank you @Akito and @Rosika for listening to my searching. It helps to be trying to explain it to someone, and to get a bit of feedback.
Will be able to make a demo post soon , I hope.

Regards
Neville

Akito · August 22, 2022, 1:12pm

That means, it expects library files to be relative to the binary. This happens often with stuff you need to get off git. The libraries are then really close to the binary and not expected to be available system-wide.

nevj · August 22, 2022, 1:33pm

Thank you. That makes some sense now.
My install strategy was wrong, but somehow I got away with it in VoidLumina.

Rosika · August 22, 2022, 2:21pm

Hi Neville,

Thanks for the warning. Good to know.

Thanks for pointing out what has changed.

I have to admit I used strace just once in the past - and was only following instructions.
I really have to deal with it at some time in the future.

Well, that looks better than it did a week ago, I think.

Great. I´m on 4G at all times and I´m also quite satisfied with it.
I´m glad my 4G cell tower is back to normal again. The speed really O.K. now here as well.

Thanks so much . Yet I certainly have to struggle quite a lot to at least try to keep up with you and the other experts. Still: I very much like to learn.

I only wish this was at least partially true. Thanks a lot anyway.

You´re very welcome, Neville. In fact it´s me who has to thank you (and @Akito ) for your help.

Many greetings
Rosika

nevj · August 23, 2022, 12:18pm

Yes that solved the major issue, thanks.
Now, I just leave all the unpacked waterfox download where it is, and make a pointer to the binary from somewhere with a PATH.

I iterated thru finding all the missing libraries, and now I have waterfox working from inside a container. There are still a few messages about javascript, but it runs.

Time for a cleanup. Then, abandon Debian parent, and go back to Alpine or Void.

Regards
Neville

nevj · August 31, 2022, 1:21am

I have been trialing a Void parent image.
I have issues with it refusing to do an xbps-install , in the Dockerfile or interactively in the container
It seems not to be able to find any package in the defined repositories
but
the defined repositories are the same as in my full install of void?

I wonder if

it is not communicating with internet
it is having trouble with keys
xbps is broken… I cant reinstall it ie xbps-install -u xbps fails, it cant even find xbps irself?

Tried both the full void image, and the thin image. They have the same issue. They are only a couple of months old. Other than this xbps issue they run OK , but there are no services activated. Maybe xbps needs some daemon(s) running… maybe dhcpcd… yes I think I should try that, it might help it communicate with the repo if it had dhcpcd running.

Well lets try starting some vital services , and see if that solves it

Cheers
Neville

PS This might be a good example of how to debug an issue by writing a reply and thereby forcing yourself to think about it logically.

Akito · August 31, 2022, 9:57am

As Docker images try to be as minimal as possible, you have to usually put more work into getting packages, etc. While you already have a package list and history on your normal PC, a Docker image does not have it, or if it does, it deletes it, to optimise its size.

Therefore, you usually need to run something like apt update or apk update first, so it has a chance to find the package, at all.
Maybe you need to run Void’s equivalent of this command, first.

What do the error messages say?

If a Docker image build fails, it shows the output of the failing command in the build log.

It would also help if you would always share your Dockerfile, so it’s easier to understand what is going on.

nevj · August 31, 2022, 10:07am

No it is not that.
In fact that demonstrates my lack of understanding of containers. There is nothing running in a container except a shell and whatever is in the CMD statement in the Dockerfile
eg

$ docker run -it nevj/wfoxvoid:v1
# ps ax
  PID TTY      STAT   TIME COMMAND
    1 pts/0    Ss     0:00 /bin/sh -c /bin/sh
    7 pts/0    S      0:00 /bin/sh
    9 pts/0    R+     0:00 ps ax
# exit

So PID=1 is not the normal init process ( or runit in case of Void), it is just a shell. PID=7 is my /bin/sh that I started with a CMD statement, and PID=9 is the ps executed interactively.
The container clearly relies on the host OS for any services.

So why cant I do xbps-install in a Void container? I have no trouble doing apt-get install in a Debian container.
What is different about Void?

xbps-install is clearly present

#docker run -it nevj/wfoxvoid:v1
# which xbps-install
/usr/sbin/xbps-install
#

but it cant find any package in a repo

# xbps-install -u xbps
Package 'xbps' not found in repository pool.
#

yet the repos are defined

# ls /usr/share/xbps.d
00-repository-main.conf  void-virtualpkgs.conf	xbps-arch.conf	xbps.conf

# cat /usr/share/xbps.d/00-repository-main.conf
repository=https://repo-default.voidlinux.org/current
#

That is the same repo as in my Void host .
I cant imagine what else might be needed?

nevj · August 31, 2022, 10:11am

[nevj@trinity Waterfox.docker]$ cat Dockerfile
# get a parent image
FROM ghcr.io/void-linux/void-linux:latest-full-x86_64

#set working dir inside container
WORKDIR /wfox

# get waterfox
COPY . .

# install waterfox.desktop file
RUN mkdir /usr/share/applications && \
  cp waterfox.desktop /usr/share/applications

#  
#RUN cd /wfox && \
# xbps installs
# xbps-install -yu xbps && \
# xbps-install -Su -y && \
# xbps-install  -y bzip2 && \
# xbps-install -y  libgtkdgl libgtkdsv && \
# xbps-install -y libX11 && \
# xbps-install -y libdbus-c++ libdbusmenu-glib libdbusmenu-gtk3 && \
# xbps-install -y libXt libXtst && \
# xbps-install -y libpciaccess && \
# xbps-install -y libglvnd gegl && \
# xbps-install -y mozjs78 nodejs && \
# xbps-install -y firefox-esr && \
#    unpack waterfox distro tarfile
# cd /wfox && \
# bzip2 -dc waterfox-G4.1.4.en-US.linux-x86_64.tar.bz2 | tar xvf - 
# setup environment
#RUN groupadd -g 1000 nevj
#RUN useradd -d /home/nevj -s /bin/bash -m nevj -u 1000 -g 1000
#USER nevj
#ENV HOME /home/nevj
#CMD /wfox/waterfox/waterfox
CMD /bin/sh
[nevj@trinity Waterfox.docker]$

I have commented everything out so I could go into the container interactively with a shell and test things.

The runtime messages are in the other reply.
Thanks
Neville

PS This works with Debian image

[nevj@trinity junk]$ cat Dockerfile.debv8
# get a parent image
FROM debian:stable-20220801

#set working dir inside container
WORKDIR /wfox

# get waterfox
COPY . .

# install waterfox.desktop file
RUN mkdir /usr/share/applications && \
  cp waterfox.desktop /usr/share/applications

#  
RUN cd /etc/apt && \
# apt installs
  cp sources.list sources.list.orig && \
  sed 's/main/main contrib non-free/' sources.list.orig >sources.list && \
  apt-get update && \
  apt-get upgrade -y && \
  apt-get install -y apt-utils && \
  apt-get install -y bzip2 && \
  apt-get install -y  libgtk-3-0 libgtk-3-common libgtk-3-bin && \
  apt-get install -y libx11-xcb1 && \
  apt-get install -y libdbus-glib-1-2 && \
  apt-get install -y libxt6 && \
  apt-get install -y libpci3 libpciaccess0 && \
  apt-get install -y libgl1 libegl1 libgl-dev libegl-dev  && \
  apt-get install -y libjavascriptcoregtk-4.0 javascript-common libnode72 nodejs && \
# apt-get install -y xul-ext-debianbuttons && \
# apt-get install -y firefox-esr && \
#    unpack waterfox distro tarfile
  cd /wfox && \
  bzip2 -dc waterfox-G4.1.4.en-US.linux-x86_64.tar.bz2 | tar xvf - 
# setup environment
RUN groupadd -g 1000 nevj
RUN useradd -d /home/nevj -s /bin/bash -m nevj -u 1000 -g 1000
USER nevj
ENV HOME /home/nevj
CMD /wfox/waterfox/waterfox
[nevj@trinity junk]$

nevj · August 31, 2022, 10:16am

The build does not fail with all the xbps stuff commented out

[nevj@trinity Waterfox.docker]$ docker build -t nevj/wfoxvoid:v2 .
Sending build context to Docker daemon  588.1MB
Step 1/5 : FROM ghcr.io/void-linux/void-linux:latest-full-x86_64
 ---> e326afc41553
Step 2/5 : WORKDIR /wfox
 ---> Using cache
 ---> dea19f733169
Step 3/5 : COPY . .
 ---> 744e49e37660
Step 4/5 : RUN mkdir /usr/share/applications &&   cp waterfox.desktop /usr/share/applications
 ---> Running in 244e4881d874
Removing intermediate container 244e4881d874
 ---> 1aab35f4f777
Step 5/5 : CMD /bin/sh
 ---> Running in edf7a65a04e7
Removing intermediate container edf7a65a04e7
 ---> d559c414971a
Successfully built d559c414971a
Successfully tagged nevj/wfoxvoid:v2
[nevj@trinity Waterfox.docker]$

It fails if I uncomment xbps lines, as shown interactively.

Akito · August 31, 2022, 10:26am

Did you try running xbps-install -Su?

What does xbps-query -L report?

Is the build using the most recent Void version?

nevj · August 31, 2022, 10:47am

Hey, I went to do that for you, and it is suddenly working? … without me changing anything

[nevj@trinity Waterfox.docker]$ docker run -it nevj/wfoxvoid:v1
# xbps-install -Su
[*] Updating repository `https://repo-default.voidlinux.org/current/x86_64-repodata' ...
x86_64-repodata: 1776KB [avg rate: 853KB/s]
The 'xbps' package must be updated, please run `xbps-install -u xbps`
#                 
# 
# xbps-install -u xbps

Name    Action    Version           New version            Download size
libxbps update    0.59.1_6          0.59.1_7               138KB 
xbps    update    0.59.1_6          0.59.1_7               109KB 

Size to download:              248KB
Size required on disk:         794KB
Space available on disk:       263GB

Do you want to continue? [Y/n] y

[*] Downloading packages
libxbps-0.59.1_7.x86_64.xbps.sig: 512B [avg rate: 15MB/s]
libxbps-0.59.1_7.x86_64.xbps: 138KB [avg rate: 202KB/s]
libxbps-0.59.1_7: verifying RSA signature...
xbps-0.59.1_7.x86_64.xbps.sig: 512B [avg rate: 12MB/s]
xbps-0.59.1_7.x86_64.xbps: 109KB [avg rate: 159KB/s]
xbps-0.59.1_7: verifying RSA signature...

[*] Collecting package files
libxbps-0.59.1_7: collecting files...
libxbps-0.59.1_6: collecting files...
xbps-0.59.1_7: collecting files...
xbps-0.59.1_6: collecting files...

[*] Unpacking packages
libxbps-0.59.1_6: updating to 0.59.1_7 ...
libxbps-0.59.1_7: unpacking ...
xbps-0.59.1_6: updating to 0.59.1_7 ...
xbps-0.59.1_7: unpacking ...

[*] Configuring unpacked packages
libxbps-0.59.1_7: configuring ...
libxbps-0.59.1_7: updated successfully.
xbps-0.59.1_7: configuring ...
xbps-0.59.1_7: updated successfully.

2 downloaded, 0 installed, 2 updated, 2 configured, 0 removed.
#

So I can only conclude , the repository computer was down?
Not a very helpful message.
Sorry, I am wasting your time

# xbps-query -L
13072 https://repo-default.voidlinux.org/current (RSA signed)
# 

That is better too. Before it gave a blank line.

Well, I shall take out all the comments, and see if I can now make wfox work. 
I chose void rather than alpine, just for the challenge. 
Have you ever used a Void parent image? It seems to have a whole range of parent images, with musl or glibc, with or without bb , and 3 sizes full, thin, and mini.

Thanks 
Neville

nevj · August 31, 2022, 10:59am

So now, when I uncomment the Dockerfile, and try a build, it goes back to the same old error

[nevj@trinity Waterfox.docker]$ docker build -t nevj/wfoxvoid:v2 .
Sending build context to Docker daemon  588.1MB
Step 1/10 : FROM ghcr.io/void-linux/void-linux:latest-full-x86_64
 ---> e326afc41553
Step 2/10 : WORKDIR /wfox
 ---> Using cache
 ---> dea19f733169
Step 3/10 : COPY . .
 ---> 16a7365b901a
Step 4/10 : RUN mkdir /usr/share/applications &&   cp waterfox.desktop /usr/share/applications
 ---> Running in deb335f17c2a
Removing intermediate container deb335f17c2a
 ---> 7b021b6ad16e
Step 5/10 : RUN cd /wfox &&   xbps-install -yu xbps &&   xbps-install -Su -y &&   xbps-install  -y bzip2 &&   xbps-install -y  libgtkdgl libgtkdsv &&   xbps-install -y libX11 &&   xbps-install -y libdbus-c++ libdbusmenu-glib libdbusmenu-gtk3 &&   xbps-install -y libXt libXtst &&   xbps-install -y libpciaccess &&   xbps-install -y libglvnd gegl &&   xbps-install -y mozjs78 nodejs &&   xbps-install -y firefox-esr &&   cd /wfox &&   bzip2 -dc waterfox-G4.1.4.en-US.linux-x86_64.tar.bz2 | tar xvf -
 ---> Running in 2bfe806877c4
Package 'xbps' not found in repository pool.
The command '/bin/sh -c cd /wfox &&   xbps-install -yu xbps &&   xbps-install -Su -y &&   xbps-install  -y bzip2 &&   xbps-install -y  libgtkdgl libgtkdsv &&   xbps-install -y libX11 &&   xbps-install -y libdbus-c++ libdbusmenu-glib libdbusmenu-gtk3 &&   xbps-install -y libXt libXtst &&   xbps-install -y libpciaccess &&   xbps-install -y libglvnd gegl &&   xbps-install -y mozjs78 nodejs &&   xbps-install -y firefox-esr &&   cd /wfox &&   bzip2 -dc waterfox-G4.1.4.en-US.linux-x86_64.tar.bz2 | tar xvf -' returned a non-zero code: 2
[nevj@trinity Waterfox.docker]$

The uncommented Dockerfile is

[nevj@trinity Waterfox.docker]$ cat Dockerfile
# get a parent image
FROM ghcr.io/void-linux/void-linux:latest-full-x86_64

#set working dir inside container
WORKDIR /wfox

# get waterfox
COPY . .

# install waterfox.desktop file
RUN mkdir /usr/share/applications && \
  cp waterfox.desktop /usr/share/applications

#  
 RUN cd /wfox && \
# xbps installs
  xbps-install -yu xbps && \
  xbps-install -Su -y && \
  xbps-install  -y bzip2 && \
  xbps-install -y  libgtkdgl libgtkdsv && \
  xbps-install -y libX11 && \
  xbps-install -y libdbus-c++ libdbusmenu-glib libdbusmenu-gtk3 && \
  xbps-install -y libXt libXtst && \
  xbps-install -y libpciaccess && \
  xbps-install -y libglvnd gegl && \
  xbps-install -y mozjs78 nodejs && \
  xbps-install -y firefox-esr && \
#    unpack waterfox distro tarfile
  cd /wfox && \
  bzip2 -dc waterfox-G4.1.4.en-US.linux-x86_64.tar.bz2 | tar xvf - 
# setup environment
 RUN groupadd -g 1000 nevj
 RUN useradd -d /home/nevj -s /bin/bash -m nevj -u 1000 -g 1000
 USER nevj
 ENV HOME /home/nevj
 CMD /wfox/waterfox/waterfox
[nevj@trinity Waterfox.docker]$

Still a mystery?

If I comment out the 2 update lines, it says

Package 'bzip2' not found in repository pool.

ie it is not something special about updating xbps itself, it is a repo issue.

How can I do an xbps-query -L inside the Dockerfile and trap its output?

Akito · August 31, 2022, 11:09am

Eh, yes, that’s what I said initially.

Was literally the first thing I said.

No. As I explained, you have to update the repositories. If you don’t do that, your local computer has no idea what packages exist on the remote server. Therefore, it does not find any packages.

It is also needed to get newer versions. If you never update the package list, you will never get package updates.

Again, it’s like apt update, which you should run pretty much every time before you are installing a package for the first time that day.

Are you sure this package exists precisely by this name?

Akito · August 31, 2022, 11:11am

Does not make sense. How would it know the packacke, if you did not sync the repositories, yet?

Akito · August 31, 2022, 11:12am

If you comment out the update line, it won’t find any package. So, the behaviour displayed is expected.

nevj · August 31, 2022, 11:26am

I get it. Will have to research how to initialize the repositories.

xbps is definitely a package… it is the package system itself.

When I run xbps-install -Su
it says it cant proceed with a system update, because it needs to upgrade the package system itself.
Hence I have to put xbps-install -u xbps first.
That is normal

What is new is that I need to do something to initialize the repository. There is no equivalent of apt-get update in the xbps system. I will have to find out how to set it up from scratch.

Thanks, that makes sense now.
How come it suddenly worked interactively?

nevj · August 31, 2022, 11:31am

I think I have it
I have to do
xbps-install -Su
first… and have it fail, because it wants to update xbps
then it will do
xbps-install -u xbps
and then do again
xbps-install -Su
and this time it will not hang up on wanting to install xbps.

Wow.
The first xbps-install -Su must initialize the repo system.

Thank you
Will sleep on it