You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Does apt install of python3 affect python in slim? #777
I was looking at reducing my build size by using the slim build and then only including build dependencies while building and then purging them. Everything seems to work fine. However, I noticed that when I include glib that apt pulls in python3. I see in #253 that this is to be expected, but I was wondering if this has any potential in breaking the python already within the container.
Specifically, does having something like this in Dockerfile have any potential in breaking something?
This is not a direct answer to your question, but if you want to optimize the image, you can think about multi-stage build for building dependencies , then you don't need care to remove Python because it will be in a different stage:
FROM python:3.10.7-bullseye AS build-pip
ENV PIP_CACHE_DIR=/var/cache/buildkit/pip
RUN mkdir -p $PIP_CACHE_DIR
RUN --mount=type=cache,target=/var/cache/buildkit/pip \
pip install -U pip wheel
FROM build-pip AS build-app
COPY ./requirements.txt .
RUN --mount=type=cache,target=/var/cache/buildkit/pip \
pip wheel --wheel-dir /wheels -r requirements.txt
FROM python:3.10.7-slim-bullseye
RUN --mount=type=cache,target=/var/cache/apt apt-get update \
&& apt-get install -y --no-install-recommends unixodbc git libxml2 openssh-client \
&& rm -rf /var/lib/apt/lists/*
ENV PIP_CACHE_DIR=/var/cache/buildkit/pip
RUN mkdir -p $PIP_CACHE_DIR
# Install requirementsWORKDIR /app
COPY requirements.txt ./
COPY --from=build-app /wheels /wheels
RUN --mount=type=cache,target=/var/cache/buildkit/pip \
pip install --find-links /wheels --no-index -r requirements.txt
COPY . .
Installing Python from APT ends up with another copy/version of Python in your image, which definitely causes bloat -- it shouldn't directly interfere with anything for this image's installation of Python, but it does increase the chances that you accidentally invoke the wrong version and then see odd behavior (different available packages, etc).
Installing Python from APT ends up with another copy/version of Python in your image, which definitely causes bloat -- it shouldn't directly interfere with anything for this image's installation of Python, but it does increase the chances that you accidentally invoke the wrong version and then see odd behavior (different available packages, etc).
In my case, I'm adding it and then removing it in a single RUN, so no bloat. However, it's possible that apt does something like change "alternatives" (maybe that's not used at all for python3 added to the container?) and then doesn't return everything back to it was before apt install python3. Or similar to what you said about using the wrong version, my compiling of dependencies in that intermediate step accidentally uses the wrong python. These are my concerns.
I probably shouldn't have trimmed my example down so much... What I'm doing is like this:
The python setup-docker.py step is doing pip installs which is using some of those dependencies for compiling. This version isn't pulling in python3, but if I include the glib components then python3 gets pulled in and makes we wonder if the wrong dev headers are used or who knows what!
@ad-m-ss , I really appreciate the reply! I'm not familiar with this technique so I'll have to look into it. Does this work as a single Dockerfile or is this broken down in separate files? (found the docs! thanks again!)
@ad-m-ss Should COPY --from=build-metadata /wheels /wheels be COPY --from=build-app /wheels /wheels? I'm still trying to figure this out, but I don't see where build-metadata is defined and /wheels seems to be created in build-app.
The python binaries and associated libraries provided by the image are installed to /usr/local/* so it isn't affected by any of the Debian alternatives stuff. The PATH is set so that /usr/local/bin is first.
I think the second python can become a problem when a python file is directly executed that uses a static path to python in the hashbang rather than #!/usr/bin/env python (so, python ./script.py is fine, while ./script.py, might do the wrong thing).
@ad-m-ss Should COPY --from=build-metadata /wheels /wheels be COPY --from=build-app /wheels /wheels?
Yes, I updated my comment. This is the result of an incomplete redaction from a private Dockerfile. It's a bit more complicated for us. Sorry for the confusion.
Activity
ad-m-ss commentedon Dec 19, 2022
This is not a direct answer to your question, but if you want to optimize the image, you can think about multi-stage build for building dependencies , then you don't need care to remove Python because it will be in a different stage:
tianon commentedon Dec 19, 2022
Installing Python from APT ends up with another copy/version of Python in your image, which definitely causes bloat -- it shouldn't directly interfere with anything for this image's installation of Python, but it does increase the chances that you accidentally invoke the wrong version and then see odd behavior (different available packages, etc).
Mongkol67 commentedon Dec 20, 2022
tisdall commentedon Dec 20, 2022
In my case, I'm adding it and then removing it in a single
RUN, so no bloat. However, it's possible thataptdoes something like change "alternatives" (maybe that's not used at all forpython3added to the container?) and then doesn't return everything back to it was beforeapt install python3. Or similar to what you said about using the wrong version, my compiling of dependencies in that intermediate step accidentally uses the wrong python. These are my concerns.I probably shouldn't have trimmed my example down so much... What I'm doing is like this:
The
python setup-docker.pystep is doing pip installs which is using some of those dependencies for compiling. This version isn't pulling inpython3, but if I include theglibcomponents thenpython3gets pulled in and makes we wonder if the wrong dev headers are used or who knows what!tisdall commentedon Dec 20, 2022
@ad-m-ss , I really appreciate the reply! I'm not familiar with this technique so I'll have to look into it.
Does this work as a single(found the docs! thanks again!)Dockerfileor is this broken down in separate files?tisdall commentedon Dec 20, 2022
@ad-m-ss Should
COPY --from=build-metadata /wheels /wheelsbeCOPY --from=build-app /wheels /wheels? I'm still trying to figure this out, but I don't see wherebuild-metadatais defined and/wheelsseems to be created inbuild-app.yosifkit commentedon Dec 20, 2022
The
pythonbinaries and associated libraries provided by the image are installed to/usr/local/*so it isn't affected by any of the Debian alternatives stuff. ThePATHis set so that/usr/local/binis first.I think the second python can become a problem when a python file is directly executed that uses a static path to python in the hashbang rather than
#!/usr/bin/env python(so,python ./script.pyis fine, while./script.py, might do the wrong thing).ad-m-ss commentedon Dec 20, 2022
Yes, I updated my comment. This is the result of an incomplete redaction from a private Dockerfile. It's a bit more complicated for us. Sorry for the confusion.
tisdall commentedon Dec 20, 2022
@ad-m-ss No worries! Just wanted to confirm I was understanding it correctly. Thank you!
tisdall commentedon Mar 3, 2023
I used a multi-stage build and that worked well for me in preventing extra versions of python.