Skip to content

Does apt install of python3 affect python in slim? #777

Closed

Description

@tisdall

I was looking at reducing my build size by using the slim build and then only including build dependencies while building and then purging them. Everything seems to work fine. However, I noticed that when I include glib that apt pulls in python3. I see in #253 that this is to be expected, but I was wondering if this has any potential in breaking the python already within the container.

Specifically, does having something like this in Dockerfile have any potential in breaking something?

RUN apt-get update && \
    apt-get install -y --no-install-recommends python3 && \
    apt-get purge --autoremove -y python3 && \
    rm -rf /var/lib/apt/lists/*

This is obviously contrived, but does apt adding it's own python3 and removing it alter things that need to not be altered?

Activity

  1. ad-m-ss commented on Dec 19, 2022

    @ad-m-ss

    This is not a direct answer to your question, but if you want to optimize the image, you can think about multi-stage build for building dependencies , then you don't need care to remove Python because it will be in a different stage:

    FROM python:3.10.7-bullseye AS build-pip
    ENV PIP_CACHE_DIR=/var/cache/buildkit/pip
    RUN mkdir -p $PIP_CACHE_DIR
    RUN --mount=type=cache,target=/var/cache/buildkit/pip \
        pip install -U pip wheel
    
    FROM build-pip AS build-app
    COPY ./requirements.txt .
    RUN --mount=type=cache,target=/var/cache/buildkit/pip \
        pip wheel --wheel-dir /wheels -r requirements.txt
    
    FROM python:3.10.7-slim-bullseye
    
    RUN --mount=type=cache,target=/var/cache/apt apt-get update \
        && apt-get install -y --no-install-recommends unixodbc git libxml2 openssh-client \
        && rm -rf /var/lib/apt/lists/*
    ENV PIP_CACHE_DIR=/var/cache/buildkit/pip
    RUN mkdir -p $PIP_CACHE_DIR
    
    # Install requirements
    WORKDIR /app
    COPY requirements.txt ./
    
    COPY --from=build-app /wheels /wheels
    
    RUN --mount=type=cache,target=/var/cache/buildkit/pip \
        pip install --find-links /wheels --no-index -r requirements.txt
    
    COPY . .
  2. tianon commented on Dec 19, 2022

    @tianon
    Member

    Installing Python from APT ends up with another copy/version of Python in your image, which definitely causes bloat -- it shouldn't directly interfere with anything for this image's installation of Python, but it does increase the chances that you accidentally invoke the wrong version and then see odd behavior (different available packages, etc).

  3. Mongkol67 commented on Dec 20, 2022

    @Mongkol67
  4. tisdall commented on Dec 20, 2022

    @tisdall
    Author

    Installing Python from APT ends up with another copy/version of Python in your image, which definitely causes bloat -- it shouldn't directly interfere with anything for this image's installation of Python, but it does increase the chances that you accidentally invoke the wrong version and then see odd behavior (different available packages, etc).

    In my case, I'm adding it and then removing it in a single RUN, so no bloat. However, it's possible that apt does something like change "alternatives" (maybe that's not used at all for python3 added to the container?) and then doesn't return everything back to it was before apt install python3. Or similar to what you said about using the wrong version, my compiling of dependencies in that intermediate step accidentally uses the wrong python. These are my concerns.

    I probably shouldn't have trimmed my example down so much... What I'm doing is like this:

    RUN apt-get update && \
        apt-get install -y --no-install-recommends \
            gcc g++ libtool libffi-dev libcurl4-openssl-dev libpq-dev libbz2-dev libc6-dev \
            libevent-dev libgmp-dev liblzma-dev libncurses5-dev libncursesw5-dev \
            libreadline-dev libsqlite3-dev libssl-dev libxml2-dev libyaml-dev zlib1g-dev \
            curl git \
            libpq5 openssl gettext && \
        python setup-docker.py && \
        apt-get purge --autoremove -y \
            gcc g++ libtool libffi-dev libcurl4-openssl-dev libpq-dev libbz2-dev libc6-dev \
            libevent-dev libgmp-dev liblzma-dev libncurses5-dev libncursesw5-dev \
            libreadline-dev libsqlite3-dev libssl-dev libxml2-dev libyaml-dev zlib1g-dev \
            curl git && \
        rm -rf /var/lib/apt/lists/*
    

    The python setup-docker.py step is doing pip installs which is using some of those dependencies for compiling. This version isn't pulling in python3, but if I include the glib components then python3 gets pulled in and makes we wonder if the wrong dev headers are used or who knows what!

  5. tisdall commented on Dec 20, 2022

    @tisdall
    Author

    @ad-m-ss , I really appreciate the reply! I'm not familiar with this technique so I'll have to look into it. Does this work as a single Dockerfile or is this broken down in separate files? (found the docs! thanks again!)

  6. tisdall commented on Dec 20, 2022

    @tisdall
    Author

    @ad-m-ss Should COPY --from=build-metadata /wheels /wheels be COPY --from=build-app /wheels /wheels? I'm still trying to figure this out, but I don't see where build-metadata is defined and /wheels seems to be created in build-app.

  7. yosifkit commented on Dec 20, 2022

    @yosifkit
    Member

    The python binaries and associated libraries provided by the image are installed to /usr/local/* so it isn't affected by any of the Debian alternatives stuff. The PATH is set so that /usr/local/bin is first.

    I think the second python can become a problem when a python file is directly executed that uses a static path to python in the hashbang rather than #!/usr/bin/env python (so, python ./script.py is fine, while ./script.py, might do the wrong thing).

  8. ad-m-ss commented on Dec 20, 2022

    @ad-m-ss

    @ad-m-ss Should COPY --from=build-metadata /wheels /wheels be COPY --from=build-app /wheels /wheels?

    Yes, I updated my comment. This is the result of an incomplete redaction from a private Dockerfile. It's a bit more complicated for us. Sorry for the confusion.

  9. tisdall commented on Dec 20, 2022

    @tisdall
    Author

    @ad-m-ss No worries! Just wanted to confirm I was understanding it correctly. Thank you!

  10. tisdall commented on Mar 3, 2023

    @tisdall
    Author

    I used a multi-stage build and that worked well for me in preventing extra versions of python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions