Skip to content

Yarn's bundled node-gyp broken #346

Closed
Closed
Yarn's bundled node-gyp broken#346
Labels
bugyarn

Description

@wyattjoh
wyattjoh
opened on Mar 1, 2017

The current version of yarn still has a pre-existing issue with the pre-bundled version of node-gyp related to yarnpkg/yarn#2266

A temporary fix relates to adding:

RUN yarn global add node-gyp

To our Dockerfile.

Activity

  1. Starefossen
    added
    yarn
    bug
    on Mar 1, 2017

  2. wyattjoh commented on Mar 1, 2017

    @wyattjoh
    wyattjoh
    on Mar 1, 2017
    Author

    You can refer to: https://circleci.com/gh/coralproject/talk/1921 with details of why the build failed.

  3. LaurentGoderre commented on Mar 2, 2017

    @LaurentGoderre
    LaurentGoderre
    on Mar 2, 2017
    Member

    To me it failed only because the build was trying to override the included yarn. I don't see any node-gyp error.

  4. LaurentGoderre commented on Mar 2, 2017

    @LaurentGoderre
    LaurentGoderre
    on Mar 2, 2017
    Member

    As you can see, several other build because of trying to install yarn via npm: #337 (comment)

  5. wyattjoh commented on Mar 2, 2017

    @wyattjoh
    wyattjoh
    on Mar 2, 2017
    Author

    I linked the wrong build logs :( below is the relevant error:

    Step 9/11 : RUN yarn install --production
 ---> Running in c936d65bcc45
yarn install v0.21.3
[1/4] Resolving packages...
[2/4] Fetching packages...
warning fsevents@1.0.17: The platform "linux" is incompatible with this module.
info "fsevents@1.0.17" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
error /usr/src/app/node_modules/bcrypt: Command failed.
Exit code: 127
Command: sh
Arguments: -c node-gyp rebuild
Directory: /usr/src/app/node_modules/bcrypt
Output:
sh: 1: node-gyp: not found
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
The command '/bin/sh -c yarn install --production' returned a non-zero code: 1

    Which was ran with the repo here https://github.com/coralproject/talk/tree/2c98aa20977155d09b08e927c574be1221d2d707 where I had removed my patch from the Dockerfile to make it the following:

    FROM node:7.6

# Add node-gyp for bcrypt build support
# RUN yarn global add node-gyp

# Create app directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

# Setup the environment
ENV NODE_ENV production
ENV PATH /usr/src/app/bin:$PATH
ENV TALK_PORT 5000
EXPOSE 5000

# Install app dependencies
COPY package.json yarn.lock /usr/src/app/
RUN yarn install --production

# Bundle app source
COPY . /usr/src/app

CMD [ "yarn", "start" ]

  6. LaurentGoderre commented on Mar 2, 2017

    @LaurentGoderre
    LaurentGoderre
    on Mar 2, 2017
    Member

    Node-gyp is not included in node though so you would always need to install it no?

  7. wyattjoh commented on Mar 2, 2017

    @wyattjoh
    wyattjoh
    on Mar 2, 2017
    Author

    Previously, using npm install it successfully installs.

  8. LaurentGoderre commented on Mar 2, 2017

    @LaurentGoderre
    LaurentGoderre
    on Mar 2, 2017
    Member

    It doesn't install with yarn?

  9. wyattjoh commented on Mar 2, 2017

    @wyattjoh
    wyattjoh
    on Mar 2, 2017
    Author

    That's correct. With the following Dockerfile:

    FROM node:7.6

# Add node-gyp for bcrypt build support
# RUN yarn global add node-gyp

# Create app directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

# Setup the environment
ENV NODE_ENV production
ENV PATH /usr/src/app/bin:$PATH
ENV TALK_PORT 5000
EXPOSE 5000

# Install app dependencies
COPY package.json yarn.lock /usr/src/app/
# RUN yarn install --production
RUN npm install --production

# Bundle app source
COPY . /usr/src/app

CMD [ "npm", "start" ]

    I got no errors during install.

  10. LaurentGoderre commented on Mar 2, 2017

    @LaurentGoderre
    LaurentGoderre
    on Mar 2, 2017
    Member

    That's weird...I ran the same thing and it worked:

    Sending build context to Docker daemon 2.048 kB
Step 1/2 : FROM node:7.6
7.6: Pulling from library/node
693502eb7dfb: Pull complete 
081cd4bfd521: Pull complete 
5d2dc01312f3: Pull complete 
54a5f7da9a4f: Pull complete 
f6ebc0704397: Pull complete 
cd487f72ac9f: Pull complete 
d3e88bf53e70: Pull complete 
88dbf525e469: Pull complete 
Digest: sha256:34d21bab49105c1ab384e50432aa6ebd6dac69c2637378e7ecb6f40cda97118b
Status: Downloaded newer image for node:7.6
 ---> 7b74673e3e78
Step 2/2 : RUN yarn global add node-gyp
 ---> Running in 58721d12a3bd
yarn global v0.21.3
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Installed "node-gyp@3.5.0" with binaries:
      - node-gyp
Done in 5.15s.
 ---> d383b79b9c88
Removing intermediate container 58721d12a3bd
Successfully built d383b79b9c88

  11. LaurentGoderre commented on Mar 2, 2017

    @LaurentGoderre
    LaurentGoderre
    on Mar 2, 2017
    Member

    Is it possible that node-gyp installs but isn't working?

  12. wyattjoh commented on Mar 2, 2017

    @wyattjoh
    wyattjoh
    on Mar 2, 2017
    Author

    This is my issue:

    When installing via yarn, I need to run:

    yarn global add node-gyp
yarn install --production

    While with npm I can run:

    npm install --production

    I don't need to explicitly install node-gyp, hence why it seems that the upstream issue with yarn (yarnpkg/yarn#2266) might be causing the issue now where when installed via yarn (specifically, the bundled version), that node-gyp fails.

  13. wyattjoh commented on Mar 2, 2017

    @wyattjoh
    wyattjoh
    on Mar 2, 2017
    Author

    Notice that in your example, you explicitly ran

    yarn global add node-gyp

    As the second step.

  14. LaurentGoderre commented on Mar 2, 2017

    @LaurentGoderre
    LaurentGoderre
    on Mar 2, 2017
    Member

    This seems to be similar to your issue: yarnpkg/yarn#1848

  15. wyattjoh commented on Mar 2, 2017

    @wyattjoh
    wyattjoh
    on Mar 2, 2017
    Author

    I can confirm that after running:

    yarn upgrade bcrypt

    Which resulted in the following changes:

    diff --git a/package.json b/package.json
index 740063dd..eaa15303 100644
--- a/package.json
+++ b/package.json
@@ -49,7 +49,7 @@
   },
   "homepage": "https://github.com/coralproject/talk#readme",
   "dependencies": {
-    "bcrypt": "^0.8.7",
+    "bcrypt": "^1.0.2",
     "body-parser": "^1.15.2",
     "cli-table": "^0.3.1",
     "commander": "^2.9.0",
diff --git a/yarn.lock b/yarn.lock
index 0331202e..f8a49205 100644
--- a/yarn.lock
+++ b/yarn.lock
@@ -1089,12 +1089,13 @@ bcrypt-pbkdf@^1.0.0:
   dependencies:
     tweetnacl "^0.14.3"
 
-bcrypt@^0.8.7:
-  version "0.8.7"
-  resolved "https://registry.yarnpkg.com/bcrypt/-/bcrypt-0.8.7.tgz#bc3875a9afd0a7b2cd231a6a7f218a5ce156b093"
+bcrypt@^1.0.2:
+  version "1.0.2"
+  resolved "https://registry.yarnpkg.com/bcrypt/-/bcrypt-1.0.2.tgz#d05fc5d223173e0e28ec381c0f00cc25ffaf2736"
   dependencies:
     bindings "1.2.1"
-    nan "2.3.5"
+    nan "2.5.0"
+    node-pre-gyp "0.6.32"
 
 big.js@^3.1.3:
   version "3.1.3"
@@ -5006,7 +5007,11 @@ mute-stream@0.0.7:
   version "0.0.7"
   resolved "https://registry.yarnpkg.com/mute-stream/-/mute-stream-0.0.7.tgz#3075ce93bc21b8fab43e1bc4da7e8115ed1e7bab"
 
-nan@2.3.5, nan@^2.3.0:
+nan@2.5.0:
+  version "2.5.0"
+  resolved "https://registry.yarnpkg.com/nan/-/nan-2.5.0.tgz#aa8f1e34531d807e9e27755b234b4a6ec0c152a8"
+
+nan@^2.3.0:
   version "2.3.5"
   resolved "https://registry.yarnpkg.com/nan/-/nan-2.3.5.tgz#822a0dc266290ce4cd3a12282ca3e7e364668a08"
 
@@ -5111,7 +5116,7 @@ node-libs-browser@^2.0.0:
     util "^0.10.3"
     vm-browserify "0.0.4"
 
-node-pre-gyp@^0.6.29:
+node-pre-gyp@0.6.32, node-pre-gyp@^0.6.29:
   version "0.6.32"
   resolved "https://registry.yarnpkg.com/node-pre-gyp/-/node-pre-gyp-0.6.32.tgz#fc452b376e7319b3d255f5f34853ef6fd8fe1fd5"
   dependencies:

    It was successfully able to build! I'm still hesitant as the fact that the same package.json + yarn.lock file can build the project successfully without explicitly installing a package (node-gyp) yet yarn required some convincing to get it to run.

    To me it still feels like there is an issue that yarn should sort out upstream to match the ability that node currently has to build these dependancies properly.

    I'll close this for now, but feel free to ask any questions if anyone wants to explore this topic more.

  16. wyattjoh
    closed this as completedon Mar 2, 2017

  17. LaurentGoderre commented on Mar 2, 2017

    @LaurentGoderre
    LaurentGoderre
    on Mar 2, 2017
    Member

    I just tried switching to bcrypt to bcryptjs and it worked, there still is the warning about fsevents though.

  18. LaurentGoderre commented on Mar 2, 2017

    @LaurentGoderre
    LaurentGoderre
    on Mar 2, 2017
    Member

    It is indeed weird. I isolated the problem

    This works:

    FROM node:7.6

RUN npm install bcrypt@0.8.7

    This doesn't:

    FROM node:7.6

RUN yarn add bcrypt@0.8.7

  19. wyattjoh commented on Mar 2, 2017

    @wyattjoh
    wyattjoh
    on Mar 2, 2017
    Author

    That would be my conclusion as well. The fact that 0.8.7 did not include "node-pre-gyp": "0.6.32" as a dep is why 1.0.2 worked. This issue is still persisting then.

  20. terribleplan commented on Mar 2, 2017

    @terribleplan
    terribleplan
    on Mar 2, 2017

    So when building a docker image descended from node:6 everything is really messed up:

    Step 6/7 : RUN yarn
 ---> Running in f1c656e5fc13
yarn install v0.21.3
[1/4] Resolving packages...
[2/4] Fetching packages...
[4/4] Building fresh packages...
error /usr/src/app/node_modules/scrypt: Command failed.
Exit code: 127
Command: sh
Arguments: -c node-gyp rebuild
Directory: /usr/src/app/node_modules/scrypt
Output:
sh: 1: node-gyp: not found
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
ERROR: Service 'web' failed to build: The command '/bin/sh -c yarn' returned a non-zero code: 1

    I'd be curious if @Daniel15 has any ideas since he is a member of yarnpkg and was involved with the PR that included yarn with this image. yarn/2266 seems to be the cause.

    All of our builds/containers have been broken for 2 days now. I'd love to see yarn get removed until node:7, since this is basically a breaking change.

  21. juliocanares commented on May 7, 2017

    @juliocanares
    juliocanares
    on May 7, 2017

    Still having the issue. Currently using @wyattjoh fix.

    Adding RUN yarn global add node-gyp before yarn install works

  22. Daniel15 commented on May 7, 2017

    @Daniel15
    Daniel15
    on May 7, 2017

    This is fixed in the next version of Yarn (currently only available as an RC, but there should be a stable release soon)

  23. juliocanares commented on May 7, 2017

    @juliocanares
    juliocanares
    on May 7, 2017

    @Daniel15 cool to know it 👍 I will try

  24. spinlock99 commented on Jul 9, 2017

    @spinlock99
    spinlock99
    on Jul 9, 2017

    I just wanted to note my experience here. I was using nvm on a debian box and getting what looked like this node-gyp issue. I had to uninstall nvm and use the system node to actually get everything working. But, I did not need to install node-gyp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugyarn

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions