Repository navigation
npm and iojs with new es6 features #269
Description
Since io.js is going to be getting a new v8, and thus a slew of new es6 features not previously enabled by default, this is going to create a situation where some modules will only work with io.js, at least, until such time as node.js updates its own v8.
The feature in npm that currently handles this kind of thing is package.json engines field.
Now there’s an obvious problem if we add an “iojs” engine type: future versions of node will be likely be api & v8 compatible with iojs, and so depending on a specific iojs version would incorrectly block those node versions.
If adding an “iojs” engine type is out, another option would be to add a “v8” engine type. This would solve the immediate problem, but end users don’t ordinarily think or care about v8 versions, and telling them “your v8 is too old” would be a very crappy user experience (and wouldn’t give them any immediate information on how to fix the problem).
So a third option would be to create a series of “feature-xxx”* modules (eg feature-generators) that have preinstall scripts that fail with meaningful messages if you try to install them in an environment without the feature. These wouldn’t import functionality, obviously, they’d just assert a engine that can handle the feature. Further, if one desired to, even in-development features could easily be supported with major version bumps as their implementation changes.
* Perhaps not actually "feature-xxx", another prefix would do just as well, maybe even just "v8-xxx"?
What do you all think? Do you have other ideas on how we might handle this?
engines
You can specify the version of node that your stuff works on:
{ "engines" : { "node" : ">=0.10.3 <0.12" } }And, like with dependencies, if you don't specify the version (or if you specify "*" as the version), then any version of node will do.
If you specify an "engines" field, then npm will require that "node" be somewhere on that list. If "engines" is omitted, then npm will just assume that it works on node.
You can also use the "engines" field to specify which versions of npm are capable of properly installing your program. For example:
{ "engines" : { "npm" : "~1.0.20" } }Note that, unless the user has set the engine-strict config flag, this field is advisory only.
engineStrict
If you are sure that your module will definitely not run properly on versions of Node/npm other than those specified in the engines object, then you can set"engineStrict": true in your package.json file. This will override the user'sengine-strict config setting.
Please do not do this unless you are really very very sure. If your engines object is something overly restrictive, you can quite easily and inadvertently lock yourself into obscurity and prevent your users from updating to new versions of Node. Consider this choice carefully. If people abuse it, it will be removed in a future version of npm.
Activity
jonathanong commentedon Jan 9, 2015
how about a
v8engine option?iarna commentedon Jan 9, 2015
@JonathonOng: Yeah, I discussed that in paragraph 4, but it does have user experience issues. Do you have thoughts on those? That also won't provide a solution to other kinds of iojs incompatibility that may not be coming this month, but could happen.
ruimarinho commentedon Jan 10, 2015
I think that there is potential for io.js to introduce new apis not directly tied to V8 that may also lead to this situation. Ultimately, all new features introduces by io.js not available under node would likely require a
feature-xxxdeclaration on the package. Correct me if I'm wrong, but I think generally npm ecosystems could help solve this problem in a simple way (e.g. limit search to "packages that leverage ES7await"). In the end, it would be first, up to developer to decide whether to give that package a try and, secondly, to feature detect and polyfill when necessary. The same is true for node right now with packages like native-or-promise or async-listener for example.There is some overlapse with the
enginesdefinition, but with time, versions will have less meaning, just like user agents, and become a bigger burden to developers to maintain.mikeal commentedon Jan 10, 2015
I'm -1 on using engines for this.
Best practice is for module authors to feature detect when possible, this would encourage them not to do that.
There actually should be a "cost" to using new features prior to them being widely adopted but I wouldn't rule out the possibility of people creating tooling for older versions of node that check for modules using new features and compile them down to ES5.
ruimarinho commentedon Jan 10, 2015
Agreed. Several packages targeted at node 0.11 (especially for generators stuff) are recommending users to rely on transpilers such as
regeneratorfor node 0.10 compatibility. And then there's always Traceur and 6to5 runtimes for those who want to ES6 features in node and do not want make the switch to io.js just yet.rlidwka commentedon Jan 10, 2015
v8engine option makes the most sense imho.jbergstroem commentedon Jan 10, 2015
I agree that testing v8 versions doesn't really give you any more information since there now will be combinations of what flags io.js (or node.js) is started with.
@mikeal how about exposing those features through package.json somehow? That at least give package to package feature dependencies over doing it within the modules (I can see scenarios where it wouldn't make sense to test for certain features since you'd probably wouldn't even install it in the first place without the feature in place). These entries could probably even be generated.
sunflowerdeath commentedon Jan 11, 2015
It is possible to automatically find out version of V8 (and any other parameters that affect compatibility) from specified engine and version, and use it, when later there will be another version compatible with specified.
For example, if specified
{ "engines": { "iojs": ">=1.0.0" } }and node 0.15+ will be compatible with iojs 1.0, when you will try to install on old node version, error message will say that you neediojs >= 1.0.0ORnode >= 0.15, which is compatible.And when you will install this package on
node >= 0.15it will just warn, that current engine is not same, that was specified, but is compatible.iarna commentedon Jan 12, 2015
@sunflowerdeath npm could certainly keep a table of iojs and node versions and their associated v8 versions, and so translate iojs >= 1.0.0 into v8 >= x.x.x. I don't think we currently can see other parameters that effect compatibility though. But yeah, io.js could expose those, but for that to work node.js would also have to be convinced to expose them.
ljharb commentedon Jan 12, 2015
I'd love to see both - a consistent "engines" notation, which indicates what it's been /tested/ on, and a parallel "features" notation (I do not care about the name, bikeshed all you want), which indicates which JS features it depends on. Then it's left up to the npm cli, or the transpiler, or the saas platform, or whatever (or some npm module) to determine dynamically what is required to make the module work, or whether it's possible at all. Thoughts?
edef1c commentedon Jan 12, 2015
Currently, all my packages that use generators have a
foo.es6.jswith the actual source, afoo.es5.jswith transpiled code that is generated by theprepublishscript, and afoo.jsthat feature-detects and requires the appropriate file.No need for magic in
package.json, and works across all engines. It might be nice to have a more convenient wrapper around this kind of feature detection / transpilation.ljharb commentedon Jan 12, 2015
I'm not thinking "magic" as much as "documentation" - your approach is great, but how am I to know that's the format you've used? As described, I'd have to duck type it, whereas with a
package.jsonnotation, you'd explicitly call out what you supported.caitp commentedon Jan 12, 2015
adding more metadata to package.json about language features used, or v8 versions tested against/supported:
These are all fine, so long as you are vigilant about updating your package.json, but if we're honest, most people aren't.
automating detection of features used:
This seems like a really, really bad thing to bake into npm, and probably not a very good thing to do in an individual project either (although it's obviously up to you to make the call on how you ship your project)
indexzero commentedon Jan 12, 2015
Yes, oh 1000 times yes. That being said it wouldn't be bad practice (although not required) for something in package.json for batch scripts doing analytics. Not everything can be polyfilled via a transpiler. For those things I wouldn't want
npmto warn me, but it's also a like riding a Ferrari to the grocery store to need static analysis to figure this out from a machine perspective.brianleroux commentedon Jan 12, 2015
👍 for using
package.json mainpoint to compiled ES5 source. 🍻 🐴11 remaining items
edef1c commentedon Jan 21, 2015
Example feature-detection package: has-generators
edef1c commentedon Jan 21, 2015
If your stuff requires multiple ES6 things,
(require('has-foo') && require('has-bar')) ? … : ….We can create these tiny little packages with ease for all the ES6 features that are available in io.js today.
ljharb commentedon Jan 21, 2015
I actually have https://www.npmjs.com/package/make-generator-function published for an implicit "has-generators" check :-)
taoeffect commentedon Jan 21, 2015
Why is this a problem?
Let people specify old versions of modules if they don't want to update node. They can already do this without any changes or feature detection stuff.
Let's not create a python2/python3 situation here.
Embrace progress. Don't create Frankensteins.
ljharb commentedon Jan 21, 2015
Anything distributed that's not in commonJS format is a python2/python3 situation. In order to support ES6 module syntax, anything we come up with will have to piggyback on top of an included CJS module as the default export, and every version (old and new) will have to work in old nodes too ¯_(ツ)_/¯
edef1c commentedon Jan 21, 2015
@ljharb Yeah, I was going for
has-*as naming pattern, exporting a bool as API — trying to create a de-facto standard. Your API changes shape along with the feature.ljharb commentedon Jan 21, 2015
@nathan7 Yours is totes clearer. I just overload mine because it's used to test "is-generator-function" methods :-)
mstade commentedon Jan 31, 2015
FWIW, I do feature detection at installation time. It has the benefit of not needlessly bloating your runtime with feature detection libraries. I suppose the detection library could do both, but this solved my immediate needs.
Qard commentedon Jan 31, 2015
IMO this is more a messaging issue.
As has already been discussed, version detection is unreliable. Feature detection is better, but could easily explode in complexity as more and more new features are added and used.
On http://iojs.org/es6.html we explicitly state the language support differences. I think people that use these features in modules should be explicitly stating the requirements to use those modules. Obviously there's the possibility of people not bothering with it, but the same can be said of the engines field.
I do like the idea of using some magic thing to convert es6 code to work with es5 though.
Fishrock123 commentedon Feb 26, 2015
Doesn't seem like there's really much we can do here. Re-open if this (es6 + node/io.js + npm) needs to be discussed more.
edef1c commentedon Mar 10, 2015
Just for anyone looking to build cross-compatible packages, here's one of mine as a reference:
callumlocke commentedon Apr 9, 2015
@mstade:
What if I
npm installyour package while I'm using io.js, but later I use n to switch to Node 0.10 for some reason? (As a dev working on several projects I switch around all the time). Then your package would just fail to work, and it wouldn't necessarily be obvious why. Feature detection at runtime seems like a safer option.mstade commentedon Apr 16, 2015
@callumlocke it could also be argued that you shouldn't have development environments depend on global set up and tooling, but rather have project local configuration – I tend to use Vagrant for things like that. Either way this is going off topic, and so I won't delve too deep into it. If you'd like to discuss feel free to tweet me or something.
balupton commentedon Nov 30, 2015
For what it's worth, we've been more or less accomplishing the aim with https://github.com/bevry/esnextguardian