[RFC] 0001 Type Extensibility - #4
RandomByte wants to merge 3 commits into
Conversation
See [RFC0001](UI5/cli#4). Identify and apply extensions in dependency tree so that they can influence the actual project processing.
As discussed today between @matz3, @tommyvinhlam, @codeworrior, @RandomByte Build phases would allow for more precise extensibility of a types build execution. Mainly scenarios where only a single task should be replaced or added could overwrite one of the build phases to hook in-between of certain sets of tasks. Instead, they will have to overwrite the whole build function containing all tasks. The main reason for this decision is the lack of known extensibility use-cases. Therefore we do not want to decide on the naming and semantics of any build phases yet as changing them later on will hardly be possible.
|
a63033f: Removed build phases as discussed today between @matz3, @tommyvinhlam, @codeworrior, @RandomByte Build phases would allow for more precise extensibility of a types build execution. Mainly scenarios where only a single task should be replaced or added could overwrite one of the build phases to hook in-between of certain sets of tasks. Instead, they will have to overwrite the whole build function containing all tasks. The main reason for this decision is the lack of known extensibility use-cases. Therefore we do not want to decide on the naming and semantics of any build phases yet as changing them later on will hardly be possible. |
See [RFC0001](UI5/cli#4). Identify and apply extensions in dependency tree so that they can influence the actual project processing.
|
As suggested in https://github.com/SAP/ui5-builder/issues/32 I'd like to add thoughts about this topic in the mix. (Disclaimer: this is a lot of text and I'd like to say sorry for that. I'm not good at being brief.) I really like the idea of being able to extend and "personalize" the whole build process to be able to tweak the tooling to a project's needs and those of its developers. I experimented with the use of babel to translate modern es6/esnext syntax to something even the poor little IE11 can understand. When working on a UI5 application I can run the application from source in chrome, even run the test in chrome (manually or using karma) and iterate quickly while doing tests on the compiled/bundled versions of the application on demand or even as late as on the CI. Our toolchain (described in a gruntfile) looks something like this With the ui5-tooling I could replace the gruntfile with just the ui5.yml and get 4 / 6 steps from just that (+ a lot of cool new things the grunt setup cannot do, like standalone builds). I currently think that my "ideal version of reality" would look something like this: The babel-plugin is then loaded when setting up the build and is able to hook into the build process on clearly defined interfaces. Since the whole idea of the ui5-tooling (as I understand it) is to be an "opinionated tool runner" purposed for ui5 application/library/extension development there is no need to allow the configuration of bells and whistles. Destination and source directory and pretty much set in stone. Coming back to the actual PR after this short journey, to solve this problem with a type extension I'd need to define a new 'babel-compiled-application', extend the base application in a class and inject babel in that pipeline. If I want to build a library using babel as a transpiler I'd need to do the same thing for a 'babel-compiled-library'. Very long story short: While I like to be able to extend and configure everything, I'm not sure if that's the best way to allow a configurable build process. I'd like to be able to build a process by picking add-ons/plugins over having to replace the whole thing. But I may have misunderstood the purpose of this PR. |
I'm totally with you on that. I'd like to see ui5-tooling as a set of functions/tasks that I can stick together and run however I want with placing additional stuff before, after or right in the middle of it. That is why I prefer gulp over grunt a lot. |
|
In our internal discussions, one of the most controversial topics is whether the UI5 Tooling shall have characteristics of a task runner or that of a collection of tasks. Personally I don't want to build another task runner here. There are plenty of those out there. However, with the CLI and Types we kind of already built one. So I have to agree that it makes sense for people to customize and enhance what The main questions I would like to get clarified as a starter here are:
I'll add my ideas once I can think of something simpler than an enormous configuration architecture like Maven and others have 😑 |
|
We discussed this again internally and would like to go forward with a simpler approach very similar to the proposal of @themasch. RFC 0004 Simple Build Extensibility Please have a read. Any feedback on the concept is appreciated. Also, thank you @themasch and @cschuff for your feedback so far! |
See [RFC0001](UI5/cli#4). Identify and apply extensions in dependency tree so that they can influence the actual project processing.
See [RFC0001](UI5/cli#4). Identify and apply extensions in dependency tree so that they can influence the actual project processing.
See [RFC0001](UI5/cli#4). Identify and apply extensions in dependency tree so that they can influence the actual project processing.
See [RFC0001](UI5/cli#4). Identify and apply extensions in dependency tree so that they can influence the actual project processing.
See [RFC0001](#4). Identify and apply extensions in dependency tree so that they can influence the actual project processing.
See [RFC0001](#4). Identify and apply extensions in dependency tree so that they can influence the actual project processing.
refactor: Correct name for relase please manifest
Generalize non-resource input tracking beyond environment variables to any value a task reads through the TaskUtil interface: isRootProject(), getDependencies(), and the getProject(name).get* accessors (a dependency's version, custom configuration, framework getters). A task whose output depends on such a value, for example generateLibraryManifest embedding a dependency's version as the manifest minVersion via getProject(dep).getVersion(), now invalidates its cached result when that value changes, where before only a changed resource did. Recording moves out of TaskUtil/ProjectBuildContext into a new MonitoredTaskUtil wrapper (a sibling of MonitoredReader): the TaskRunner hands each task a monitor that records the tracked reads and drains them via getInputRecording(). It is a Proxy, so it preserves the wrapped interface shape (a custom task's limited interface stays limited) and only intercepts the tracked reads; reads made outside a task by build orchestration holding the raw TaskUtil stay untracked. The lookup side re-derives current values through ProjectBuildContext.resolveInputValue(type, name), which reaches process.env and the current project graph, so a cache lookup reflects the build performing it. Record and lookup share normalizeInputValue. Rename InputHashTree to TaskInputSet: it is a flat, hashed key->value set, not a Merkle tree like HashTree, and the old name implied structure it does not have. It keeps HashTree's cache-object conventions (version field, tolerant fromCache). Excluded from tracking: tag mutations (already captured as tag operations), resourceFactory constructors, reader accessors (their reads are tracked as resource requests), registerCleanupTask, and getRootPath/getSourcePath (absolute paths would make cache entries non-portable). The dependency-integration test's build #4 now correctly rebuilds library.a/b/c when library.d is removed, because their manifest embeds library.d's version; the previous expectation encoded the stale-cache behavior this change fixes. The sap.ui.core version-invalidation integration test flips from test.failing to passing, with its fixture helper's path bug fixed.
A task's output can depend on inputs that are not resources: an environment
variable, or a value read through the TaskUtil interface (isRootProject(),
getDependencies(), a dependency's version via getProject(name).getVersion(),
getCustomConfiguration(), the framework getters). None of these feed the
resource indices or the build signature, so changing one between builds left a
stale cached result being served. The canonical case: generateLibraryManifest
embeds a dependency's version as the manifest minVersion via
getProject(dep).getVersion(), so removing or bumping that dependency must
re-run the task even though no source resource changed.
Tracking has a record side and a lookup side, mirroring the resource-request
flow. On record, the TaskRunner hands each task a MonitoredTaskUtil instead of
the raw TaskUtil: a Proxy that preserves the wrapped interface shape (a custom
task's limited interface stays limited) and records every tracked read as
{type, name, value}. getInputRecording() drains the recording after the task,
and recordTaskResult folds it into a TaskInputSet whose signature is combined
into the task's project-component signature. On lookup,
BuildTaskCache.getInputSignature recomputes the signature by re-reading each
recorded input's current value via ProjectBuildContext.resolveInputValue(type,
name), which reaches process.env and the current project graph. Record and
lookup normalize through the same normalizeInputValue, so equal values compare
equal; a differing value misses both the per-task stage cache and the
project-level result cache and forces re-execution.
TaskInputSet is a flat, hashed type+name -> value set, not a Merkle tree like
HashTree: task inputs are few, unordered and non-hierarchical. It keeps
HashTree's cache-object conventions (a version field, a tolerant fromCache) but
none of its structure. Only entry type/name are persisted (task_metadata type
"input"), never values.
Read an env var through taskUtil.getEnv(name) rather than process.env so the
monitor observes the read. Excluded from tracking: tag mutations (already
captured as tag operations), the resourceFactory constructors, the reader
accessors (their reads are tracked as resource requests), registerCleanupTask,
and getRootPath/getSourcePath, whose absolute paths would make cache entries
non-portable.
Tasks that declare no inputs hash to a fixed empty-input digest that is still
combined into every signature, so CACHE_VERSION moves to v0_8 and stale dev
caches are ignored rather than misread.
The dependency-integration test's build #4 now rebuilds library.a/b/c when
library.d is removed, because their manifest embeds library.d's version; the
previous expectation encoded the stale-cache behavior this fixes. The
sap.ui.core version-invalidation integration test moves from test.failing to
passing, with its fixture helper's path bug fixed.
Co-authored-by: Matthias Osswald <mat.osswald@sap.com>
A task's output can depend on inputs that are not resources: an environment
variable, or a value read through the TaskUtil interface (isRootProject(),
getDependencies(), a dependency's version via getProject(name).getVersion(),
getCustomConfiguration(), the framework getters). None of these feed the
resource indices or the build signature, so changing one between builds left a
stale cached result being served. The canonical case: generateLibraryManifest
embeds a dependency's version as the manifest minVersion via
getProject(dep).getVersion(), so removing or bumping that dependency must
re-run the task even though no source resource changed.
Tracking has a record side and a lookup side, mirroring the resource-request
flow. On record, the TaskRunner hands each task a MonitoredTaskUtil instead of
the raw TaskUtil: a Proxy that preserves the wrapped interface shape (a custom
task's limited interface stays limited) and records every tracked read as
{type, name, value}. getInputRecording() drains the recording after the task,
and recordTaskResult folds it into a TaskInputSet whose signature is combined
into the task's project-component signature. On lookup,
BuildTaskCache.getInputSignature recomputes the signature by re-reading each
recorded input's current value via ProjectBuildContext.resolveInputValue(type,
name), which reaches process.env and the current project graph. Record and
lookup normalize through the same normalizeInputValue, so equal values compare
equal; a differing value misses both the per-task stage cache and the
project-level result cache and forces re-execution.
TaskInputSet is a flat, hashed type+name -> value set, not a Merkle tree like
HashTree: task inputs are few, unordered and non-hierarchical. It keeps
HashTree's cache-object conventions (a version field, a tolerant fromCache) but
none of its structure. Only entry type/name are persisted (task_metadata type
"input"), never values.
Read an env var through taskUtil.getEnv(name) rather than process.env so the
monitor observes the read. Excluded from tracking: tag mutations (already
captured as tag operations), the resourceFactory constructors, the reader
accessors (their reads are tracked as resource requests), registerCleanupTask,
and getRootPath/getSourcePath, whose absolute paths would make cache entries
non-portable.
Tasks that declare no inputs hash to a fixed empty-input digest that is still
combined into every signature, so CACHE_VERSION moves to v0_8 and stale dev
caches are ignored rather than misread.
The dependency-integration test's build #4 now rebuilds library.a/b/c when
library.d is removed, because their manifest embeds library.d's version; the
previous expectation encoded the stale-cache behavior this fixes. The
sap.ui.core version-invalidation integration test moves from test.failing to
passing, with its fixture helper's path bug fixed.
Co-authored-by: Matthias Osswald <mat.osswald@sap.com>
A feature to customize how a specific UI5 project is being built.
CC: @tommyvinhlam @matz3