Skip to content

Validation of patterns against pattern template syntax #159

New issue
New issue
Closed
Closed
Validation of patterns against pattern template syntax#159
Labels
⚙️ Type - MetaImproving how we collaborate in this repo is the main focus of this issue / PR

Description

@spier
spier
opened on May 4, 2020

It could be helpful to validation patterns against the Pattern Template for various reasons:

  • bring existing patterns to a more consistent format, which helps us with the preparation for the creation of the ebook
  • help new contributors by validating their pattern contributions, and pointing them to specifics parts of the Pattern Template that they missed

Validation ideas

  1. all mandatory headlines/sections are present
  2. the top level headlines of the pattern all use ## (h2) (i.e. no #)
  3. no duplicate headlines/sections exist
  4. headlines/sections are in a certain order
  5. no headlines/sections exist that are not part of the pattern template

Risks/Questions:

  • the validations could be frustrating or discouraging for new contributors. however if we do it right, the validations could help them to understand the pattern format.
  • what to do with sections where the headline is present but then there is actually no text filled into that section?

Status

I tried out markdownlint, together with the the GitHub Action markdown-linting-action.

As many of the existing patterns contain optional headlines as well as custom headlines, we will likely have to write custom validation rules. I have already tried this out and it looks manageable. (see experimental code).

My current suggestion would be to validate rules 1-4 above, but don't validate rule 5.

Activity

  1. spier commented on May 4, 2020

    @spier
    spier
    on May 4, 2020
    MemberAuthor

    The validation results could look similar to this

    $ markdownlint -c lint/pattern-template.yml -r lint/pattern-template.js start-as-experiment.md

start-as-experiment.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template headlines/sections [Required headline/section is missing] [Context: "Solutions"]
start-as-experiment.md:43 PATTERN-TEMPLATE-RULE-002 Standard Headlines [Allowed types are: Title, Patlet, Problem, Story, Context, Forces, Solutions, Resulting Context, Known Instances, Status, Author(s), Acknowledgements] [Context: "## Solution"]
start-as-experiment.md:92 PATTERN-TEMPLATE-RULE-002 Standard Headlines [Allowed types are: Title, Patlet, Problem, Story, Context, Forces, Solutions, Resulting Context, Known Instances, Status, Author(s), Acknowledgements] [Context: "## Related Patterns"]
start-as-experiment.md:105 PATTERN-TEMPLATE-RULE-002 Standard Headlines [Allowed types are: Title, Patlet, Problem, Story, Context, Forces, Solutions, Resulting Context, Known Instances, Status, Author(s), Acknowledgements] [Context: "## Author"]

  2. spier commented on May 5, 2020

    @spier
    spier
    on May 5, 2020
    MemberAuthor

    The GitHub action generates emails if the linter fails. However the email itself does not contain the info about what exactly failed. So one has to click through to the GitHub repo to figure out what went wrong. That isn't very helpful :)

    Example screenshot of the emails that get generated if the linter fails:

    linter-failure-email

  3. spier commented on May 5, 2020

    @spier
    spier
    on May 5, 2020
    MemberAuthor

    I have created a version of these checks that is very lax (on purpose).

    When run against all existing patterns, it mostly finds these things:

    • patterns that are using # instead of ## for the headlines
    • differences in spelling for Author/Authors/Author(s) and Solution/Solutions

    I think we can live with the difference in Author/Solution spelling for now. Will try to create a linter that accepts all different versions of that spelling.

    See full output of the current linter below.

    $ markdownlint -c lint/pattern-template.yml -r lint/pattern-template.js -i README.md -i TRUSTED-COMMITTERS.md -i CONTRIBUTING.md *.md

30-day-warranty.md:1 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Title"]
30-day-warranty.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Title" is missing. Required headline "Patlet" is missing. Required headline "Problem" is missing. Required headline "Context" is missing. Required headline "Forces" is missing. Required headline "Solutions" is missing. Required headline "Resulting Context" is missing. Required headline "Status" is missing. Required headline "Author(s)" is missing.]
30-day-warranty.md:5 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Context"]
30-day-warranty.md:11 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Problem"]
30-day-warranty.md:15 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Forces"]
30-day-warranty.md:41 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Solution"]
30-day-warranty.md:51 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Resulting Context"]
30-day-warranty.md:58 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Known Instances"]
30-day-warranty.md:62 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Authors"]
30-day-warranty.md:66 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Acknowledgement"]
30-day-warranty.md:73 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Status"]
30-day-warranty.md:79 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Variants"]

bad-weather-for-liftoff.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing.]

change-the-developers-mindset.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

common-requirements.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

contracted-contributor.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

crossteam-project-valuation.md:1 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Title"]
crossteam-project-valuation.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Title" is missing. Required headline "Patlet" is missing. Required headline "Problem" is missing. Required headline "Context" is missing. Required headline "Forces" is missing. Required headline "Solutions" is missing. Required headline "Resulting Context" is missing. Required headline "Status" is missing. Required headline "Author(s)" is missing.]
crossteam-project-valuation.md:5 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Patlet"]

dedicated-community-leader.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

developer-incentive-alignment-for-innersource-contribution.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

discover-your-innersource.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Patlet" is missing. Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

improve-findability.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Resulting Context" is missing. Required headline "Author(s)" is missing.]

introducing-metrics-in-innersource.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Status" is missing. Required headline "Author(s)" is missing.]

junkyard-styled-innersourcing.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Patlet" is missing. Required headline "Solutions" is missing. Required headline "Resulting Context" is missing. Required headline "Author(s)" is missing.]
modular-code.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Author(s)" is missing.]

overcome-acquisition-based-silos-developer.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

overcome-acquisition-based-silos-manager.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

review-committee.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

share-your-code-to-get-more-done.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

shared-code-repo-different-from-build-repo.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

start-as-experiment.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Solutions" is missing. Required headline "Author(s)" is missing.]

  4. spier commented on May 5, 2020

    @spier
    spier
    on May 5, 2020
    MemberAuthor

    I built a version that accepts multiple different spellings of Author/Solution.

    The linter now complains about a much smaller number of issues.

    $ markdownlint -c lint/pattern-template.yml -r lint/pattern-template.js --ignore-path lint/.markdownlintignore *.md
30-day-warranty.md:1 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Title"]
30-day-warranty.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Title" is missing. Required headline "Patlet" is missing. Required headline "Problem" is missing. Required headline "Context" is missing. Required headline "Forces" is missing. Required headline "Solutions" is missing. Required headline "Resulting Context" is missing. Required headline "Status" is missing. Required headline "Author(s)" is missing.]
30-day-warranty.md:5 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Context"]
30-day-warranty.md:11 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Problem"]
30-day-warranty.md:15 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Forces"]
30-day-warranty.md:41 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Solution"]
30-day-warranty.md:51 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Resulting Context"]
30-day-warranty.md:58 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Known Instances"]
30-day-warranty.md:62 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Authors"]
30-day-warranty.md:66 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Acknowledgement"]
30-day-warranty.md:73 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Status"]
30-day-warranty.md:79 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Variants"]
change-the-developers-mindset.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Author(s)" is missing.]
crossteam-project-valuation.md:1 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Title"]
crossteam-project-valuation.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Title" is missing. Required headline "Patlet" is missing. Required headline "Problem" is missing. Required headline "Context" is missing. Required headline "Forces" is missing. Required headline "Solutions" is missing. Required headline "Resulting Context" is missing. Required headline "Status" is missing. Required headline "Author(s)" is missing.]
crossteam-project-valuation.md:5 PATTERN-TEMPLATE-RULE-001 No h1 headlines [Use of #-headlines (h1) is not allowed in patterns] [Context: "# Patlet"]
discover-your-innersource.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Patlet" is missing.]
improve-findability.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Resulting Context" is missing.]
introducing-metrics-in-innersource.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Status" is missing.]
junkyard-styled-innersourcing.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Patlet" is missing. Required headline "Resulting Context" is missing.]

    I have not been able to run this correctly as a GitHub Action yet. So far the linter also validates files like README.md which is not very useful. Not sure if you can see that, but the output of the GitHub action is visible here on my fork: https://github.com/spier/InnerSourcePatterns/runs/645897295?check_suite_focus=true

  5. spier commented on May 8, 2020

    @spier
    spier
    on May 8, 2020
    MemberAuthor

    After some quick fixes to two patterns, the outstanding things that the linter complains about are:

    $ markdownlint -c lint/pattern-template.yml -r lint/pattern-template.js --ignore-path lint/.markdownlintignore *.md
30-day-warranty.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Patlet" is missing.]
change-the-developers-mindset.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Author(s)" is missing.]
crossteam-project-valuation.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Problem" is missing. Required headline "Context" is missing. Required headline "Forces" is missing. Required headline "Solutions" is missing. Required headline "Resulting Context" is missing. Required headline "Status" is missing. Required headline "Author(s)" is missing.]
discover-your-innersource.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Patlet" is missing.]
improve-findability.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Resulting Context" is missing.]
introducing-metrics-in-innersource.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Status" is missing.]
junkyard-styled-innersourcing.md:1 PATTERN-TEMPLATE-RULE-003 Mandatory template sections [Required headline "Patlet" is missing. Required headline "Resulting Context" is missing.]

  6. spier commented on May 8, 2020

    @spier
    spier
    on May 8, 2020
    MemberAuthor

    Especially the issues where a Patlet is missing will take some time to fix. Writing the Patlets requires to read and condense the whole pattern in a 2-sentence structure. Also getting the PRs for those fixes merged will take some time, as we want to make sure that the Patlet represents the pattern correctly.

    In the next days I will send a PR with the code for the custom markdownlinter rules that I use to validate the pattern syntax as shown above.

  7. spier commented on May 10, 2020

    @spier
    spier
    on May 10, 2020
    MemberAuthor

    This work-in-progress PR shows how the markdown validations would look like in a PR.
    https://github.com/InnerSourceCommons/InnerSourcePatterns/pull/168/files#diff-ebf8d058832d99bac8c6533c359a50c5

    pr-annotations

  8. spier
    added
    ⚙️ Type - MetaImproving how we collaborate in this repo is the main focus of this issue / PR
    on Dec 28, 2020

  9. spier commented on Jan 29, 2021

    @spier
    spier
    on Jan 29, 2021
    MemberAuthor

    This has been implemented in #257. Closing issue.

  10. spier
    closed this as completedon Jan 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ⚙️ Type - MetaImproving how we collaborate in this repo is the main focus of this issue / PR

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions