Twig PHP Engine in Pattern Lab node

[EvanLovely]

🎉 OK, here we go! This adds a Twig PHP Engine and implements it in the Twig Edition. This is really sweet because this will let us Unify Pattern Lab Core by bringing the most essential part of the PHP Pattern Lab world in: a real Twig environment using the actual engine.

I'll have to get into more later, but I've created @basalt/twig-renderer which is a JS module that takes config and gives back a render JS function that when called spins up an internal PHP server that exposes an API endpoint that renders Twig templates. It's all async and it can render 1,000 simple Twig templates in about 5s - and only spins up 1 PHP server for that @aleksip 😉

Take a peak @pattern-lab/devs and tell me what you think!

To test:

npm install
npm run bootstrap
cd packages/edition-twig
npm start

Bugs to fix:

• Header & foote _meta patterns not rendering on individual pattern page (but do on view all pages)

[aleksip]

Tried to install, but getting the same error as Travis:

npm ERR! enoent ENOENT: no such file or directory, rename '/home/travis/build/pattern-lab/patternlab-node/packages/engine-twig-php/node_modules/.staging/@basalt/twig-renderer-83c3d739/node_modules/@babel/code-frame' -> '/home/travis/build/pattern-lab/patternlab-node/packages/engine-twig-php/node_modules/.staging/@babel/code-frame-0e54f88a'

[EvanLovely]

🤔 hmmm... @bmuenzenmeyer - any ideas on above error?

[bmuenzenmeyer]

@EvanLovely will take a look, first glance doesn't ring any bells for me

[coveralls]

Coverage increased (+0.06%) to 74.127% when pulling a408bbc on feature/engine-twig-php into d7fa04d on dev.

[bmuenzenmeyer]

Fixed this - originally fixed in 5ab3995 after some research into lock-file behavior with lerna bootstraps being unreliable. Since we pin all dependencies, we should still have deterministic installations

[bmuenzenmeyer]

🎉 📢 ⚡️ 💣 💥
This is a pretty significant accomplishment in the history of Pattern Lab. Great work Evan! It's the culmination of so many discussions and technical hops.

[bmuenzenmeyer]

I don't like doing this - as npm has been known to break itself even with minor or patch releases. Since the .travis.yml file should be supplying Node versions, it also reliably gets the npm version bundled with Node

I think this was added as a troubleshooting step. If so, please remove.

[sghoweri]

Sorry, another PL Node newb question here: any particular reason why aren’t we using Yarn + Lerna Workspaces here?

[bmuenzenmeyer]

We focused on Lerna for our initial monorepo conversion. I don't want to get into the game of documenting everything with npm and yarn. npm 5+ has come a long way., blunting the original benefits of yarn to me. We do not use lockfiles (as you can see the problems in this PR) anyways, further blunting the need. I understand we could use yarn for dev and posture to users only npm, but I don't really understand the need for yet another tool. Please let me know if I am missing something here 😅

[hdwebpros]

Less tools is better IMO :)

[EvanLovely]

Was done as a troubleshooting step; removed now.

[bmuenzenmeyer]

What does this do? I cannot find this flag on https://github.com/lerna/lerna

[sghoweri]

Shouldn’t we move these to the lerna.json config in the root so these flags always get added?

[bmuenzenmeyer]

I'd be fine with moving it there in case someone has lerna installed globally and instead runs lerna bootstrap It sounds like you might have more experience with lerna, could you offer up a focused PR to do this @sghoweri ?

[sghoweri]

Definitely - happy to help! 👍🏼

[EvanLovely]

The --no-ci flag is part of v3, those docs are v2. https://github.com/lerna/lerna/tree/v3.0.0-beta.21#--ci

When I bumped npm from 5 to 6, lerna uses npm ci instead of npm install, and I had error messages surrounding it. Since I removed the Travis npm update, I also removed this tweak.

[bmuenzenmeyer]

This file is an artifact of sorts and should not be initially available as part of generation. It should not be ignored either,

[sghoweri]

Why can’t this be gitgnored?... Directly committing auto generated assets = 😰

[bmuenzenmeyer]

Let's think through this outloud - perhaps it could be? or perhaps not? My current thinking is that teams would want to check this in along with changes - so that another dev pulls down the changes and can run PL with incremental builds on - allowing PL to only build what's changed. If this is gitignored, users pulling down changes will have incorrect dependencyGraphs upon build, leading to odd behavior. Perhaps we need more testing around how this works for teams.

[hdwebpros]

I'm not following @bmuenzenmeyer 's logic. We work with usually 2 devs here doing Patternlab. 99% of the time we are editing patterns, assets, or corresponding json. If one of the devs pushes changes, we just pull and then start where they left off. I don't see how this autogenerated file would help.

My $0.02 -- If they push changes, this file would autogenerate regardless because I'd have to run PL regardless. I don't see a case for pushing this file so that someone could investigate this file and see the state that everything is in.

In fact, if someone really wanted to see this, they could edit their own .gitignore to allow it.

That's my $0.02, not sure if I'm helping or not here

[tburny]

As the person who implemented that feature:
This file caches the current build status of each pattern, so if either the data or the meta data changes after the last build, the file gets recompiled incrementally with all descendant patterns.

It can be regarded as a local build cache and thus should be added to .gitignore.

The alternative would be hell:
Version patterns, data and the generated output files with correct modification timestamps. This would lead to lots of potential merge conflicts in the dependency graph file when a few developers collaboratem, so
don't do this, ever.

[bmuenzenmeyer]

@hdwebpros

I don't see a case for pushing this file so that someone could investigate this file and see the state that everything is in.

Worth noting that no one would really look at this file manually.

@tburny

It can be regarded as a local build cache and thus should be added to .gitignore.

thanks for chiming in! ignore it is then

[bmuenzenmeyer]

Do you intend the Twig edition to contain the demo? This leads to a ton of duplicate code compared to starterkit-twig-demo. It's just a departure from the other editions, which have always been empty with the expectations that users install the starterkits separately.

Truer still, the CLI will now ask for the edition and then the starterkit, so baking them both into the edition reduces the value of the CLI process.

[bmuenzenmeyer]

This PR would be a ton smaller without all these files from the starterkit, too.

[bmuenzenmeyer]

can we pin these dependencies?

[EvanLovely]

done

[sghoweri]

Why can’t this be gitgnored?... Directly committing auto generated assets = 😰

[sghoweri]

Sorry, another PL Node newb question here: any particular reason why aren’t we using Yarn + Lerna Workspaces here?

[sghoweri]

Shouldn’t we move these to the lerna.json config in the root so these flags always get added?

[sghoweri]

@EvanLovely three big questions here:

1. Are these recursive by default?
2. How do we configure PHP Twig extensions?
3. For that matter, how would we configure any of the other Twig Renderer config options:
   https://github.com/basaltinc/twig-renderer/blob/master/config.schema.json

[EvanLovely]

1. Not by default, but that's just a recursive: true away. I want to update the Twig StarterKit demo to change things like @atoms/04-folder/01-item.twig to @atoms/01-item.twig as well.
2. Just set engines.twig.alterTwigEnv - you can see where the config is passed in here
3. I want to review the config a bit more on TwigRenderer before I lock it down (and release 1.0.0). Then we just pass more config in like the above link.

[sghoweri]

@EvanLovely woah woah woah - hold up! I’m like 99% sure we need to swap these out with out verbatim with the Twig versions of these (and add this to the monorepo here):

I already went ahead and ported these original Twig Templates over/ updated to work with the latest CSS / JS changes in the UIKit Workshop if this helps:
https://github.com/bolt-design-system/bolt/tree/master/packages/styleguidekit-twig-default

[EvanLovely]

I think it's working right now as this supports rendering Twig and Mustache files. I'm fine with getting Twig ones in there as well.

[hdwebpros]

I did a light test of this. It seems initially to work great.

[EvanLovely]

OK, all crucial feedback from above has been addressed. There's one semi-major thing left to consider: Twig Namespaces & Starterkits.

Since we don't automagically figure out where the atoms-button template is (nor do we want to), we use Twig Namespaces like @atoms/button.twig which let's us declare that @atoms refers to an array of directories that should be looked in for the file button.twig. This is a very good thing as Twig Namespaces is core to Twig and can be implemented in any other system that uses Twig - so for example in Drupal we register @atoms to look in the same directories as PL does and then we can use the same templates in both systems.

As this PR currently stands, the patternlab-config.json file contains this chunk that is passed into the Twig engine and then it sets up the Namespaces:

{
  "engines": {
    "twig": {
      "namespaces": [
        {
          "id": "atoms",
          "paths": [
            "source/_patterns/00-atoms"
          ]
        },
        {
          "id": "molecules",
          "paths": [
            "source/_patterns/01-molecules"
          ]
        },
        // more namespace config...
      ]
    }
  },
  "cacheBust": true,
  // more pl config...
}

And so if the edition-twig has the patternlab-config.json file, but it can install any StarterKit, which is the collection of files that contain the folders source/_patterns/00-atoms for @atoms and source/_patterns/01-molecules for @molecules, but a different StarterKit could want to use @components or @layouts, then how do we config those?

It seems to me that a Twig Starterkit should come with some config for Namespaces and that's where this config should live, not edition-twig. Let's discuss.

[cybtachyon]

I've always used Twig namespaces as a library mechanism in previous projects: @myComponentLib/00-atoms/button/button.twig @myOtherLib/01-molecules/button/button.twig, @patternfly/patterns/dropdown/dropdown.hbs etc. etc.

To me you'd want to configure namespaces in a central location where it's easy to require uniqueness.

[sghoweri]

@EvanLovely anything I can do to help with this one? What’s the next step here?

[EvanLovely]

Got this all working!!

Main issues addressed. Should be stable enough to merge down to dev and iterate if needed.

[EvanLovely]

Released as @pattern-lab/engine-twig-php at version 3.0.0! (former php twig engine was 2.x.x). Wow! Anyone who want to play with this can start by copying out the ./packages/edition-twig folder in this repo, adjusting the config to account for the different ui-kit paths and it all works!

It's basically the difference between this:

../uikit-workshop/views-twig

and this:

node_modules/@pattern-lab/uikit-workshop/views-twig

That's all I needed to change and then it worked! Any thoughts on how to fix it? Allowing config files to be .js instead of only .json would fix it. Anyone want to tackle?

[sghoweri]

Released as @pattern-lab/engine-twig-php at version 3.0.0! (former php twig engine was 2.x.x). Wow! Anyone who want to play with this can start by copying out the ./packages/edition-twig folder in this repo, adjusting the config to account for the different ui-kit paths and it all works!

It's basically the difference between this:

../uikit-workshop/views-twig

and this:

node_modules/@pattern-lab/uikit-workshop/views-twig

That's all I needed to change and then it worked! Any thoughts on how to fix it? Allowing config files to be .js instead of only .json would fix it. Anyone want to tackle?

Thanks @EvanLovely!! I’d be happy to tackle this one — can’t imagine this’ll be super tricky.