Automated releases in a cross-language monorepo
Monorepo is a popular way of organizing code bases in large organizations like Google, Facebook and Twitter. You can also find many open source projects that use a single git repository to store and develop multiple sub-packages. Some examples are Babel and Laravel.
If you are not familiar with the term “monorepo”, check out this 5 min video from Fabien Potencier, creator of Symfony framework.
I personally have been using this technique for many years without knowing its name. The idea is very simple: you store your
frontend/ folders in one repository. This is a natural approach for full stack developers who usually touch both folders on a daily basis. It is very handy to have entire project codebase in one place when you develop and test the application. Below just a few benefits that came to my mind:
- Fewer repositories to maintain on GitHub (less
git pullcommands to execute)
- Fewer steps to install full application on local machine
- Less CI configuration to spin up and test the entire project
- No mocking needed (all services are installed and available for testing without faking their APIs)
- Easier to deliver entire features that involve backend and frontend work
On the other hand, this approach isn’t applicable to all teams and all organizations. Read an article “Don’t ask if a monorepo is good for you – ask if you’re good enough for a monorepo” to learn more about monorepo disadvantages. It’s important to remember Conway’s law:
Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.
In case of SystemSeed, monorepos feel very natural because they reflect communication structures in a flat team of full stack developers. For different kind of organizations, setting up a monorepo may require significant changes in development & collaboration practices, which is not always worth it.
Step 1: install and configure semantic-release
From project description:
semantic-release automates the whole package release workflow including: determining the next version number, generating the release notes and publishing the package.
This removes the immediate connection between human emotions and version numbers, strictly following the Semantic Versioning specification.
In my case, I had to install semantic-release for both packages separately.
- Analyze all commit messages since previous release.
- Decide which version number to assign to the new release based on Angular Commit Message Conventions.
- Add a new git tag for this version and push the tag to the remote origin (along with changelog notes).
- Release to NPM (optional, switched off for PHP).
- Notify the team in Slack.
Step 2: Publishing to Packagist
To add a new package to Packagist you need to link a repository with a valid composer.json file in the repo root.
It means that I have to push PHP code from my monorepo to a separate PHP-only repo for Packagist to be happy.
To mirror monorepo to separate child repositories, I used splitsh-lite, the tool that is used by Laravel and Symfony for the same purpose. The tool finds commits that affected the package’s codebase and pushes them to another repository. Obviously, it preserves all commit information so that semantic-release can decide what kind of release to create for each package.
Step 3: Configuring CI
In SystemSeed, we’ve been using Circle CI long before I joined the team in 2015. I’ve never worked with other CI tools but they all seem to be quite similar.
You can find example setup for Circle CI in a demo project on GitHub.
CI setup contains two steps. First, on merging in master
splitsh-lite script is triggered. It mirrors new commits (of any) to the corresponding child repos.
- See Circle CI config here: https://github.com/kalabro/mixed-monorepo/blob/master/.circleci/config.yml#L13
./bin/split.shcode here: https://github.com/kalabro/mixed-monorepo/blob/master/bin/split.sh (inspired by the script in Laravel repo).
The second step is to trigger semantic-release for each child repo. Again, a simple Circle CI config handles it on every push into child repo: https://github.com/kalabro/mixed-monorepo/blob/master/js/.circleci/config.yml
You can find full monorepo setup for a real project here: https://github.com/systemseed/falcon
Falcon repo includes Drupal distribution on backend (falcon/ folder) and complementary React frontend (falconjs/ folder). The latter uses yarn workspaces inside to organize multiple JS packages & templates.
All you need to start developing is to clone the repo and run make install. When you are done and your pull request is merged in master, CI will figure out what to release and which version numbers to assign.
Give it a try and let me know your thoughts!