diff --git a/static/pages/Set-up-self-hosted-CI-CD-git-pipelines-with-Platypush.md b/static/pages/Set-up-self-hosted-CI-CD-git-pipelines-with-Platypush.md index b20a61c..ca99120 100644 --- a/static/pages/Set-up-self-hosted-CI-CD-git-pipelines-with-Platypush.md +++ b/static/pages/Set-up-self-hosted-CI-CD-git-pipelines-with-Platypush.md @@ -23,7 +23,7 @@ Github has [too often taken the wrong side](https://www.bleepingcomputer.com/news/software/github-threatens-to-ban-users-who-bypass-youtube-dl-takedown/) in DMCA disputes since it's been acquired by Microsoft. The [CEO of Github has in the meantime tried to redeem himself](https://devrant.com/rants/3366288/seems-like-even-githubs-ceo-has-now-chimed-in-on-the-youtube-dl-takedown-https-t), -but the damage in the eyes of many developers, myself included, was done, all in spite of the friendly olive branch +but the damage in the eyes of many developers, myself included, was done, despite the friendly olive branch to the community handed over IRC. Most of all, that doesn't change the fact that Github has [taken down more than 2000 other repos](https://github.com/github/dmca/tree/master/2020) in 2020 alone, often without any appeal or legal support - the CEO bowed down in the case of `youtube-dl` only because of the massive @@ -31,13 +31,13 @@ publicity that the takedown attracted. Moreover, Github has yet to overcome its it advertises itself like the home for open-source software, but its own source code is not open-source, so you can't spin up your own instance on your own server. There's also increasing evidence in support of my initial suspicion that the Github acquisition was nothing but another -old-school [Microsoft triple-E operation](https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish). When you want -to clone a Github repo you won't be prompted anymore with the HTTPS/SSH link by default, you'll be prompted with the -Github CLI command, which extends the standard `git` command, but it introduces a couple of naming inconsistencies here -and there. They could have contributed to improve the `git` tool for everyone's benefit instead of providing their new -tool as the new default, but they have opted not to do so. I'm old enough to have seen quite a few of these examples in -the past, and it never ended well for the _extended_ party. As a consequence of these actions, I -have [moved the Platypush repos to a self-hosted Gitlab instance](https://git.platypush.tech/platypush) - which comes +old-school [Microsoft triple-E operation](https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish). Nowadays, when +you want to clone a Github repo you won't be prompted anymore with the HTTPS/SSH link by default. You'll be prompted +with the Github CLI command, which extends the standard `git` command, but it introduces a couple of naming +inconsistencies here and there. They could have contributed to improving the `git` tool for everyone's benefit instead +of providing their new tool as the new default, but they have opted not to do so. I'm old enough to have seen quite a +few of these examples in the past, and it never ended well for the _extended_ party. As a consequence of these actions, +I have [moved the Platypush repos to a self-hosted Gitlab instance](https://git.platypush.tech/platypush) - which comes with much more freedom, but also no more Github actions. And, after the announcement of the @@ -75,17 +75,17 @@ In this article we'll how to use Platypush to set up a pipeline that: - Runs a suite of tests. -- If the tests succeed, then it proceeds with packaging the new version of the codebase - in my case, I run the +- If the tests succeed, it proceeds with packaging the new version of the codebase - in my case, I run the automation to automatically create the new [`platypush-git`](https://aur.archlinux.org/packages/platypush-git) package - for Arch Linux on new pushes, and the new [`platypush`](https://aur.archlinux.org/packages/platypush) Arch package and - the [`pip` package](https://pypi.org/project/platypush/) on new tags. + for Arch Linux on new pushes, and the new [`platypush`](https://aur.archlinux.org/packages/platypush) Arch package as + well as the [`pip` package](https://pypi.org/project/platypush/) on new tags. - If the tests fail, it sends a notification (over [email](https://docs.platypush.tech/en/latest/platypush/plugins/mail.smtp.html), [Telegram](https://docs.platypush.tech/en/latest/platypush/plugins/chat.telegram.html), [Pushbullet](https://docs.platypush.tech/en/latest/platypush/plugins/pushbullet.html) or - [whichever plugin supported by Platypush](https://docs.platypush.tech/en/latest/)). Also send a notification if the - latest run of tests has succeeded and the previous one was failing. + [whichever plugin supported by Platypush](https://docs.platypush.tech/en/latest/)). It also sends a notification if + the latest run of tests has succeeded and the previous one was failing. Note: since I have moved my projects to a self-hosted Gitlab server, I could have also relied on the native Gitlab CI/CD pipelines, but I have eventually opted not to do so for two reasons: @@ -152,7 +152,7 @@ backend.http: Gitlab webhooks are a very simple and powerful way of triggering things when something happens on a Gitlab repo. All you have to do is setting up a URL that should be called upon a repository event (push, tag, new issue, merge request -etc.), and set up an automation on the endpoint that reacts to the event. +etc.), and set up a piece of automation on the endpoint that reacts to the event. The only requirement for this mechanism to work is that the endpoint must be reachable from the Gitlab host - it means that the host running the Platypush web service must either be publicly accessible, on the same network or VPN as the @@ -388,7 +388,7 @@ And here you go - you should now be ready to create your automation routines on Even though I have moved the Platypush repos to a self-hosted domain, I still keep a mirror of them on Github. That's because lots of people have already cloned the repos over the years and may lose updates if they haven't seen the announcement about the transfer. Also, registering to a new domain is often a barrier for users who want to create -issues. So, even though me and Github are no longer friends, I still need a way to easily mirror each new commit on my +issues. So, even though I and Github are no longer friends, I still need a way to easily mirror each new commit on my domain to Github - but you might as well have another compelling case for backing up/mirroring your repos. The way I'm currently achieving this is by cloning the main instance of the repo on the machine that runs the Platypush service: @@ -511,10 +511,10 @@ def on_repo_push(event, **_): Upon push event, the latest version of the repo will be cloned under `/tmp/repo` and the suite of tests will be run. The output of each session will be stored under `/var/log/tests` in a file formatted like -`_.log`. To make things even more robust, you can create a new virtual environment under +`_.log`. To make things even more robust, you can create a new virtual environment under the temporary directory, install your repo with all of its dependency in the new virtual environment and run the tests from there, or spin a Docker instance with the required configuration, to make sure that the tests would also pass on a -fresh installation and prevent the _"but if works on my box"_ issue. +fresh installation and prevent the _"but it works on my box"_ issue. ## Serve the test results over HTTP @@ -526,7 +526,7 @@ $ cd /var/log/tests $ python -m http.server 8000 ``` -The logs will be served on `http://host:8000`. You can also serve the directory through a proper web server like nginx +The logs will be served on `http://host:8000`. You can also serve the directory through a proper web server like Nginx or Apache. ![CI logs over HTTP](../img/ci-1.png) @@ -563,12 +563,12 @@ relying on any cloud runner. ## Automatic build and test notifications -Another useful feature of the popular cloud services is the ability to send notification when a build status changes. -This is quite easy to set up with Platypush, as the application provides several plugins for messaging. Let's look at -an example where a change in the status of our tests triggers a notification to our Pushbullet account, which can be -delivered both to our desktop and mobile devices. [Download the Pushbullet app](https://pushbullet.com) if you want the -notifications to be delivered to your mobile, get an [API token](https://docs.pushbullet.com/) and then install the -dependencies for the Pushbullet integration for Platypush: +Another useful feature of most of the popular cloud services is the ability to send notifications when a build status +changes. This is quite easy to set up with Platypush, as the application provides several plugins for messaging. Let's +look at an example where a change in the status of our tests triggers a notification to our Pushbullet account, which +can be delivered both to our desktop and mobile devices. [Download the Pushbullet app](https://pushbullet.com) if you +want the notifications to be delivered to your mobile, get an [API token](https://docs.pushbullet.com/) and then install +the dependencies for the Pushbullet integration for Platypush: ```shell $ [sudo] pip install 'platypush[pushbullet]' @@ -717,15 +717,15 @@ And here you go - you now have an automated way of building and releasing your a ## Continuous delivery of web applications We have seen in this article some examples of CI/CD for stand-alone applications with a complete test+build+release -pipeline. The same concept also applies to web services and applications. If your repository stores the source code -of a website, then you can easily create automations that react to push events and pull the changes on the web server -and restart the web service if required. This is in fact the way I'm currently managing updates on the Platypush -[blog](https://blog.platypush.tech/) and [homepage](https://platypush.tech/). Let's see a small example where we have -a Platypush instance running on the same machine as the web server, and suppose that our website is served under -`/srv/http/myapp` (and, of course, that the user that runs the Platypush service has write permissions on this +pipeline. The same concept also applies to web services and applications. If your repository stores the source code of a +website, then you can easily create pieces of automation that react to push events and pull the changes on the web +server and restart the web service if required. This is in fact the way I'm currently managing updates on the +Platypush [blog](https://blog.platypush.tech/) and [homepage](https://platypush.tech/). Let's see a small example where +we have a Platypush instance running on the same machine as the web server, and suppose that our website is served +under `/srv/http/myapp` (and, of course, that the user that runs the Platypush service has write permissions on this location). It's quite easy to tweak the previous hook example so that it reacts to push events on this repo by pulling -the latest changes, runs e.g. `npm run build` to build the new `dist` files and then copies the `dist` folder to our -web server directory: +the latest changes, runs e.g. `npm run build` to build the new `dist` files and then copies the `dist` folder to our web +server directory: ```python import os