The 2023 editon of the anual Plone conference happend from October 2nd to 8th in Eibar, Basque Country

The kitconcept team was present with 8 developers. Our team members gave plenty talks, keynotes and trainings.

kitconcept and friends having team dinner

Trainings

Two days filled with trainings. Free for all conference attendees. This offer is unique in the Plone community and kitconcept team members were the backbone of the trainings about how to use Plone 6. From deployment to development to deep insides into how Volto and Plone 6 works.

Alok Kumar, Jakob Kahl: Volto and React

Alok Kumar and Jakob Kahl did a two day training to help developers get started with Volto and React:

https://2023.ploneconf.org/schedule/training/volto-and-react

Check out their trainings online if you want to catch up:

• React Training

• Volto Hands-On Training

Érico Andrei : Installing Plone

Our colleague Érico Andrei gave a training about installing Plone on Day 2, the 3rd of October

https://2023.ploneconf.org/schedule/training/installing-plone

Víctor Fernandez de Alba, Tiberiu Ichim: Effective Volto

Víctor Fernandez de Alba is kitconcept CTO and the Volto Release Manager. Tiberiu Ichim is a Volto team member and one of the key contributors to Volto. They gave key insights into how to work effectively with Volto. If you have experience with Volto and you want to become a real pro, this is the training you should go to.

https://2023.ploneconf.org/schedule/training/effective-volto

https://training.plone.org/effective-volto/index.html

Day One

On day one, kitconcept team members presented two talks, including the main keynote of the day.

Keynote State of Plone

Team members Érico Andrei, Víctor Fernández de Alba and Timo Stollenwerk together with Maurits van Rees of Zest Software and Eric Steele of Salesforce presented the very first Keynote of the Conference titled “State of Plone”.

Breaking boundaries: Plone as headless CMS by Víctor Fernández de Alba

Our colleague Víctor Fernández de Alba gave a presentation about the challenges faced by the Plone content management system in keeping up with modern frontend developments and the growing popularity of headless CMSs.

Breaking boundaries: Plone as headless CMS

https://2023.ploneconf.org/schedule/breaking-boundaries-plone-as-headless-cms

Day Two

Day Two was a informative Day, packed with interesting Talks, Panels and Presentations.

Volto Team Meeting

Panel: The Future of Search in Plone, 2023 Edition

Timo Stollenwerk, Sally Kleinfeldt, Tiberiu Ichim,, Eric Steele, Eric Bréhault, Rikupekka Oksanen, Érico Andrei and Guido Stevens hosted a very interesting Panel about the Future of Search Algorithms in Plone.

This panel provided a brief history and modern examples of Plone search, followed by a discussion of what improvements are needed - both from a marketing and technical perspective. This topic was first discussed at the 2011 conference and it was interesting to see how opinions had changed.

Alok Kumar : Is your Volto addon developer friendly ?

Meanwhile, kitconcept frontend developer Alok Kumar held a Presentation about what makes a developer friendly Volto Addon, and how we as a developer ourselfes can improve on the way we develop addons for Volto.

https://2023.ploneconf.org/schedule/is-your-volto-addon-developer-friendly

Rob Gietema : How to build a site using Nick

Later in the afternoon kitconcept developer Rob Gietema held an intriguing Talk about Nick, a headless CMS written in Node.js and how easy it is to build a website with it.

https://2023.ploneconf.org/schedule/how-to-build-a-site-using-nick

David Glick : Tales from a production Plone Cluster

Following Rob, kitconcept Employee David Glick shared some Details and Stories on hosting large Plone sites in a Docker Swarm Cluster.

https://2023.ploneconf.org/schedule/tales-from-a-production-plone-cluster

Érico Andrei : Unlocking the Power of plone.distribution : A Hands-On Guide

In this talk, Érico Andrei guided us through the feature-rich world of Plone Distributions.

https://2023.ploneconf.org/schedule/unlocking-the-power-of-plone-distribution-a-hands-on-guide

Local sport showcase and party

In the evening CodeSyntax organized a showcase of different local sports, including stone lifting, wood chopping and wood sawing. Timo represented kitconcept in this together with Phillip Bauer of Starzel. After that we concluded the day with cold drinks and Pinxos at the conference party.

Day 3

Day 3 was filled with quite technical presentations, providing Information on the Cutting Edge Technology Plone has to offer.

Fred van Dijk : How the Plone Foundation Ai.team manages its websites with CI/CD

On the third Day of the Plone Conference, kitconcept Employee Fred van Dijk shared the News on automating a plone Release and how to host and operate a small Docker Swarm cluster using Plone.

https://2023.ploneconf.org/schedule/how-the-plone-foundation-ai-team-manages-its-websites-with-ci-cd

Víctor Fernández de Alba : volto-light-theme: Volto Theming, Reimagined

After a quick coffee break Víctor Fernández de Alba shared the progress on the Volto-Light-Theme and its inner workings.

https://2023.ploneconf.org/schedule/volto-light-theme-volto-theming-in-2023

Timo Stollenwerk : How we built the Website for the German Aerospace Center (DLR) in less than six months

CEO Timo Stollenwerk indulged us in the Story of the Challenges of migrating large, goverment-owned websites into a Plone project.

https://2023.ploneconf.org/schedule/how-we-built-the-website-for-the-german-aerospace-center-dlr-in-less-than-six-months

Érico Andrei : Testing your Plone codebase with Pytest

A little later in the afternoon, Érico Andrei presented us with a better, improved way to test Plone codebases.

https://2023.ploneconf.org/schedule/testing-your-plone-codebase-with-pytest

Rohit Kumar : Workflow Manager with Volto

In his presentation Rohit Kumar shared the progress regarding implementing a visual workflow Manager in Volto.

https://2023.ploneconf.org/schedule/workflow-manager-in-volto

Summary

The kitconcept team continues to drive innovation in the Plone community. Volto is the default frontend for Plone 6 and dominated the topics during the conference. We are happy to be part of such an amazing community.

The 2022 editon of the anual Plone conference happend from October 10th to 16th in Namur, Belgium.

The kitconcept team was present with nine developers. Our team members gave three trainings, two keynotes, and six talks.

The kitconcept team in action at #ploneconf2022. pic.twitter.com/wQV4qeVCxB

— kitconcept (@kitconcept_gmbh) October 13, 2022

Trainings

Two days filled with trainings. Free for all conference attendees. This offer is unique in the Plone community and kitconcept team members were the backbone of the trainings about how to use Plone 6. From deployment to development to deep insides into how Volto and Plone 6 works.

Alok Kumar, Jakob Kahl: Volto and React

Alok Kumar and Jakob Kahl did a two day training to help developers get started with Volto and React:

https://2022.ploneconf.org/schedule/training/volto-and-react

Check out their trainings online if you want to catch up:

• React Training
• Volto Hands-On Training

Érico Andrei, Jens Klein: Installing Plone

Our colleague Érico Andrei gave a training about installing Plone with Jens Klein on day one about how to install Plone:

https://2022.ploneconf.org/schedule/training/installing-plone

Victor Fernandez de Alba, Tiberiu Ichim: Effective Volto

Victor Fernandez de Alba is kitconcept CTO and the Volto Release Manager. Tiberiu Ichim is a Volto team member and one of the key contributors to Volto. They gave key insights into how to work effectively with Volto. If you have experience with Volto and you want to become a real pro, this is the training you should go to.

https://2022.ploneconf.org/schedule/training/effective-volto

https://training.plone.org/2022/effective-volto/index.html

Day One

On day one, kitconcept team members presented three talks, including the main keynote of the day.

Keynote Érico Andrei

Plone Foundation president, and our teammate, Érico Andrei kicked off the first day of the conference with a keynote about the Plone community.

Case Study by Timo Stollenwerk

kitconcept CEO Timo Stollenwerk presented a case study about how kitconcept relaunched hundreds of websites with the Plone 6 stack.

Plone at Scale: How Plone Powers Hundreds of Websites at one of the Largest Research Institutions in Europe

Alok Kumar

Alok Kumar presented “A Deep Dive Into Internals Of Volto” where he explained the different configuration options in Volto.

Day Two

Plone 6 beyond 2022

Volto Release Manager Victor Fernandez de Alba presented the roadmap for Plone 6 and Volto beyond 2022.

https://2022.ploneconf.org/schedule/talks/plone-6-beyond-2022

David Glick: The past, present, and future of how we test Plone

https://2022.ploneconf.org/schedule/talks/the-past-present-and-future-of-how-we-test-plone

Érico Andrei: Brazil still loves Plone

30 Million visitors on a single day. 670k content objects. If you are running a Plone site at scale you definitely want Érico Andrei on your team. He helped the Electoral Justice department in Brazil to run and scale Plone for the upcoming elections.

https://2022.ploneconf.org/schedule/talks/brazil-still-loves-plone

Rob Gietema: Nick: A Nearly Headless CMS

Rob Gietema started plone-react together with Roel Bruggink at the Beethoven Sprint 2017 in Bonn at the kitconcept office. This first became Volto and later Plone 6.

For Rob, this isn’t enough. Check out his latest “pet project” Nick CMS. A NodeJS-based CMS headless CMS backend that works flawlessly with Volto.

https://2022.ploneconf.org/schedule/talks/nick-a-nearly-headless-cms

Day 3

Timo Stollenwerk, Victor Fernandez de Alba: State of Plone

Plone Release Managers Maurits van Rees, Eric Steele, Victor Fernández de Alba, and Timo Stollenwerk started day three of the Plone Conference with the third keynote about “The State of Plone”:

https://2022.ploneconf.org/schedule/talks/state-of-plone

Summary

The kitconcept team continues to drive innovation in the Plone community. Volto is the default frontend for Plone 6 and dominates the topics during the conference. We are happy to be part of such an amazing community.

Volto continues to innovate at a fast pace towards Plone 6. Today we released another major milestone on our road to Plone 6: Volto 14.

Volto 14 was in the making since September 2021 and it is in active use in various projects at Eau de Web, Red Turtle, Rohberg, kitconcept, and others.

Volto 14 comes with a set of new exiting features: a new search block that supports faceted search, locking support, a new seamless mode that makes deploying Volto easier, a new mobile navigation and support for Node 16.

Faceted Search Block

The new search block allows editor to create sophisticated faceted searches through the web without writing a single line of code.

Editors can define criteria for the content that is listed, like in the existing listing block in Volto.

Then editors can then choose arbitrary facets that are displayed to the users to choose from to narrow down the search.

Locking

Locking in a Content Management System is a mechanism to prevent users from accidentially overriding each others changes.

When a user edits a content object in Plone, the object is locked until the user hits the save or cancel button. If a second user tries to edit the object at the same time, she will see a message that this object is locked.

Content object in Volto 14 that is locked by another user

Locking requires at least plone.restapi 8.9.0 or plone.restapi 7.4.0 to be installed.

Seamless Mode

The new “seamless mode” allows zero configuration deployment by avoiding hardcoded environment variables in builds involved, and establishing good sensible defaults when setting up deployments (and also in development). So the developer/devOps doesn’t have to overthink their setups.

These are its main features:

• Runtime environment variables
• Unified traversal ++api++
• Use Host header to auto-configure the API_PATH

and these immediate benefits:

• Avoid having to expose and publish the classic UI if you don’t really need it
• If possible, avoid having to rewrite all API responses, since it returns paths that do not correspond to the original object handled and “seen” from Volto, so you have to adjust them (via a code helper) in a lot of call responses.
• Simplify Docker builds, making all the configuration via the runtime environment variables

Seamless Mode requires at least plone.rest 2.0.0a1 to be installed.

New Mobile Navigation

We polished the mobile navigation for Volto 14 to improve the user experience on mobile.

New mobile navigation in action on the Plone Conference 2021 website

Node 16

Volto 16 will support Node 16 in addition to Node 14 and Node 12. Node 16 will receive security updates until April 2024.

Node 16 release schedule from nodejs.org

Plone 6

Volto 14 is an important step towards Plone 6. Volto 15 will switch the default editor from DraftJS to Slate editor and it is planned to be ready in Q1/Q2 2022. This is the last big step for Volto before Plone 6 can be released.

Two weeks ago we cut two important releases on the road to Plone 6. Plone REST API 7 introduces a new link integrity feature for blocks-based pages. Volto 12 improves the add-on ecosystem by introducing a new configuration registry to avoid circular dependencies.

Photo by Stephen Leonardi on Unsplash

Plone REST API 7

Linking pages is one of the core idea of the world wide web. Keeping links within a website intact is therefore one of the core features that any Content Management System needs to provide.

One of the core features of Plone has always been that editors can copy and move single pages as well as large content trees without breaking internal links to other parts of the site. This is accomplished by using unigue IDs (UUIDs) instead of relative or absolute paths when adding a link to another page. Plone uses “portal transforms” internally to rewrite those links in RichText fields on save operations.

Plone 6 (aka Volto) introduces blocks-based page layouts, that store a JSON structure internally instead of HTML. This more structured way of storing page content and layout allows more complex page layouts. Though, because of this change, the existing portal transforms mechanism that rewrites links to UUIDs did not work any longer.

Blocks-based Volto page in edit mode (with kitconcept-blocks-grid)

Werkbank, a Plone agency from Bochum, Germany, stepped up to sponsor the development of link integrity in 2020, since they needed that for a client project.

I started to draft a possible solution and wrote a first prototype. Thomas Buchenberger from 4teamwork picked up that work at the Plone Conference sprint in Ferrara, Italy.

Andrea Cecchi from RedTurtle joined our efforts and refactored the resolveUID algorithm into a blocks transformer, that we started to use for other use cases and that made the resolveUID transformer more generic and flexible.

After that, we cut a first plone.restapi 7 alpha release and entered a longer period of quality assurance and testing.

Two weeks ago, after we tested the new feature in multiple projects at kitconcept, I cut a final release of Plone REST API 7.

After the first alpha release, the ResolveUID transformation was added to links and images. In addition plone.restapi 7 comes with a new blocks serialization mechanism and an important fix that makes sure files are opened directly by Plone for anonymous users.

We also added a “smart fields” concept that allows integrators to mark a blocks field as searchableText field.

A new @contextnavigation endpoint was added that allows for local navigations. We enhanced the navigation endpoint to expose an optional navigation title (“nav_title”) field.

Since the final 7 release, we cut six more releases and REST API 7 was included in Plone 5.2.4 release.

Volto 12

Photo by Gaelle Marcel on Unsplash

Tiberiu and Victor worked on a new Volto configuration registry which is the new central point to store and retrieve Volto configurations. The configuration registry is a singleton that ensures a setting only exists once. With more an more Volto add-ons that started to depend on each other, we started to run into circular dependencies issues that we can avoid now with the new configuration registry.

You can find more details about it in the Volto docs and in the Volto 12 upgrade guide.

Plone 6

Plone REST API 7 and Volto 12 were two very important releases on our road to Plone 6.

We plan to cut another Plone REST API 8 branch and release for Plone 6, which will support Python 3 only.

Volto will continue to move at very high pace towards Plone 6. We started to organize a series of Plone 6 “Micro-Sprints” to push things further.

Check out the Volto Roadmap on github for more details.

21 developers from nine different countries gathered in Bonn, Germany between June 20th and 24th to work on implementing the upcoming Plone 6. The sprint at the office of the kitconcept GmbH has been declared a “strategic sprint” by the Plone framework team. Sprint topics included working on Volto, the Plone REST API, and Guillotina.

Volto

Volto is a new ReactJS-based frontend for Plone. It was started by Rob Gietema in 2017 and it is actively developed since then. Volto will become the default frontend for Plone 6. It implements a complete new user interface called Pastanaga UI, which was developed by Albert Casado.

Victor Fernandez de Alba and Rob Gietema led the efforts to further enhance Volto together with Paul Roeland, Nicola Zambello, Piero Nicolli, Janina Hard, Jakob Kahl, Thomas Kindermann, Maurizio Delmonte, Stefania Trabucchi, Rodrigo Ferreira de Souza, Fred van Dijk, and Steffen Lindner.

Pastanaga UI Toolbar

The new editing toolbar has been part of Volto right from the start. Though, so far we did not fully implement every detail that Albert has imagined for the Pastanaga UI toolbar. Victor has been working on this for quite a while already. During the sprint, he integrated his work and polished it. Victor and Rob also worked on a new sidebar that shows up on the right side of the Volto user interface that holds additional information like page metadata and controls the settings of the individual tiles.

The new Volto edit toolbar

New Tiles: Collection, Slider and Table

At the Plone conference in Tokyo, we came up with a list of tiles for the new Volto composite page editor that we would like to implement. On top of that list were the listing/collection tile, the slider tile and the table tile. Piero, Rodrigo, and Rob started to work on the listing/collection tile that is supposed to become a replacement for the Collection Type in Plone.

Jakob and Janina worked on copying over the code of a slider tile that was developed by kitconcept. Rob implemented the table tile. Before you could ask Rob about the progress, he already finished his work, like always…

The new Volto table tile

Accessibility

Accessibility has always been a first class citizen in the Plone community. Plone is used by many government and public websites where accessibility is a basic requirement. Paul, Thomas, Timo, and Victor worked on improving the accessibility of Volto. Our goal is to fully support the WCAG 2.1 standard with the “AA” level of conformance.

Timo set up ESlint-based static code analysis checks and Cypress-based axe accessibility tests. Paul, Thomas and Victor then started to fix the reported accessibility issues. At the end of the sprint, we were able to fix all the reported violations. Our CI build now fails if new accessibility violations are committed.

Documentation

Sven Strack, the Plone documentation team leader, joined our sprint remotely from Amsterdam to discuss how to maintain and structure the Volto documentation in the future. We all agreed that we will use Markdown as format for our docs, since this is the de-facto standard in the JavaScript community as well as in most other Open Source communities. As much as we like Restructured Text, it seems that Markdown won that battle.

We currently use MKdocs for the Volto documentation and Styleguidist to document our React components. There are lots of different tools to choose from, Docz, MKDocs, MDX, Gatsby to just name a few. We agreed that we need to do more research to make an educated decision regarding our doc toolchain.

There will be a Google Season of Docs project this year where a technical writer will help us to enhance the Volto docs further. More Volto Features Nicola added a feature to display the currently used Volto version in the Plone control panel, he added animations for page transitions and added an Italian translation for Volto. Together with Victor, he worked on a toast component for user notifications.

More Volto Features

Piero worked on improving and fixing the event type view and added styling for the listing tile.

Victor and Steffen worked on the image sidebar.

Rodrigo added a feature that allows the user to use SHIFT+RETURN to create a new line in a Volto text. By default Volto creates a new text tile on RETURN. Rodrigo also looked into the Add-ons control panel work that Eric Steele started some time ago.

Nilesh joined us remotely from India and continued his work on the users and groups control panel for Volto.

Steffen fixed a bug in the subjects search. Rodrigo and Thomas looked into our Cypress-based acceptance tests.

Fred looked into what consequences it would have for us to get rid of the description tile and how we could customize the domain in Plone to allow to send portal emails via the Volto frontend.

Stefania and Maurizio worked on a product definition of Volto and an elevator pitch for clients and users. Together with Paul, Fred and Timo, they had a longer discussion about how to position Volto and Plone 6 in the market.

Plone REST API

The Plone REST API builds the foundation for the new Volto frontend and Plone 6. It has been under active development in the last four years and seen more than 80 releases in that time period. The REST API is stable, battle-tested and production-ready and part of the Plone core since the release of Plone 5.2.

Thomas Buchberger and Lukas Graf led the REST API efforts during the sprint and worked with Carsten Senger, Roel Bruggink, Janina Hard and Timo Stollenwerk to further enhance the REST API, fix bugs, improve the documentation, and the error handling.

Plone REST API discussion in the garden

Querystring Endpoint for the Collection Type

One of the most important new features, that has been developed during the sprint, is the new querystring endpoint. This endpoint will allow us to implement the Collection type for the Volto frontend. Lukas implemented this essential feature with help from Rob, who took care of the frontend implementation in Volto.

New Features, Bugfixes, and first time contributions

Thomas added support for the retrieval of additional metadata fields in summaries in the same way as in search results. Lukas made the @types endpoint expandable and the @users endpoint easier to customize. Janina fixed setting the effective date on workflow transitions. This was her first contribution to the Plone core. On the same day she signed the contributor agreement, got her fix merged and released, which is quite an accomplishment for a single day. Way to go Janina!

In addition to that, the team fixed lots of smaller bugs (#780, #729, #764, #755, #777, #732, #738) during the sprint. REST API Error handling Carsten, Nathan, Thomas, Lukas and Timo had a longer discussion about error handling in the Plone REST API and about how we can improve things. We considered and discussed implementing RFC7807 and decided against adopting it.

Sprint discussions…

The RFC did not see much adoption and it only slightly differs from what we are already doing. A breaking change does not seem to be worth the effort. We decided to create a separate error reporting component unifying the error reporting and ensuring consistency. We also started to fix a few inconsistencies right at the sprint.

API Documentation

Guillotina is using Swagger to document its API. Nathan explained the approach he took for that and Carsten started to integrate Swagger to generate dynamic api docs in plone.restapi.

REST API Releases

Most of the work we did on Plone REST API during the sprint has been released in Plone REST API 4.1.4, 4.2.0, 4.3.0, and 4.4.0.

Dexterity Site Root

Roel Bruggink joined us on Saturday and Sunday to continue his work on turning the Plone site root into a Dexterity content object. He fixed some complex recursion and acquisition problem when migrating the site root to Dexterity. His Plone Improvement Proposal (PLIP #2454) has been accepted by the Plone framework team and it will build the foundation for a new Plone 6 branch that we plan to cut this year.

Guillotina

Guillotina is a new AsyncIO REST data API. It has been written from the scratch by Ramon Navarro Bosch and Nathan van Gheem. It can easily scale billions of objects via its REST API. With the CMS addon, it is compatible with the Plone REST API. Therefore you can run Volto on a Guillotina backend and Guillotina might be able to replace our existing Plone backend in the future.

The Guillotina master thinking hard. :)

Nathan and Ramon worked on Guillotina 5. The Guillotina 5 release brought Python 3.7 support and the use of the context vars APIs. They refactored quite a bit, moved parts of addons into core(swagger, caching, pg catalog), implemented OOTB PostgreSQL catalog, added a Helm/Kubernetes/Docker configurations for the Guillotina CMS addon that can be configured to act as a Volto backend, worked on improving caching and filestorage.

Websockets for Plone

At the Plone Open Garden 2019, Asko Soukka started to work on bringing websocket support to Plone. Websockets allow bi-directional communication between the server and the browser. With websockets, the server can send messages and notifications to the browser. This can be used to implement notification systems or collaborative editing to just name a few possible applications.

Asko continued his work during the Beethoven sprint and created an example where comments that are added to Plone automatically pop up as toast notifications for other users. He published a Docker image with a Twisted-based ZServer and implemented the foundation to use that feature for gatsby-source-plone. This will allow us to create an extension that will allow editors to live-preview changes they do in Plone in a GatsbyJS site.

Karaoke, Barbecue and Garden Parties

A Plone sprint is much more than just programming. Collaborating, meeting old friends, talk, discuss and having a good time together are the ingredients that make a Plone sprint. We started our sprint on Thursday with a free React and Volto training for newbies. At the evening when everybody arrived, we went to a local Karaoke bar for a revival of the epic Karaoke sessions we had in Tokyo last year.

Karaoke at the “Nyx”

On Friday evening we ordered Pizza and hacked the night away at the office and our garden. On Saturday we went to the “Arithmeum”, a mathematics museum owned by the Research Institute for Discrete Mathematics at the University of Bonn.

Our tour at the “Arithmeum”

Afterwards we had a garden party with a barbecue and we spend the afternoon sitting in the garden and the office and continued to drink and hack on Plone.

Barbecue in the garden of the kitconcept office

Summary

The Beethoven Sprint 2019 was a major success. We were able to start working on the main missing pieces for Volto. The new edit toolbar is in place and the foundation for the new tiles/metadata toolbar on the right side of the UI has been finished. We also implemented a new table tile.

We implemented a proof-of-concept for the new listing / collection tile and implemented the required querystring endpoint in Plone REST API. Both Volto and the Plone REST API got lots of enhancements and bugfixes.

Both the Plone REST API and Volto are stable and used in production. We will use the next month to further enhance Volto and implement the missing bits and pieces. Our plan is to present Volto 4 at the Plone conference in Ferrara at the end of this year and cut a Plone 6 branch afterwards that contains the Dexterity Site root, folderish content types and other enhancements that are needed make Volto the default frontend for Plone 6.

Guillotina and the Websockets support for ZServer together with Gatsby as another frontend for Plone are amazing projects by the smartest folks in our community. Our diversity is one of our main strength and we are looking forward what the future will bring!

We had a great time at the sprint. Thank you to everyone who attended! We thank the Plone and Python community for their ongoing support. The Plone Foundation, the Python Software Foundation, and the German Python Software Verband made this event possible with their sponsorship as well as our company sponsors Abstract, Starzel, Werkbank and Zest.

We are looking forward to see you all in Bonn next year!

The “Hofgarten” near the kitconcept office

During the Plone Beethoven Sprint in Bonn, we worked hard on creating a first version of a new content editor for Plone-React.

Here is a short demo of what the editor looks like right now:

We already had a first implementation based on DraftJS that allows inline styles (e.g. bold, italic), block styles (headlines, (un)ordered lists), and links to remote URLs.

The new version of the editor is based on a “tiles” backend, that is build by Victor Fernandez de Alba during the sprint and released with plone.restapi 3.2.0.

This allows us adding more complex content elements such as images, videos, and in the future more complex layout elements.

With the new backend in place, Rob Gietema went ahead and implemented the basic editor that Albert Casado designed as part of the new Pastanaga UI for Plone.

Mobile Pastanaga Editor design by Albert Casado

The user can type in the title, description and the text content of the document without worrying about form fields or be distracted by tabs and fieldsets.

In addition to the standard text editing it is now possible to add an image tile that can be placed on the left or right side, on the center of the page or in full page width.

Rob also added a YouTube tile that allows the editor to add a YouTube video URL and then displays the video within the editor and the page view.

Text, image, and video tiles can be added to a page. They can be deleted and moved up and down to change the order of the elements.

Next Steps

The new editor is a great accomplishment. The tiles endpoint in plone.restapi allows us to further enhance the current version of the editor with more advanced layout variants and tiles.

We plan to continue with our iterative and agile approach of building a useful, fully functional version of the editor with each step, that allows Plone companies to use the editor and Plone-React today in their client projects.

The next steps are polishing the editor and the existing tiles. Work out some UX issues that we found when working with the editor and building more advanced tiles.

Stay tuned for more news and features after the Costa Brava sprint…

Report of the third day of the Plone Beethoven Sprint 2018 in Bonn, Germany.

The Beethoven Sprint was a “strategic” sprint that took place June 21-25 at the kitconcept office in Bonn, Germany. The focus of the sprint was to work on the Pastanaga editor, Plone-React, plone.restapi, and Guillotina. This is the report of the third day of the sprint.

Day three wrap-up meeting at the kitconcept office

UTC DateTime Discussion

After having breakfast and doing a quick stand-up, we started a discussion about how to store datetimes in Plone. This was a discussion that came up when we implemented plone.restapi. Though, this question is more related to plone.app.event. As seen many times in the Plone community, when it comes to complex technical matters, offline discussions work way better than discussing issues back and forth via github comments.

Since we had all the experts at the sprint, we decided to schedule a discussion to solve the issue. In the end we reached an agreement that dates should be stored as UTC and that the local time zone should be stored separately in an additional field, for instance to calculate recurring dates.

plone.restapi

Thomas Buchberger worked on the workflow endpoint and added a feature to change the workflow state recursively, to set the effective and expiration date and to allow workflow comment.

Lukas Graf finished his work on implementing redirects in plone.rest based on plone.app.redirector. The redirects do not only work for simple GET requests but also redirects POST, PATCH, und DELETE requests.

Sune Brøndum Wøller finished the portlets endpoint for plone.restapi.

Mikel Larreategi made plone.restapi return cachable image resources to improve the caching of images in plone.restapi.

Roel finally wrote a PLIP for the IPloneSiteRoot and fixed edge cases and a few bugs in the current implementation.

Plone-React

Victor completed his work on plone.schema, allowing to use a JSON field in Plone that validates the JSON structure within the field and also allows through-the-web editing. He also finished the backend behavior for tiles and blocks to store the tiles and the tiles layout on a Plone content object.

Eric Steele added a backend implementation for the add-ons control panel that he wrote for Plone-React. Eric presented a fully working add-ons control panel that allows to install add-ons via Plone-React by the end of the day.

Plone-React add-ons control panel presented by Eric Steele

David completed his work on the vocabularies endpoint by implementing the frontend widgets that rely on those vocabularies to make sure the endpoint serves its purpose.

Davi Lima and Victor continued to work on the override mechanism for Plone-React (JBOT) to customize widgets and views in a config.js file using a babel plugin.

Pastanaga Editor

Rob presenting the new video tile for the Pastanaga editor

As always, Rob did not give us any time to breath and added a YouTube video tile to the Pastanaga Editor, that shows a YouTube video in the editor itself and on the content view.

Of course that wasn’t enough for a day, so he fixed a few other smaller issues and worked on exposing the sitemap.xml via Plone-React.

Dinner and some late night hacking

Dinner at “Hans im Glück”

Rob might not admit it, but after claiming he could fix all those issues mentioned above in one day, he continued to work on the sitemap during our dinner at a local burger restaurant in the inner part of the city. We’ve been there ourselves, so we didn’t mind, while we were enjoying our tasteful cocktails. After a good meal, some drinks, and having to say good by to Eric, we went back to the office.

We said good bye again to our fantastic Dutch Plone-React team Rob and Roel and then went on to hack a little bit more on the stuff we were (and still are) so enthousiastic about.

Report of the second day of the Plone Beethoven Sprint 2018 in Bonn, Germany.

The Beethoven Sprint was a “strategic” sprint that took place June 21-25 at the kitconcept office in Bonn, Germany. The focus of the sprint was to work on the Pastanaga editor, Plone-React, plone.restapi, and Guillotina. This is the report of the second day of the sprint.

Guillotina

After having breakfast at the office, we started the day with a stand-up/wrap-up meeting with Ramon Navarro Bosch presenting Plone-React running on Guillotina. The user can add new content, edit existing content, and browse the content with Plone-React and Guillotina, which is a huge acomplishment and a promise for the future of Plone. After the stand-up Ramon had to say good bye to the other sprinters and head home to Barcelona.

Second day at #Plone #Beethoven sprint 2018! Standup time!! pic.twitter.com/qmkCP0K6oR

— kitconcept (@kitconcept_gmbh) June 23, 2018

Pastanaga Editor

Rob Gietema continued his work on the Pastanaga editor. At the end of the day, he was able to present a working version where the user can add a title and a description tile as well as a text tile with inline styles such as bold, italic, headlines, links, and lists.

Pastanaga Editor with basic text editing

Rob also added an image tile that allows you to upload images. The uploaded images then can be aligned left, right, in the middle and in full width or just be deleted.

Image tile that can be aligned left, right, center and displayed with full width.

Victor Fernandez de Alba finished his work on the tiles backend for the Pastanaga editor (https://github.com/plone/plone.restapi/pull/547), so Rob could present a fully functional editor that actually stores the content (text and images) in Plone tiles.

Plone-React

Eric Steele continued his work on the add-on control panel and was able to present a first version at the wrap-up at the end of the day.

Add-ons control panel written in React by Eric Steele

Carsten Senger worked on implementing the users and groups control panel in Plone-React which now allows to add and delete users.

Victor worked on being able to use Plone-React as a library that allows developers to override React components. The system is supposed to work the same way as JBOT (just-a-bunch-of-templates) works in Plone.

Johannes Raggam and Andrea Cecchi continued their work on the reference widget and were able to present a first prototype. We agreed that we have to put some effort into the UX/UI of that widget, but that was beyond the scope of a sprint and requires the help of a UX specialist.

plone.restapi

Thomas Buchberger finished the pull request for the object create order and started to to work on the workflow endpoint enhancements.

David Glick worked on enhancing the vocabularies endpoint with batching and filtering.

Sune Brøndum Wøller fixed some nasty bugs with time freezing and transactions errors in the Plone 5.2 of plone.restapi (e.g. https://github.com/plone/plone.restapi/pull/562, https://github.com/plone/plone.restapi/pull/561).

Lukas Graf continued his work on the translations of the REST response data, fixed a few bugs and wrote a script that tests the Sphinx-based documentation for warnings and errors and fails the Travis build if there is a problem.

Mikel Larreategi finished the pull request for the history endpoint and documented the Accept-Language headers in plone.restapi. He also started working on making the image scales that plone.restapi returns cachable.

Roel Bruggink continued his quest on the folderish site root and fixed errors in CMFPlone, plone.app.content, and in plone.app.dexterity.

We were able to release plone.rest 1.1.1 and plone.restapi 2.1.0 as well as plone.schema 1.2.0 with a JSON field that is required for the tiles endpoint in plone.restapi.

Hacking Night

Germany was playing at the soccer world cup in the evening, so we decided to order Pizza for dinner and then split up between people that wanted to code and people that wanted to watch the game. As you might have guessed the latter group was rather small. We ended up in a bar because the public viewing was already full. Though I guess the sprinters got a taste of the German soccer culture. After the game we went back to the office to join forces with the others again to wrap up the day and hack the night away.

Report of the first day of the Plone Beethoven Sprint 2018 in Bonn, Germany

The Beethoven Sprint was a “strategic” sprint that took place June 21-25 at the kitconcept office in Bonn, Germany. The focus of the sprint was to work on the Pastanaga editor, Plone-React, plone.restapi, and Guillotina. This is the report of the first day of the sprint.

Stand-up

Stand-up of the first day

We started the day with a stand-up meeting giving people a heads up on the current state of affairs of plone.restapi, plone-react, and Guillotina.

I started with plone.restapi, which is considered stable and battle tested in production.

Victor Fernandez de Alba then gave a brief introduction to our first Plone-React based project VHS-Ehrenamtsportal, that we successfully shipped to our client a few weeks ago and run without any issues since then.

Victor introducing VHS Ehrenamtsportal

After this, Rob Gietema gave a short introduction to the current state of Plone-React.

Last but not least, Ramon Navarro Bosch presented “Guillotina”, an async server written in Python with a Cockroach DB / ElasticSearch backend that adopts some of the core concepts of Zope and Plone.

With a group of 15 sprinters, we decided to split up in four different groups for the main sprint topic. Thomas Buchberger led the plone.restapi group, Rob the Plone-React group, Victor the Pastanaga Editor group, and Ramon the Guillotina group.

Pastanaga Editor

Rob Gietema going though the different approaches we discussed during the Tiles planning meeting

Right after the stand-up, we had a longer discussion about the “tiles” endpoint in plone.restapi and the editor implementation in Plone-React. We already reached an agreement of the API design at the Plone-React sprint a few months ago. Though, it turned out that implementing that on top of the existing plone.tiles implementation was harder than we thought and we did not anticipated all the problems that came along with that.

We decided to keep the API design and to write a simple Dexterity behavior that adds a “tiles_layout” field for the layout information and a “tiles” field that holds the actual data of the tiles. Ramon already wrote a JSON-field in the Guillotina code that we decided to re-use for our implementation.

Rob Gietema already wrote a first prototype of the new editor at the Plone-React sprint and he was waiting for the backend code to be implemented. While we were working on the backend implementation, he focused on the prototype.

plone.restapi

Planning board with plone.restapi issues

Lukas Graf added missing translations in plone.restapi responses and simplified the test setup and did some clean up on the code (https://github.com/plone/plone.restapi/pull/545).

Thomas Buchberger worked on fixing the plone.restapi object creation logic to behave more like through-the-web object creation (https://github.com/plone/plone.restapi/pull/549) and separated the object creation from firing the events.

Sune Brøndum Wøller cleaned up and upgraded multiple Plone versions in plone.restapi (https://github.com/plone/plone.restapi/pull/537), worked on portlets and portletmanager serialization and fixed a ReadConflictError in the plone.restapi tests for the documentation that was bugging us for quite some time (https://github.com/plone/plone.restapi/pull/543).

David Glick and me worked on Zope 4 compatibility for plone.rest and plone.restapi. It turned out that one of my fixes on plone.rest was already sufficient and that the test failures in plone.restapi were caused by a plone.testing issue that David found and fixed (https://github.com/plone/plone.testing/pull/49).

Roel Bruggink continued his efforts on turning the Plone site root into a Dexterity object. He worked on making the IPloneSiteRoot interface / content object behave more like content and he attached behaviours to the IPloneSiteRoot to edit them without relying on a default page.

Mikel Larreategi finished his work on the translation of the content-type names on the @types endpoint (https://github.com/plone/plone.restapi/pull/540) and the translation of the actions and workflow state and transitions on the @history endpoint (https://github.com/plone/plone.restapi/pull/541).

Plone-React

Rob gave an introduction to the Plone-React codebase and explained the basic concepts and libraries that we use in Plone-React.

#Plone #React hands-on by @robgietema pic.twitter.com/PSBQm0zfrN

— kitconcept (@kitconcept_gmbh) June 22, 2018

Eric Steele started to work on creating the add-ons control panel in React. Carsten Senger took over the work that Rob started before the sprint to bring the users and groups control panel to Plone-React.

Andrea Cecchi. and Johannes Raggam worked on the React-based widget for references in Plone.

Guillotina

After giving an introduction to Guillotina to the sprinters, Ramon went ahead and made Plone-React work on top of Guillotina. Since it origins, plone.restapi and Guillotina were supposed to share the same API to allow us to switch the backend for our new frontend at some point in the future. Ramon was also heavily involved in the API design of plone.restapi and wrote the first version of plone.rest before he decided to invent Guillotina. Over the time both APIs differed because of differences in the underlying implementation.

Ramon and Rob worked on this and by the end of the day they could present a working version that allows basic content editing and browsing.

Roadmap / Plone 6

We had a hangout with Philip Bauer from Munich who is leading the efforts to migrate Plone to Python 3 and Zope 4.

Hangout with ⁦@StarzelDe⁩ discussing the roadmap to Plone 6. pic.twitter.com/Wl1H4xVXcx

— kitconcept (@kitconcept_gmbh) June 22, 2018

Philip and I already had a longer discussion about a possible roadmap for Plone 6 and how to bring our efforts on the frontend together with the efforts of the group that works on Python 3 and Zope 4. We discussed the outlined roadmap and the upcoming sprints where we plan the implementation of the roadmap.

Lunch / Dinner / Evening

We went to have lunch in a Vegan cafe (Black Veg) and to a traditional brewery in the old part of the town for dinner (Bierhaus Machold). After dinner and a few drinks, we decided to head back to the office for some late night hacking. Not without stopping by at a local “Kiosk” for some customary buying of beverages for the evening.

From March 12th to 15th, we hosted a Plone-React sprint at our office in Bonn. The main goal of this three day sprint was to contribute back the work we did for a recent client project. We also planned to upgrade Plone-React to React 16 and improving the error handling.

A small and very focused sprint with the core contributors of Plone-React was the perfect opportunity to do so. Participants were Rob Gietema, Roel Bruggink, Asko Soukka, Victor Fernandez de Alba, Carsten Senger and me.

Sprint Day 1

We started the first day of the sprint by doing a quick stand up and planning meeting. We used the days before the sprint to outline our main goals and objectives, so we could get started right away.

Planning board after the stand up

React 16

On the first day Rob and Victor worked on upgrading plone-react to use React 16 and Webpack 4.

We postponed the upgrade to React router version 4, because some of the dependencies that we need, are still in alpha phase and staying with the old version does not hurt right now.

Card to upgrade to React 16

Error Boundaries

The main reason why we were eager to upgrade to React 16, was a new feature called “error boundaries”, which allows catching errors in React components and handle them gracefully, without failing the entire app.

Victor implemented error boundaries for client and server side components and for the Redux middleware. It is also possible to pass those errors to Sentry for aggregation and further error handling.

Since we are about to launch our first Plone-React-based project in the next weeks, this was something that was especially important for us.

Error message in plone-react

Token Expiration Middleware

Rob added a token expiration middleware to plone-react that improves the handling of JWT auth token in Plone-React. Working with plone-react on a daily basis revealed a few edge cases where the current authentication failed. The new middleware solves those issues.

plone.restapi

Roel and Carsten focused on plone.restapi. Even though this was not planned to be a major topic for the sprint, we had to fix some issues that were causing troubles in the Plone-React frontend.

Carsten worked on fixing an issue that was preventing to allow the API consumer to reset field values to “None”.

Roel started to work on exposing widgets with tagged values via plone.restapi. This is necessary to support autocomplete or reference widgets in Plone-React.

Test Setup

Asko, who joined the sprint remotely, worked on a brand new end-to-end testing setup with Robot Framework. Because this would be way to easy for a super-smart guy like Asko, he decided to wrap the components into a Docker containers, to make it easier for non-Python devs to set it up. Making Plone PIP-installable is also something that we want to have for quite a while. Asko decided to also mix that in. To make this fun, he decided to add support for Jupyter notebook, which makes it super easy to write Robot Framework tests.

Hangout with Asko

I started the day discussing the acceptance testing setup with Asko, quietly listening to Rob and Victor and discussing the plone.restapi issues with Roel and Carsten.

Pastanaga Toolbar

During the first day, we started a discussion how plone.restapi could support a toolbar that automatically adapts to the permissions of the logged in users and shows only the actions that this particular user has permission for.

Before the sprint Victor worked on implementing the new super fancy adaptable Pastanaga toolbar, and we were eager to build a proper backend implementation for this.

🥕Pastanaga Toolbar Demo (WIP, heating engines in preparation for Plone-React Bonn Sprint 2018) So far, basic behavior, support for lazy load menu components, push-pull navigation. #Plone #Pastanaga #🥕 #React #CSS #UI #LetsMakeItReal pic.twitter.com/t88twlwc9j

— Victor Fernandez de Alba (@sneridagh) March 4, 2018

Sprint Day 2

Create-React-App

On the second day of the sprint, Rob and Victor started to look into building a “create-react-app”-like functionality for Plone-React. create-react-app is a widely popular code skeleton generator by the React team at Facebook. It hides lots of complexity from the user (e.g. Webpack, libraries, configuration) and makes it easy to get started with React. This is super important for the adoption of Plone-React because it also allows to use Plone-React as a library and basis for custom client projects.

Rob and Victor created a proof-of-concept that kind of works but it became clear that this requires a lot more effort before this becomes ready-to-use.

Pastanaga Editor

We scheduled a time slot for a discussion and planning session about the new “Pastanaga editor” user experience. We already implemented basic text editing based on the DraftJS editor from Facebook. The current editor allows basic text editing as well as inline (italic, bold, etc.) and block styles (e.g. headlines, bullet points) and external links.

The next step on our agenda to make this editor based on Tiles to make it extendable and allow the user to add images, videos and other media objects.

We agreed on moving forward with an agile approach of building something useful step-by-step, making sure to build a fully functional and useful editor at any point of the development stage, rather than building the full Mosaic-like functionality at once.

Right after this was settled, we started to draft a tiles endpoint that would build the basis for the next iteration of the Pastanaga editor.

Toolbar

Rob started to work on implementing the context-aware toolbar. This was a challenge because the toolbar needs to adapt to the content that is shown in the main column. This means a component deep down the component hierarchy (content) needs to be able to update a component that lives outside of the DOM hierarchy of the parent component. Luckily for us, “Portals” in React support exactly that use case.

Victor worked on the flexbox styling of the toolbar as well as on optimizations of the Webpack configuration.

plone.restapi, Tiles, and Testing

Roel and Carsten continued to work on plone.restapi issues. Roel continued to work on the tagged values representations and Carsten finished the actions endpoint for the context-aware toolbar.

I followed our established documentation-first approach on plone.restapi and wrote the docs for the new tiles endpoint.

With the basic test setup already in place, Asko struggled with the Travis CI setup running different versions of Node and Python at the same time. Problem is that the ZEO version requires the latest Python 2.7.13 which is not shipped by default with Travis CI.

He also worked on a Docker compose option to support an option to run the API server. This would allow faster development of Robot Framework tests since the API server does not need to restart for each test (iteration).

Sprint Day Three

We started the last day of our sprint discussing the create-react-app use case we worked on the day before. Afterwards, we did a hangout with a student who is interested in working on this during the Google Summer of Code 2018.

Hangout with a possible Google Summer of Code student

Right after this call, Asko gave me a tour of the new, Docker compose based, PIP-installable, Jupyter notebook enhanced test setup.

My head was still spinning from Asko’s amazing work, when Rob asked me for a minute to show me a prototype of the new Medium-like Pastanaga editor.

My head is still spinning from the #robotframework based acceptance testing setup walkthrough by @datakurre earlier today. Then @robgietema comes along and shows me a working prototype of the new #Plone Pastanaga editor for plone-react. Mind-blowing! pic.twitter.com/2VLLvIkgQf

— Timo Stollenwerk (@timostollenwerk) March 14, 2018

Summary

The sprint was extremely productive and fun. Having a small and dedicated group of developers with a clear goal and focus really worked well for us.

Group photo in front of the kitconcept office

We contributed back all our re-usable code from our client project, upgraded Plone-React to React 16. We fixed some important issues in plone.restapi and build a super fancy test setup that will allow us to further improve the software quality of Plone-React.

We also laid the groundwork for the next important steps forward: Building the new Pastanaga editor with a tiles-based backend and allowing to use Plone-React as an extensible library with a “create-react-app”-like functionality.

We are already looking forward to the upcoming Beethoven Sprint and the Costa Brava sprint where we will continue to push Plone-React, Pastanaga and plone.restapi.

The way we deal with icons in the web has evolved over the years. Images, images sprites, fonts, SVG, SVG sprites… I’ve been looking lately for the current best practice in order to include it in Pastanaga and I wanted to share with you my results. Please note that it’s not closed and I’m open to suggestions. PRs are welcome too!

Abandon font-based systems

It was clear to me that font-based icon systems are a no longer an option today. For several reasons:

• The font is loaded every single time on the page, regardless if we use all the icons or none of them. This bloats the application size and forces an additional request (in the best case scenario).
• An existing font is difficult to create, maintain and update. You can use some online (free) services to do that and even you can forge your custom icon font with them but it’s cumbersome and not practical.
• Forces you to maintain a parallel CSS that maps the icon name with its actual character in the font (which is obscure). The font creation tool helps you with that, but…
• Extending them with new icons is also complex, especially for newbies, and you need access to the source and reload the source in the same tool that was created.

only to name a few.

Time to move on: inlining SVG

The rise of SVG in modern web developing is for a good reason:

• SVG is a vector format, so it looks great in HiDPI displays
• It’s lightweight and portable, since it’s not a binary file
• It can be styled (and animated) easily using CSS (provided they are inlined, not used with the <img /> tag
• You can control it via JS

My initial feeling was that using a SVG sprite based system would be the best approach, but I soon was discouraged, after reading to Chris Coyier, CSSTricks: A Pretty Good SVG Icon System

All in to inlining SVGs, then.

So, we need an SVG icon system. Luckily for us, Pastanaga already has a complete set of icons based on SVG organized in one file per icon.

Goals

Our main goal is to provide inline SVGs in our applications, having in mind:

• It should be performant and small in size
• Only the used icons should be loaded in the given view, compatible with lazy loading
• Has to be a no-brainer and clutter-less from the developer point of view
• You should be able to extend (or override) the default icon set with your own icons easily
• Valid for all modern frameworks, with focus on Angular and React

Harnessing the power of Webpack and modern JS

As developers we want to use the tooling that we have at our hands in the best possible way. So our icon system should use simple ES6/7/* and TypeScript conventions.

import myIcon from './icons/my-nice-icon.svg';
import Icon from './components/Icon';

and the from JSX:

<Icon name={myIcon} />

or angular template:

<Icon [name]="myIcon"></icon>

or

<div icon [name]="myIcon"></div>

Deconstructing the SVG and put it back together again

According to all the use cases shown in this interesting article by Amelia Bellamy-Royds in CSSTricks: How to Scale SVG the most sensible approach when inlining SVGs is to simply just set the viewBox on your <svg> tag, and set one of height or width to auto. The browser will adjust it so that the overall aspect ratio matches the viewBox. As Amelia points out, that would work for all modern browsers back until 2014. If we have to support older ones, we will need to apply for those the famous padding-bottom hack. Let’s keep things simple for now.

Let’s assume that our SVG is not perfect, and we want to have the all the flexibility that a modern browser can achieve handling SVGs. We will take the existing SVG, deconstruct it and get all the SVG attributes, then the content. We will then put it all together in our components, exactly the way we want it.

The Webpack part

We can accomplish all our goals by using a Webpack loaders combo for loading SVG:

{
    test: /\.svg$/,
    include: path.join(paths.appSrc, 'icons'),
    use: [
        {
        loader: 'svg-loader',
        },
        {
        loader: 'svgo-loader',
        options: {
            plugins: [
            { removeTitle: true },
            { convertPathData: false },
            { removeUselessStrokeAndFill: true },
            { removeViewBox: false },
            ],
        },
        },
    ],
},

We will use svg-loader a super simple inline svg loader that provides you extra flexibility when handling your SVG. Initially I tried the popular Webpack Team’s svg-inline-loader but it was not that flexible at the end. svg-loader returns an object with the contents and the attributes of the svg separatedly that we can later manipulate in our components. We are also filtering the SVG using the well known SVGO utility svgo-loader, we can extend or add more filtering options to optimize our SVGs thanks to it.

We are also restricting this loader to the icons folder, just in case we are handling the other SVGs in our app differently, but of course, you can use it for all SVGs removing the include key.

React

Make it work in React is very straight forward. We need to add the loader to our Webpack config, then add an icons folder and the Icon component.

import React from 'react';
import PropTypes from 'prop-types';

const defaultSize = '100%';

const Icon = ({ name, size, color }) => (
  <svg
    xmlns={name.attributes.xmlns}
    viewBox={name.attributes.viewBox}
    style={{ height: size, width: 'auto', fill: color }}
    dangerouslySetInnerHTML={{ __html: name.content }}
  />
);

Icon.propTypes = {
  name: PropTypes.shape({
    xmlns: PropTypes.string,
    viewBox: PropTypes.string,
    content: PropTypes.string,
  }).isRequired,
  size: PropTypes.string,
  color: PropTypes.string,
};

Icon.defaultProps = {
  size: defaultSize,
  color: null,
};

export default Icon;

That’s it. Our React component takes as props: the name of the imported module of the SVG, the size, and the color. If not given, the SVG will inherit the fill color set in the parent element (or itself). Also, if not specified, the SVG will scale to the parent container height.

Take a look into the JSX of the example

<div style={{ height: '100px' }}>
    <Icon name={Add} />
</div>
<Icon name={Add} size="45px" />
<Icon name={Add} size="45px" color="red" />
<Icon name={Plone} size="60px" color="#1782BE" />
<Icon name={Guillotina} size="60px" color="#EC5528" />

Angular

For the angular icon component we needed the same recipe for the Webpack config and this icon component.

import {
    Component,
    Input,
    HostBinding,
    ViewEncapsulation,
    ChangeDetectionStrategy } from '@angular/core';

import { DomSanitizer, SafeHtml } from '@angular/platform-browser';
import { OnInit } from '@angular/core';

const defaultSize = '100%';

@Component({
  // tslint:disable-next-line:component-selector
  selector: '[icon], icon',
  template: `
  <svg
    [attr.viewBox]="name.attributes.viewBox"
    [attr.xmlns]="name.attributes.xmlns"
    [innerHTML]="svgContent"
    [style.height]="height"
    [style.width]="'auto'"
    [style.fill]="color"
    >
  </svg>
  `,
  encapsulation: ViewEncapsulation.None,
  changeDetection: ChangeDetectionStrategy.OnPush
})
export class IconComponent implements OnInit {

  constructor(private sanitizer: DomSanitizer) {}

  svgContent: SafeHtml;
  defaultSize = defaultSize;
  height: string;

  @Input() color: string;
  @Input() size: string;
  @Input() name;

  ngOnInit() {
    this.svgContent = this.sanitizer.bypassSecurityTrustHtml(this.name.content);
    this.height = this.size ? this.size : defaultSize;
  }

}

We also use the same approach using the component, the Angular template way:

<div icon [name]="Add"></div>
<div icon [name]="Add" color="green"></div>
<icon [name]="Add" color="red" size="45px"></icon>
<icon [name]="Plone" color="#1782BE" size="60px"></icon>
<icon [name]="Guillotina" color="#EC5528" size="60px"></icon>

Our Angular component takes the same three properties as the React one.

In addition, Typescript forces us to overcome some tiny things.

Typings

In order to be able to import the SVG as a module, we need to add this typing to our app:

declare module "*.svg" {
  const content: any;
  export default content;
}

Add the imported SVG object as a Class member

The Angular template won’t be able to use it if the imported SVG object is not a Class member, like:

import { Component } from '@angular/core';
import Add from '../icons/add.svg';
import Plone from '../icons/plone.svg';
import Guillotina from '../icons/guillotina.svg';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  Add = Add;
  Plone = Plone;
  Guillotina = Guillotina;
}

Conclusion

While there are other approaches out there like the Icon component that @angular/material has, they all feel to me like too much and all of them are bloated with lots of options that we don’t really need. I’d like to use a more lightweight and approachable solution like the exposed here that only does what we really need. At the end, it’s not rocket science.

If you have any suggestion, please contact me or open an issue on Github. PRs are welcome!

After more than three years of development and 25 alpha and one beta release, we are very happy and proud to announce the release of plone.restapi 1.0.0.

plone.restapi is a RESTful hypermedia API for the Plone Open Source Content Management System. It exposes the unique and powerful features of Plone, including the core content management features as well as dynamic content type creation, workflows, permissions, versioning and more.

plone.restapi builds a bridge between a stable and mature Open Source CMS that has been around for more than 15 years and modern state-of-the-art JavaScript-based solutions like React, Angular, Vue and others.

A Little Bit of History

PLOG 2014

The development of plone.restapi started in beautiful Sorrento, Italy at the Plone Open Garden in 2014 after I gave a talk about building an AngularJS application on top of Plone.

A long discussion with Simone Deponti under the Italian sun, about REST API design principles and hypermedia (of course), led to the first commit and the development of a first proof-of-concept implementation.

plone.rest and PLOG 2015

One year later we gathered in Sorrento again. Laurence Rowe, Ramon Navarro Bosch and I spent our days and nights discussing the details of the REST API design and drafted multiple endpoints.

One of the main obstacles to building a RESTful API on top of Plone was the missing ZPublisher support for HTTP verbs such as PATCH, PUT or DELETE. In 2015, I sat together with Ramon Navarro Bosch in Sorrento (again) and we (he really did all the heavy lifting) started to build plone.rest, a small package that adds support for HTTP verbs to Plone.

Archetypes and Serializers

We never planned to support Archetypes in plone.restapi. Though, when Thomas Buchberger and Lukas Graf came along and offered to build it, we did not object (of course not, this is Open Source). Their company 4teamwork planned to build a REST api on top of Plone for their OneGov GEVER platform.

Instead of building something on their own, they decided to join forces and share their work and code with the community. Along the way, they heavily refactored the code, added tons of adapters for loose coupling and the ability to customize the JSON serialization.

After this, we were confident to do a first alpha release of plone.restapi on June 14th 2016.

Beethoven Sprint

In March 2017, fourteen Plone developers from eight different countries gathered in Bonn, at the kitconcept office, for the Beethoven Sprint to work on plone.restapi and related topics. In addition to sorting out the last remaining design decision, many exciting new projects were started and announced.

Angular

At the Beethoven sprint, Eric Brehault started to work on an Angular SDK for plone.restapi. A release followed soon and Eric gave a very successful and crowded training at the Plone Conference 2017 in Barcelona.

Today, Angular SDK is a mature package for Angular 2 that makes it really easy for front-end developers to interact with Plone and a fantastic starting point for newbies.

Eric and I mentored Noel Varghese during last year’s Google Summer of Code to build a Progressive Web App for Plone in Angular 2. Noel gave a nice presentation of his successful project at the Plone Conference in Barcelona.

React

Rob Gietema and Roel Bruggink started to build a React-based front-end on top of plone.restapi at the Beethoven sprint in Bonn. Later that year, they went to Toulouse in September 2017 to implement the Pastanaga CSS together with the Plone Angular team.

In November they visited Bonn again for the Pastanaga Sprint where we started to implement the new Pastanaga UI for plone-react.

At kitconcept, we started to use plone-react with Pastanaga for an ongoing project. We can’t wait to release our work and contribute it back to the community.

Vue.JS

Inspired by the Angular SDK and plone-react, Kevin Bieri started to build a VueJS plone-vuejs implementation on top of Plone at the Plone Conference 2017 in Bareclona.

Guillotina

Ramon Navarro Bosch and Nathan van Gheem revelead the name of “Guillotina”, a blazing fast async Python framework that shares the public API with plone.restapi at the Beethoven sprint in Bonn.

Successful Open Source Collaboration

plone.restapi started with an idea and discussions. People and companies jumped in and contributed in many ways that haven’t been dreaming about at first.

Simone, Laurence, Ramon and other helped to shape the initial idea. Lukas, Thomas, Roel, Carsten, Victor, Mikel and many others contributed new endpoints, bugfixes, etc.

Eric, Rob, Noel, Kevin, and others started to build frameworks and solutions on top of plone.restapi.

Many companies such as 4teamwork, Code Syntax, Markina Corpus, VNC invested and contributed to plone.restapi.

The Plone Foundation always supported our efforts by funding sprints.

plone.restapi is a true community effort and the joy that we feel when collaborating with wonderful people pays us back for the countless hours we spend on hacking on code.

Future Plans

A Plone Improvement Proportal (PLIP) to ship Plone 5.2 with plone.restapi has been accepted by the Plone Framework Team:

https://github.com/plone/Products.CMFPlone/issues/2177

With plone.restapi considered stable and close to being feature complete, we will continue working on what could become the next Plone…stay tuned.

Lighthouse is an open-source, automated tool for improving the quality of web pages by Google. It measures the performance of a website and provides metrics for accessibility, best practices for modern web apps, search engine optimization, and assess web applications for adherence to Progressive Web App standards.

Lighthouse Logo

Together with WebPageTest and Google Page Speed Insights it is an indispensable tool to optimize your website performance.

Installation

Lighthouse can be installed in any JavaScript-based project by just running ‘npm install’:

$ npm install lighthouse -g

If you don’t have a package.json in your project, just install npm and run ‘npm init’ before installing.

Running Lighthouse

You can check the performance of any website by calling the ‘lighthouse’ command with the URL of the website you want to test. Append the --view parameter to show the HTML report, right after the command has finished:

$ lighthouse https://kitconcept.com --view

The report will give you five different ratings about PWA, performance, accessibility, performance best practices, and SEO.

Lighthouse Results

Continuous Performance Measurements

If you run your performance test every now and then, you always risk to hurt your website performance without noticing. If a performance regression happens unnoticed, it is usually very hard and time consuming to figure out which change caused the performance regression.

You can easily fix this and save lots of time when you run your performance tests and analysis continuously.

Unfortunately Lighthouse does not allow you to set performance test specifications that your CI system can test against, like WebPageTest or Google Page Speed Insights do (we will cover those tools in later blog posts). Though, it is still very convenient to run the performance test on a regular basis for each commit and include them into your CI report.

Install Lighthouse locally for CI

When it comes to a Continuous Integration, a local installation is prefered over a global one, which is usually harder to manage and to maintain. Especially if you have multiple projects with different sets of package versions on your CI.

Therefore we install Lighthouse locally in our project directory:

$ npm install lighthouse --save-dev

This command will install Lighthouse to your local package.json file. We recommend to use yarn or npm package-lock.json to lock down the package version you are using for a repeatable and stable project build.

For convenience, we add a “lighthouse” script to our package.json:

"scripts": {
  "lighthouse:ci": "node_modules/lighthouse/lighthouse-cli/index.js \
  --output-path=./lighthouse-report.html --quiet \
  --chrome-flags='--headless' https://kitconcept.com"
}

We call the locally installed lighthouse binary and set a static output path (by default, Lighthouse creates a file with the current date/time in the filename which makes it harder to publish on your CI).

We also include the --quiet option and run it on headless chrome, so we don’t need to install and run an X server on our CI system.

At the end, we hard-code our project URL into the command so we do not have to type it manually each time we run this command.

Now we can just run:

$ npm run lighthouse:ci

and it will create a nice HTML report that we can publish in our CI.

Configure Lighthouse for your local development environment

For convenience, we also add a command that you can run locally:

"scripts": {
  "lighthouse": "node_modules/lighthouse/lighthouse-cli/index.js \
  --output-path=./lighthouse-report.html --quiet \
  --chrome-flags='--headless' https://kitconcept.com/blog"
}

The --view parameter will fire up a browser with the report at the end of the performance analysis. This is something we clearly don’t want on our CI system.

Publish Lighthouse Reports in Jenkins CI

Travis and other lightweight CI system usually lack the option to publish any reports except the command line output. Though, if you are using Jenkins CI, you can use the HTML publisher plugin to publish your Lighthouse report.

sh 'npm install'
sh 'npm run lighthouse'
publishHTML (target: [
  allowMissing: false,
  alwaysLinkToLastBuild: false,
  keepAll: true,
  reportDir: '.',
  reportFiles: 'lighthouse-report.html',
  reportName: "Lighthouse"
])

After adding publishHTML to your Jenkins pipeline, you will see a “Lighthouse” link under the ‘Artifacts’ tab:

Link to Lighthouse report in Jenkins

There is a caveat though. Jenkins 1.641 / 1652.3 introduce the Content-Security-Policy header to static files served by Jenkins. The default header is set to a very restrictive set of permissions to protect Jenkins users from malicious HTML/JS files in workspaces.

To allow Jenkins to display the Lighthouse reports, we have to add the following JAVA_ARGS to the Jenkins startup (for instance by adding the following line to your /etc/default/jenkins file):

JAVA_ARGS="-Dhudson.model.DirectoryBrowserSupport.CSP=\"sandbox
allow-scripts; default-src 'unsafe-inline'; img-src * data:\""

For more details see the Content Security Policy Reference and the Jenkins docs on configuring Content Security Policy.

After you fixed the Content Security Policy of your Jenkins you will see the full report when clicking on the ‘Lighthouse’ link on the ‘Artifacts’ tab on your Jenkins build:

Lighthouse Report in Jenkins

Jenkins Declarative Pipeline Stage for Performance Tests

A full declarative pipeline stage for lighthouse looks like this:

stage('Performance Tests') {
  agent {
    label 'master'
  }
  when {
    branch 'master'
  }
  steps {
    deleteDir()
    checkout scm
    sh 'npm install'
    sh 'npm run lighthouse'
  }
  post {
    always {
      publishHTML (target: [
        allowMissing: false,
        alwaysLinkToLastBuild: false,
        keepAll: true,
        reportDir: '.',
        reportFiles: 'lighthouse-report.html',
        reportName: "Lighthouse"
      ])
    }
  }
}

We run the performance test stage on ‘master’ agents and only on the master branch. The steps performed are a simple “npm install” to set up the project build and then we run ‘npm run lighthouse’ to produce the HTML report. If you already have an npm build from a previous step you can of course just unstash the build artifact.

Jenkins pipeline with Lighthouse performance tests stage

Summary

Lighthouse is a valuable and indispensable tool if you want to deliver a fast and user friendly website. Running the analysis on a continuous basis on your CI is a good idea if you take performance seriously. Setting it up is fast and easy. Maybe in the future Lighthouse will also provide a testspec feature that will allow us to fail a CI build (or mark it as unstable) on performance regressions. Though, if you run WebPageTest or Google Page Speed Insights additionally, this is not really needed.

Pastanaga is a new user experience framework for the web, designed by Albert Casado.

Pastanaga was first presented in March 2017, at the Plone Open Garden in Sorrento. In July, we started with an initial implementation during the Midsummer Sprint in Jyväskylä, Finnland.

Pastanaga was also present at the recently held Plone Conference in Barcelona, where Albert gave a presentation on it. In addition, Eric Steele, the Plone release manager, gave us the opportunity to present Pastanaga to the audience during his keynote on the first day of the conference.

With all the positive feedback and energy we took from the Plone Conference, we wanted to push things further and we just couldn’t wait until our “Beethoven Sprint”, which is planned for early 2018. Therefore we decided to organize a small and focused sprint at our office in Bonn to work on the implementation of Pastanaga.

The Pastanaga Minimal Viable Product

As an Open Source community (and software engineers) with many years of experience in designing and building complex Content Management System applications, we sometimes have the tendency to try to solve all problems at once.

Over the years we encountered and solved many complex problems and when we build something new, this can be both a source of wisdom as well as a baggage that you carry around.

This sometimes led to a situation where we were over-engineering solutions, to solve all the problems that we encountered over the years at once. Enhancements sometimes stayed around for years without really becoming production ready and usable in real-world projects.

To avoid this from happening when working on implementing Pastanaga, we decided in Jyväskylä to focus on a Minimal Viable Product.

A Minimum Viable Product (MVP) is a product with just enough features to satisfy early customers, and to provide feedback for future product development. The Pastanaga MVP needs to provide what we consider the essentials of a Content Management System:

• A site administrator can and add, edit, and delete a page

• A user can view the created pages and navigate the site structure

In order to be usable for public facing website projects, we added two additional technical requirements:

• The page should be fully rendered within 500 milliseconds

• Google should be able to crawl the contents of the website

Those requirements might sound very simple, but they are actually not.

Pastanaga aims to leverage the editing experience and reduce the complexity that we took for granted over the years. We aim to simplify the user experience for the editors by getting rid of things that we got used to. For instance, adding an image to a page should be as simple as just dragging and dropping an image to the page and Plone will take care about the heavy lifting of automatically uploading and resizing the image.

You can find a list of all the user stories that we plan to implement as part of the MVP here:

• https://github.com/plone/pastanaga#minimal-viable-product

Having the goals and scope for this set the only thing that was needed was a bunch of Plone devs and three days and nights of coding.

Sprint Day One

After the sprinters arrived, we started with our sprint planning session. We decided to focus on the implementation of the Pastanaga MVP and work on the other issues (e.g. plone.restapi) only if we need them for the MVP.

.@robgietema giving us an introduction to plone-react. #plone #sprint pic.twitter.com/Ga6Bsd3l2J

— kitconcept (@kitconcept_gmbh) November 15, 2017

After the planning meeting, Rob gave us an introduction to plone-react, a ReactJS-based implementation of the Plone UI that he and Roel worked on over the past months and that we decided to use as a basis for our MVP.

Pastanaga sprint planning meeting at our office in Bonn. #plone #Sprint pic.twitter.com/FDpjv5wx5t

— kitconcept (@kitconcept_gmbh) November 15, 2017

We went through all components, reducers, bells and whistles of the application and discussed best practices, developer environments and developer approachability.

After that session, Rob and Victor started with the implementation of Pastanaga. Davi created a pull request that adds an uninstall profile for plone.restapi and started to learn about React. Roel started to look into a way to turn the Plone site root into a Dexterity object, something that we would need to simplify the Plone editing experience. I worked on the basic Robot Framework acceptance test setup and updated the contents of the Pastanaga github repository, which is supposed to be just an entry point for all our initiatives around Pastanaga:

• https://github.com/plone/pastanaga

Day Two

On the second day, Victor finished the login form and made the error messages work.

Rob implemented the document edit accordion menu, fixed the button styling, made plone-react use the Pastanaga icons and started to work on the toolbar.

Davi added a search widget to the header, implemented the breadcrumbs navigation and added styles for the document heading and description.

Right before the wrap-up meeting of day two, Roel showed us a Plone site with a “containerish” Dexterity-based site root. We did not really expect that much progress and went to bed (some of us a lot later) still very impressed by his accomplishment.

Day Three

On day three, Rob started to work on the new Pastanaga document edit view. He made the new edit view to show multiple content items (e.g text, image, video) and allowed to change the order of those content items via drag and drop.

Davi continued to work on the header and breadcrumbs styling. Victor looked into the mobile views of our responsive design, fixed some issues with the status messages and briefly started to look into GatsbyJS (which we plan to use to implement pastanaga.io).

Summary

After three days (and nights) of hacking, we had:

A fully functional login form with error messages and password forgotten functionality:

A fully functional Pastanaga Toolbar that can be collapsed or expanded. With all the menu items present and the personal toolbar functionality available:

A view to add and edit pages with all the existing functionality:

In three sprint days, we accomplished our main goals and were able to create the first iteration of a Minimal Viable Product that we can use to build things upon. We plan to continue to work on this, use it in our current and upcoming projects, and of course: contribute back as much as we can.

Stay tuned for more updates on this soon!

Summary of two presentations by Victor Fernandez de Alba, Albert Casado and Timo Stollenwerk during PLOG 2017

This is the summary of two presentations that we (Victor Fernandez de Alba, Albert Casado, and Timo Stollenwerk) gave during the Plone Open Garden 2017 in Sorrento, Italy. The views expressed in our talks and in this summary are just personal opinions and it is up to the Plone community to decide to implement them.

A RESTful API for Plone

plone.restapi was started at Plone Open Garden in Sorrento three years ago. We did our first Angular 1 project on top of Plone in 2013 and we felt the need for a standardized out-of-the-box solution to expose and manage Plone content through a RESTful API.

The Beauty and the Beast - Modern Javascript Development with AngularJS and Plone from Timo Stollenwerk

The first alpha release of plone.restapi was published in July 2016. 4teamwork provided support for Archetypes and helped to refactor the serializers/deserializers. Since then plone.restapi has been used in production by 4teamwork, CodeSyntax, kitconcept, VNC, and others.

Current status of plone.restapi

The core functionality of plone.restapi is a content API to manage Plone content that can be serialized Dexterity and Archetypes-based content from and into JSON. The content API allows manipulating content together with workflow states, access permissions, and local roles.

plone.restapi provides service endpoints for registry entries, vocabularies, and search. A types endpoint returns JSON schema to auto-generating forms. A users and groups endpoint allows managing users and groups. Users can authenticate with basic authentication as well as with a token-based authentication (JWT).

Beethoven Sprint in Bonn

In March 2017 fifteen Plone developers from eight different countries gathered in Bonn, Germany at the kitconcept office to finish the missing features and solve the last blockers for to road to a final plone.restapi release. At the end of the sprint we were able to release plone.restapi 1.0a10 which included more features than any release before. This release introduces a sharing, vocabularies, copy/move, and principals endpoints. We finished the API design for components and expansion, which were the main blockers because they require some minor breaking API changes.

Roadmap to plone.restapi 1.0

The current plone.restapi release is 1.0a13 and we plan to do another release this month that includes a file upload endpoint that allows resumable uploads via TUS. A pull request by Thomas Buchberger is already in place that needs to be reviewed and merged.

We plan to do another release is in May 2017, that will include an expansion mechanism that allows embedding information such as navigation, breadcrumbs, workflow, permissions, etc. within a content response. After this release, we might be able to declare the API stable and enter the beta stage.

We hope to be able to do a final 1.0 release before the Plone Conference in Barcelona in October 2017. Features that we are actively working on and that might be included are a history/versioning endpoint and a multilingual/translation endpoint.

Towards a Headless CMS

Web technologies continue to change at a very high pace and so does the CMS market. Traditional Content Management Systems have a hard time finding the balance between the stability that is required to manage long living content on large projects and the requirements of modern websites and with state-of-the-art JavaScript front-end technologies.

Content Management Systems need to evolve from systems that manage and render content to managing arbitrary “resources” and factor out the rendering part in order to be able to keep the pace of a fast-changing front-end ecosystem. Plone has been able to provide stability and scalability for long lasting and large projects over the last decade. plone.restapi will allow us to combine that with latest and greatest JavaScript front-end technologies and libraries to build state-of-the-art user-friendly user interfaces.

API Approachability and a Browsable API

A critical factor to the success of any API is approachability, ease of use, and quality of the documentation. Hypermedia APIs such as plone.restapi allow the user (or the developer) to browse through the API as a human user would do on a website by following links to the next resource. This allows consumers of the API to explore the API instead of having to look up every single detail in the documentation.

A browsable API should also allow the consumer to manage content via the API by providing auto-generated forms for adding or editing content.

Django REST Framework is an example of hugely popular REST API that is successful because, among other things, it provides a browsable API to the user. The Django community managed to raise funding of 30.000 £ for Django REST Framework 3. plone.restapi core won’t need this kind of funding because the development is mostly covered by real-world project needs. Though, the development of a browsable API is hard to rationalize within such a context.

Building Bridges

The idea of plone.restapi was to create an abstraction layer around the main features of the Plone Content Management System to expose our unique set of features that not many systems on the market can match.

It was always part of our plan to allow to implement different backend solutions at some point. Due to the hard work and visionary of Ramon, with the help of Nathan and Asko, we have this option available sooner than any of us would have imagined. This leaves us in a very comfortable position. If you start a new Plone project today you have three very good options. All of them embody the main treats and the unique Plone essence that we all know and love. You can:

• Use plone.restapi today with Plone 4.x or Plone 5.x with both Archetypes and Dexterity. This allows you to build state-of-the-art front-end solutions on top of existing projects for basically any Plone project (Plone 4 was released in 2010). This is the recommended option if you work on a traditional content-centric Website or Portal that requires fast first-time-page-rendering and requires to be found on Google (SEO).

• Use plone.restapi with Plone 4.x or Plone 5.x with a custom modern JavaScript front-end build entirely on Angular2 or React. Using modern JS frameworks allows you to largely re-use existing libraries, which reduces the amount of work that is necessary to build a custom web application. This is the preferred solution for any intranet like application with a highly customized UI, that requires logged-in users.

• Use Guillotina with its plone.restapi compliant API with a custom front-end. This is the preferred solution if you do not require all the functionality of standard Plone but rather look for a highly scalable solution with and modern backend.

The Challenges

Everything in life comes with a cost. And so does following three different paths, instead of just one. Traditionally, Plone supports a wide variety of use cases. We need to provide the stability that is needed for a large portal that is built over 5 or 10 years, as well as the agility of a state-of-the-art web-application that uses the latest JS framework.

In order to do so, we need to build bridges between the three options outlined above, to not waste too much time and energy building the same components two or three times for the different branches.

The two “bridges” that we need to build are: keeping Guillotina and plone.restapi in sync and share JavaScript libraries and widgets between standard Plone and new front-end solutions on top of Plone.

Keeping Guillotina and plone.restapi in Sync

The first challenge is to keep the APIs between plone.restapi and Guillotina in sync. This might sound easy at first. Though, keep in mind that Plone and Guillotina use a different storage, different routing and a different set of functionality. They are brothers (or sisters) in the spirit. Though, they significantly differ under the hood. To keep the APIs in sync, the first thing we need is a formal specification that allows us to automatically test the APIs for compatibility.

Unfortunately, this is not as easy as it sounds because non of the existing API frameworks such as Swagger, Blueprint, or RAML support arbitrary API routes that are required for traversal.

Though, if we can manage to keep the APIs in sync we can share front-end libraries between the two and might even be able to fully replace the traditional Plone backend with Guillotina at some point in the future.

Keeping two APIs in sync and sharing libraries across systems is not a trivial task. Though, we believe that this is doable and worth the effort in the end.

Sharing JS Libraries and Widgets

The second challenge is to allow to share JavaScript libraries between a traditional Plone site and the modern JS libraries and the new front-end solutions we are building.

The main problem here is that the JavaScript world is moving so fast that it is hard for us to keep the pace and at the same time provide the stability that is needed by our clients.

The Plone 5.x JavaScript stack has been started five years ago, which is a very long time in a field where we see ground breaking changes or new libraries at least once a year.

Plone currently uses Bower as a package manager, RequireJS to define dependencies between JavaScript components, Gulp as a bundler and task manager, jQuery as the main JavaScript framework, and Patternslib/Mockup for widgets.

Bower has been replaced by NPM for front-end developing long time ago. Yarn, a modern package manager by Facebook which is based on NPM, allows repeatable and stable bundling that we know from Python and Buildout. RequireJS is barely used these days since Webpack and SystemJS became the de-facto standard for declaring dependencies between JavaScript components and for bundling.

There are lots of large and well-maintained widget libraries out, some with more contributors than Plone itself (e.g. Angular Material, Semantic UI), that allows us to become consumers of libraries instead of the burden of having to maintain our own library.

Though, as mentioned before, stability is a major factor for Plone and we can not effort breaking Plone sites and consumers in the Plone 5.x branch.

The always forward-thinking Asko Soukka worked on allowing to use Webpack as a bundler for Plone 5.x. As mentioned earlier, Webpack is the current de-facto solution for bundling JS applications and it is used by both the Angular as well as the React community.

If we manage to simplify the Resource Registry and publish Plone JS packages on NPM, we might be able to go into the right direction without breaking too much existing stuff.

The Plone framework team already approved a PLIP by Nathan van Gheem to use React in Plone core. This will allow us to replace existing widgets with modern solutions and might allow us to share widgets between existing Plone 5.x sites and modern front-end libraries.

Having plone.restapi in core is a pre-condition to this. Though, there are a few blockers (unrelated to plone.restapi itself) that need to be fixed in Plone 5.1.

Mobile First UI

Mobile is overtaking desktop on web browsing in the recent years. Content Management Systems need to take this into account. The Barceloneta theme that was introduced with Plone 5 is a mobile ready theme that works on both mobile and desktop. In 2017 we believe it is time for a mobile first theme, that also allows the user to manage content via mobile phones.

Albert Casado, who is responsible for the Barceloneta theme, re-imagined Plone as a mobile first Content Management System with the Pastanaga UI:

Invisioning UX and UI from kitconcept GmbH

The Pastanaga UI is the result of studying how to improve the user experience and user interactions in Plone. It is still work in progress and Albert plans to continue to work on this during the next months.

The results will be included in a new User Experience style guide for Plone. This guide will provide core and add-on developers with a guide to maintain consistency and coherence through all the content management system UI.

One of the core principles of the new user experience is reducing the mental overhead for the user because they will have to understand two basic UX principles: go ahead, and go back.

Visual reassurance will be also provided to the user of what they are doing (or going to do) via visual hints (e.g. state colors).

For more, please take a look at the presentation.

Pastanaga UI will also provide a new set of widgets, along with their UI/UX and user interactions, and a new set of icons. It will also provide a set of visual patterns to structure and define a hierarchy of elements that will guide the user in all their actions. They will be easily reusable in add-ons too in order to preserve visual coherence everywhere.

Last but not least, Pastanaga UI will require a new theme to complete the picture. Stay tuned for updates on the Pastanaga UI!

Summary

We sincerely believe that the Plone community and the Plone CMS have a unique opportunity to re-imagine how content is managed on the web. We have both the community and the knowledge on how to do content management right. We have a secure, stable and mature platform with an exclusive set of features.

plone.restapi allows us to expose this to the world in an approachable way. New JavaScript technologies and frameworks are at our disposal and we just need to use and integrate them. The new Pastanaga UI will allow us to attract new users, plone.restapi together with plone.api has the potential to attract new developers by making Plone development more familiar.

The positive spirit and enthusiasm that we felt during the Beethoven Sprint in Bonn spread to the Plone Open Garden and the Plone community. Let’s continue to do great things together!

Report of the third day of the Plone Beethoven Sprint in Bonn, Germany

Angular 2

Eric Brehault included angular-traversal in the plone.restapi-angular2 library. He added views for interfaces and support for default views. He also implemented support for basic plone.restapi components, such as navigation and breadcrumbs. At the end of the day, Eric presented an Angular 2 example application that runs on the plone.restapi-angular2 library.

React

Rob worked on his React-based Plone demo and presented it during the wrap-up meeting.

He completed the page rendering with a Barceloneta-like user interface. A working login form and basic form rendering are in place. Content editing works, including a basic WYSIWYG editor. Navigation and a basic toolbar are working. It is now possible to change the workflow of a content object. The basic CRUD operations are also supported.

Documentation

Lukas integrated the Sphinx extension Asko created to transform pseudo-code HTTP request into multiple different output formats.

This drastically reduces the amount of example code we have to write in the plone.restapi docs and also makes sure our examples work properly. We currently create Python requests, curl, and httpie examples using sphinxcontrib-httpexample.

plone.api / Apitaizer

We had a discussion about integrating plone.api into plone.restapi. Andi created a refactoring tool helper called apitaizer to automatically replace functions and import locations and replace them with their plone.api counterpart.

Testing

After Florian and Andi fixed the coverage problem in the last days, we released a new plone.testing version.

Swagger

Victor continued to work on the Swagger docs for plone.restapi, which was mostly discussion.

Guillotina

Ramon and Nathan continued to work on Guillotina and GuillotinaDB. Ramon gave a presentation of their work in the wrap-up session. He showed us how to create a Plone site and showed a demo of the Postgres implementation model. They are now able to partition data by content type. He showed performance improvements based on jMeter tests. They removed a lot of layers, providing the same API as ZODB, using GuillotinaDB but of course in an async way. They decided to drop ZODB support in Guillotina which is still available in the “pre-Guillotina” plone.server version.

TUS Upload

Thomas started to work on TUS upload and was able to come up with a basic PoC implementation. This is still work-in-progress but it looks promising and we think we are not too far away from a final solution.

Translations

Mikel continued his work on the @translations endpoint. He worked on the Plone 5 integration and the LinguaPlone endpoints for Plone. He also looked into a redirect issue with plone.restapi.

JSON Schema

Roel worked on making schemas more approachable to API consumers and developers. He worked on the choices and reference field.

Summary

The third day of the sprint was as productive and successful as the previous days. We presented our work at a wrap-up meeting in the late afternoon and then headed to a restaurant to eat, drink and discuss the future of Plone as a headless CMS.

Report of the second day of the Plone Beethoven Sprint in Bonn, Germany

plone.restapi Enhancements

Lukas refactored the @types endpoint to support all tagged value modes. He also improved the JSON schema generation in this endpoint.

Roel worked on the addable types constraints on the @types endpoint and on an issue with choices/related items.

plone.restapi Test coverage

Florian and Andi fixed a threading issue in plone.testing that prevented the test runner from reporting the test coverage properly.

mr.developer

Florian enhanced mr.developer to report which branches are actually checked out. This is especially helpful if you are testing a Plone core pull request on Jenkins and want to double check if the correct pull request branch has been checked out. Florian pushed a new release of mr.developer that includes the fix.

Documentation / API Specification

Armin started to look into the plone.restapi documentation and postman examples how to use plone.restapi.

Victor participated in multiple discussions and started to write a Swagger/Open API specification.

Timo worked on documenting our API conventions and best practices.

Asko continued his work on the Sphinx extension to generate example for different request libraries and tools. He has a first version of the extension running and starts to work on going through the current plone.restapi docs to cross-check his extension.

Guillotina

Ramon worked on Guillotina database and he and Nathan refactored and removed lots of lines from the Guillotina code base.

Translations

Mikel worked on a translation endpoint for plone.restapi that supports both plone.app.multilingual and LinguaPlone.

React

Rob continued to work on the React front end example for plone.restapi, he created basic add and edit forms and implemented a WYSIWYG editor. for the rich text fields.

Angular

Eric worked on the Angular2 SDK, he created services for the basic resource API and created an example application. This example application does not only show how to use the SDK but also serves as basis for our Plone clients’ acceptance test suite.

plone.api usage in plone.restapi

Roel, Andi, Victor, Ramon, Nathan, and Timo discussed if we want to introduce plone.api as a dependency to plone.restapi. We decided that our long term goal would be that a developer with Python knowledge should be able to develop plone.restapi with just plone.api, without the need to master the underlying structure. We are not 100% sure yet if this is doable. Though, as our favorite Catalan Plone visionary puts it:

Nobody told us that this wasn’t doable.

Google Summer of Code 2017

Eric, Rob, Victor and Timo discussed possible plone.restapi and JavaScript related GSoC projects for 2017.

We already published a proposal for a progressive web app written in Angular 2. Other ideas we came up are:

• A progressive web app written in React

• A browsable API for plone.restapi

• PloneFormGen written in Angular 2

Framing/Expansion

We had a long discussion about how to implement framing/expansion in plone.restapi. We finally reached an agreement and a draft that is pretty close to what Thomas already implemented in his pull request.

Timo will now write the documentation for the endpoint and after a final review we will adapt the existing pull request.

Summary

Sprinters were incredibility productive today and we made considerable progress on some of the blockers of a first final release of plone.restapi.

Report of the first day of the Plone Beethoven Sprint in Bonn, Germany

We started the day with a stand-up meeting giving people a heads up on the current status of plone.restapi (Timo), Plone server (Ramon and Nathan), and Plone client (Eric Brehault).

plone.restapi is in a good shape and it saw a lot of development lately. We are still in alpha. Though, we hope to finalize the last issues on the API design (framing, security, new features, etc.) during the sprint.

Plone server was also pretty actively developed lately and we are about to reveal a new name for the package (see below).

Plone client was a good proof-of-concept project for the Barcelona sprint. Though, starting from the bottom up and building lower level libraries for Angular 2 seems to be a better approach that promises faster results.

After the wrap up of the current status Ramon and Nathan revealed the new name for Plone server, which is “Guillotina”.

Swagger / Open API

Roel investigated Swagger/Open API to see if we can use it to specify a common API for plone.restapi and Guillotina. It turned out Swagger has no support for arbitrary routes that we would need to specify the content travel API. He also worked on the addable type constraint for plone.restapi.

plone.restapi Test Coverage

Florian and Andi fixed an issue with threading coverage on plone.testing.

Guillotina

Ramon and Nathan pushed Guillotina to the Plone repository, a Travis CI setup is in place and the documentation is automatically built and pushed to [readthedocs](http://guillotina.readthedocs.io/en/latest/.

They also cleaned up the code of Guillotina and removed zope.location and zope.security from Guillotina.

They started to work on the Guillotina DB. Though, this will be a longer process with no quick results.

plone.restapi Enhancements

Victor created pull requests for a fix on the @search endpoint. Fixes for several serializers. He also started to look into multi-segment URLs.

Thomas participated in the discussion on framing/expansion and started to implement the copy/paste endpoints.

Asko worked on an extension for Sphinx that allows us to write pseudo-code request that is automatically turned into multiple formats such as Python requests, curl, wget, etc.

Mikel worked on a translations endpoint for plone.restapi. It is currently unclear, which combinations of Plone versions and LinguaPlone and plone.app.multilingual we are going to support.

Lukas worked on fixing a bug in plone.restapi on batching and the @search endpoint. He implemented more tagged values for omitting fields and started to work on the tagged values for field ordering.

React

Rob started with an example how to use ReactJS with Plone. He created a basic login form and logout functionality. As well as basic content views.

Angular SDK/Library

Ramon, Victor, Eric, and Timo had a discussion about an Angular wrapper around plone.restapi and Guillotina. We made decisions on licensing, scope, package naming, and the repository.

plone.restapi 1.0a9

Timo released plone.restapi 1.0a9 today. He also added documentation for best practices and conventions we follow for plone.restapi.

Framing

We had a longer discussion about framing in plone.restapi and Guillotina and we identified the major issues we need to figure out before we can finish the current proof-of-concept implementation. There is one API design issue left, that we hope to solve tomorrow.

Summary

We are really happy with the progress we made on the first sprint day. The sprinters were incredibly productive. And as always, it was a lot of fun hacking on Plone.

How we set up a new distributed Continuous Integration infrastructure for Plone with Jenkins, Ansible, Jenkins Job Builder, and mr.roboto.

The Plone Testing & Continuous Integration team worked hard over the last few months setting up an a new Continuous Integration (CI) infrastructure for Plone. Yesterday we were able to finally go live with our new system on jenkins.plone.org.

jenkins.plone.org after the migration to the new server. All builds are green.

A New Jenkins Server

Gil Forcada, Ramon Navarro Bosch and I moved our Jenkins master server to a new and more powerful machine. We are using a distributed CI system with a Jenkins server, to collect and serve the test results, and a few nodes that actually run the tests. This allows us to easily scale our CI infrastructure and makes sure the Jenkins web interface is always responsive, even though the nodes are under heavy load.

For the server setup we are using Ansible, a Python-based Open Source provisioning system. We are now able to automatically generate our entire CI infrastructure, including the Jenkins master, all nodes and our custom middleware component (mr.roboto) with those two Ansible playbooks:

• https://github.com/plone/jenkins.plone.org/blob/master/jenkins_server.yml

• https://github.com/plone/jenkins.plone.org/blob/master/jenkins_node.yml

In order to allow other people to re-use our work, we factored out the generic parts of the configuration into two Ansible playbooks:

• https://github.com/plone/plone.jenkins_server

• https://github.com/plone/plone.jenkins_node

Sven Strack and I moved our Jenkins node configuration to Ansible during Plone Open Garden 2014, after Rackspace kindly provided us with new servers. Gil pushed things further and wrote the Ansible server configuration.

mr.roboto

mr.roboto is a Pyramid-based middleware component that Ramon wrote during the 2012 Testing/CI-sprint in Barcelona. Plone core has more than 300 Python packages in separate github repositories, which makes it challenging (to say the least) to decide which package needs to be build and which CI jobs needs to be triggered for a specific commit. In addition, the way the Jenkins git/github plugins are written, makes it very hard to pass git commit messages between Jenkins jobs that use a different base repository.

mr.roboto handles this kind of Plone core-specific complexity for us. It detects commits to any Plone core repository by scanning the sources.cfg file in buildout.coredev (our main Plone development repository) and automatically creates post-commit hooks for those repositories. When mr.roboto detects a commit, it automatically adds those commits to the corresponding buildout.coredev branch.

This way we get buildout.coredev branches that contain all commits for a specific Plone version, as if Plone would have only one single repository.

Say for example you do a direct commit to the plone.formwidget.namedfile master branch. mr.roboto will automatically create two buildout coredev commits, because the master branch is used by both Plone 4.3. and 5.0. If the package would only be used by one Plone version, mr.roboto would only commit to that specific buildout.coredev branch. Those mr.roboto commits look like this:

[fc] Repository: plone.formwidget.namedfile

The “[fc]” stands for “fake commit” since it was not a direct commit to buildout.coredev. Here is an example of a real commit that is added by mr.roboto:

• https://github.com/plone/buildout.coredev/commit/17ce1ff961baf5687f342234236487ea15a275d2

Having a single repository that contains all our commits, the Jenkins github plugin can take over. It automatically starts a Jenkins job for every buildout.coredev commit. In addition the plugin is smart enough to detect which branch needs to be build. For instance, a mr.roboto commit to the 4.3 branch will only trigger 4.3 jobs and not the 5.0 jobs.

With this approach, we make sure we run the right jobs for each commit while keeping the mr.roboto logic separated from Jenkins.

Jenkins Job Builder

Jenkins Job Builder is a Jenkins plugin, which is developed and maintained by the Open Stack community. It allows us to automatically create all our Jenkins jobs within minutes. It uses a simple YML syntax and, since it is written in beautiful Python, it is easy to extend. Gil did a fantastic job over the last months, moving all our job configurations from a custom buildout recipe I wrote some time ago (collective.recipe.jenkinsjobs) to Jenkins Job Builder. Here is our configuration file:

• https://github.com/plone/jenkins.plone.org/blob/master/jobs.yml

Coredev Jenkins Jobs

We have more than 8000 tests in Plone core, including some long running acceptance tests. Running all those tests takes more than one and a half hour.

An effective CI system needs to give developers a response as fast as possible. Therefore we split up the Plone 5.0 Jenkins job into three separate jobs; one for the core tests; one for all Archetypes-related tests; and one for Robot Framework-based acceptance tests. This allows us to reduce the test execution time to 30 minutes.

30 minutes is already quite good, taking into consideration how many tests we have to run. Though, there is more to come. Cloudbees, the company behind Jenkins, just released a Jenkins workflow plugin which looks very promising. It might become a game changer for Open Source CI systems. This plugin will allow us to present the test results (which are currently spread across three jobs) in a single job, while still being able to run our tests in parallel.

Since the workflow plugin is relatively new, other Jenkins plugins need to catch up before we can use it in production (e.g. there is currently no support for the xvfb plugin, which we need to dynamically set up an X-server for our Selenium tests). Here is a sneak preview of the new unified Plone 5 job:

• https://github.com/plone/jenkins.plone.org/blob/master/flow.groovy

We think the Jenkins workflow plugin will allow us to create a complex CI and deployment pipeline in the future. This will hopefully enable us to give developers a first response within less than 10 minutes and will also significantly improve and speed up the entire Plone release process.

Github Authentication

Gil set up the Jenkins Github plugin to authenticate against github instead of LDAP. This will allow every Plone core developer to log into jenkins.plone.org without having to provide a password and start jobs manually if necessary.

Jenkins Theme

We branded and customized our Jenkins theme with the Dooney Jenkins theme and the Simple Theme plugin. This is how it looks now:

The new jenkins.plone.org theme.

Documentation and Testing

We fully documented our work in a Sphinx-based documentation that we uploaded to readthedocs.org:

• http://jenkinsploneorg.readthedocs.org/en/latest/

This way, everybody with the proper credentials (currently, Gil, Ramon, Sven and I) can set up or fix our CI infrastructure at any point.

Gil also set up a Vagrant images so we can test our Ansible and Jenkins Job Builder provisioning before pushing things to the live machine.

Future Steps

We have tons of ideas how we could further improve our setup and there are lots of things to do. See https://github.com/plone/jenkins.plone.org/issues for details.

Though, there are a few issues that Plone core developers were asking for for quite some time now.

1. Auto-generated PLIP (Plone Improvement Proposals) jobs to make it easier for core developers to test those PLIPs.

2. Testing github pull requests. This is a feature that CI systems like travis or drone.io provide out of the box but that are not easy to implement with Jenkins. We plan to allow developers to manually trigger pull request tests. In the future we might want to fully automate this procedure.

With the new infrastructure we hope to be able to provide those further enhancements soon. Stay tuned.