Planet Plone - Where Developers And Integrators Write

Lightning talks Friday

Posted by Maurits van Rees on November 29, 2024 10:25 PM

Bonnie Tyler Sprint

On 12 August 2026 there is a total solar eclipse that can be seen from Valencia, Spain. So we organise a sprint there.

This conference

We had 291 participants, 234 in person and 57 online. 13 Brazilian states (that is all of them), 14 countries.

24.5 percent women, was 13% in 2013, so that has gone up, but we are not there yet. Thank you to PyLadies and Django Girls for making this happen.

We had more than 80 presenters, about 30 lightning talks, lots of talk in the hall ways.

Thanks also to the team!

Ramiro Luz: Yoga time

Yoga exercise.

Rikupekka: University case student portal

We have a student portal at the university. But mostly:

Welcome to Jyväskylä university in Finald for Plone conference 2025, October 13-19!

Jakob: Beethovensprint

26-30 mei 2025 in Bonn, Duitsland.

Afterwards, on May 30 and June 1 there will be FedCon in Bonn, a SciFi convention.

Piero/Victor: BYOUI

Add-ons first development with @plone/registry. See https://plone-registry.readthedocs.io/

It allows for development that is framework agnostic, so it is not only for Plone. It is around configuration that can be extended and injected, which is tricky in most javascript frameworks.

Imagine it.

Ana Dulce: 3D printing

For a difficult model I had trust the process, it took a week, but it worked.

Renan & Iza: Python Brasil

We organised the Python Brasil conference from 16 to 23 October this year in Rio de Janeiro.

Next year 21-27 October in São Paulo.

Erico: Python Cerrado

31 July to 2 August 2025 is the next Python Cerrado conference.

Paul Roeland: The value of longevity

Posted by Maurits van Rees on November 29, 2024 08:58 PM

Link to talk information on Plone conference website.

I work for the Clean Clothes Campaign: https://cleanclothes.org/

After three large disasters in factories in 2012 and 2013 with over 1000 deaths, it took three years to get an agreement with clothes manufacturers to get 30 million dollar compensation. It does not bring lives back, but it helps the survivors.

See Open Supply Hub for open data that we collected, for checking which brands are produced in which factories.

Documenting history matters. Stories must be told.

The global closing industry is worth around 1.8 trillion dollars, in a country that would put them on the 12th place in the world. 75 million workers.

Our strongest weapon: backlinks. We have links from OECD, UN, wikipedia, school curriculum, books. Especially those last two don't change ever, so you should never change urls.

Plone: enable the sitemap, please, why not by default? Create a good robots.txt. I weekly check Google Search console, looking for broken links. Tag early, tag often, great tool, even if you have an AI do it.

Our website: started 1998 written in Notepad, 2004 Dreamweaver, 2006 Bluefish, 2010 Joomla, 2013 Plone 4, 2020 Castle CMS (opinionated distribution of Plone, but does not really exist anymore) 2024 Plone 6 with Volto Light Theme (work in progress). Thank you kitconcept for all the help, especially Jonas.

Migrations are painful. Along the years we used wget to csv to SQL to csv, Python script, "Franken-mogrifier", collective.exportimport.

Lessons learned: stable urls are awesome, migrations are painful. Please don't try to salvage CSS from your old site, just start fresh in your new system. Do not try to migrate composite pages or listings.

What if your website does not provide an export? Use wget, still works and is better than httrack. sed/awk/regex are your friend. archivebox (WARC).

Document your steps for your own sanity.

To manage json, jq or jello can be used. sq is a Swiss knife for json/sql/csv. emuto is a hybrid between jq and GraphQL.

Normalize import/export. We have `plone.exportimport` in core now.

In the future I would like a plone exporter script that accepts a regex and exports only matching pages. Switch backends: ZODB, relstorage, nick, quantum-db. Sitewide search/replace/sed. Sneakernet is useful in difficult countries where you cannot send data over the internet: so export to a usb stick.

A backup is only a backup if it regularly gets restored so you know that it works.

Keeping content and URL stability is a superpower.
Assuming that export/import/backup/restore/migration are rare occurrences, is wrong.
Quick export/import is very useful.

Do small migrations, treat it as maintenance. Don't be too far behind. Large migrations one every five years will be costly. Do a small migration every year. Do your part. Clients should also do their part, by budgeting this yearly. That is how budgeting works. Use every iteration to review custom code.

Make your sites live long and prosper.

Fred van Dijk: Run Plone in containers on your own cluster with coolify.io

Posted by Maurits van Rees on November 29, 2024 07:58 PM

Link to talk information on Plone conference website.

Sorry, I ran out of time trying to set up https://coolify.io

So let's talk about another problem. Running applications (stacks) in containers is the future. Well: abstraction and isolation is the future, and containers is the current phase.

I am on the Plone A/I team, with Paul, Kim, Erico. All senior sysadmins, so we kept things running. In 2022 we worked on containerisation. Kubernetes was the kool kid then, but Docker Swarm was easier. Checkout Erico's training with new cookieplone templates.

Doing devops well is hard. You have a high workload, but still need to keep learning new stuff to keep up with what is changing.

I want to plug Coolify, which is a full open source product. "Self-hosting with super powers." The main developer, Andras Bacsal, believes in open source and 'hates' pay by usage cloud providers with a vengeance.

Coolify is still docker swarm. We also want Kubernetes support. But we still need sysadmins. Someone will still need to install coolify, and keep it updated.

I would like to run an online DevOps course somewhere January-March 2025. 4-6 meetings of 2 hours, maybe Friday afternoon. Talk through devops and sysadmin concepts, show docker swarm, try coolify, etc.

Ziming Yuan: How a Beginner Built an RSS Feed Provider in Volto

Posted by Maurits van Rees on November 29, 2024 06:52 PM

Link to talk information on Plone conference website.

I am studying computer science. This summer I have participated in the Google Summer of Code for Plone. Thank you to mentors Paul, Roeland and Mike!

I have built several full-stack web apps with React, but am new to open source.

What is RSS? It is Really Simple Syndication. It is used by websites to publish updated information, such as blog or news. It lets users decide what they read every day, instead of algorithms.

In Plone the RSS feed should use a catalog query to filter contents, for example a feed of science articles or of events.

For an RSS feed content type we could use a collection behavior or a blocks behavior with a listing block. The listing block is the way to go for Volto. So I did that. Plus extra fields like maximum title length and maximum description length.

I created a view component for the content type. Important is the Express middleware with which we got information from the backend.

We decided on `rss` as Node package for the feed. We looked at another newer one called `feed`, but it did not work for us or was missing features.

There were challenges for me as person new to Plone. Do I really understand the code? Often the answer is: no. For example ZCML? i18n? XML files in the profiles? There is a really nested structure. Create a new project, then you have backend / src / rss / src / rss, at least something very deeply nested. It was always confusing to me as I did not really know what should end up in git or in a release. The worst is that I don't know what are the most important files.

And what if my code fails, it was hard to understand errors on the front end. There are just too many things that may be the source of the error.

Another challenge: what if I want to create something new? The training lets you learn by example. But what does each line of the example code actually do? Are there other options?

Missing documentation: how do prevent the user from removing the title or rss block? We found out about initialBlocks per content type. And prevent adding new blocks? There is a feature disableNewBlocks, that could be set on an individual block, but what about doing this for all instances of this content type?

An overview of the code base would be good. How different components are connected, how different files work together. A guide about how to approach the source code, what files I should look at first.

For developers new to Plone: the source code is your friend, the Plone community is helpful, but be specific in the questions you ask, and tell what you have already tried.

Source code:

Érico Andrei: Plone + pytest: Tips and tricks

Posted by Maurits van Rees on November 29, 2024 06:52 PM

Link to talk information on Plone conference website.

Plone loves tests. The first was committed in October 2002 by Sidnei da Silva. We have used lots of different testing framework. I like using pytest. The unittest framework was inspired by Java testing and you see it. pytest is simple and pythonic. It does dependency injection, which can look magical, okay. There is a large ecosystem of plugins. It has support for unittest. There is a Brazilian maintainer.

I have created pytest-plone, built upon zope.pytest. It provides fixtures and helpers. it is the default in cookieplone templates. Take a look at plone.distribution and plone.exportimport for examples.

Golden rules

If you need a Plone site in your test, you need to wrap your test in a class. That helps for not tearing down everything after every test. Start with a method that gets portal and probably get_fti, and use @pytest.fixture(autouse=True).

Use @parametrize to run the same test function with multiple inputs and output.

Use fixtures.

Trust and love pytest plugins. There are plugins for running the tests in random order, or in parallel. Code coverage is a plugin. You can start a docker container.

Use helpers in pytest-plone to check that your latest upgrade step matches the metadata profile version. Check that your add-on is installed.

plone-pytest 1.0 is coming. Small problem: the tests are breaking on Python 3.9.

Write more tests with less code. I will sprint on pytest-plone.

Fabiano Weimar: Plone on Kubernetes

Posted by Maurits van Rees on November 29, 2024 02:39 PM

Link to talk information on Plone conference website.

Kubernetes (k8s) is an open source system for managing containerised applications It provides mechanisms for deployment and scaling. It is part of the Cloud Native Computing Foundation.

Kubernetes' benefits are scalability, fault tolerant, self-healing, efficient resource utilisation, declarative configuration, rich ecosystem.

Kubernetes basics:

pods: the smallest deployable unit of computing, consisting of one or more containers. Containers in one pod can talk with each other like they are all on localhost.
deployments: manage the replication and updates of pods
RepicaSets: manage a fixed number of identical pods.
StatefulSets: manage stateful apps, lik databases
Jobs: one-off tasks
CronJob: regular tasks
ConfigMap: store configuration data
Secrets: store sensitive information
PersistentVolumeClaims: request storage
PersistentVolumes: actual storage
Pod Disruption Budgets (pdb): limit the number of pods that can be unavailable at any given time, for example during upgrades, self-healing during disaster recovery.
Autoscaler: automatically scale the number of nodes in a cluster based on workload. Resource optimisation.
Ingress: routing external traffic to services, SSL termination, load balancing, caching, rate limiting.
Operator: software extensions to Kubernetes that make use of custom resources to manage applications and their components. Think of this as a human operator who would go in and do some stuff.

Plone components as Kubernetes resources:

Plone backend: StatefulSet
Plone frontend: Depoloyment
PostgreSQL: StatefulSet
Varnish: Deployment
nginx
cronjobs

Kubernetes for Devops. It has a steep learning curve. For your laptop you have microk8s and k3d so you have the Kubernetes api, although without the options that you would have with multiple machines. Basics: `apt install docker.io` and `snap install kubectl --classic` and `snap install heml --classic`. And read https://k3d.io/

At the Alpine City Sprint this year we worked on helm charts for Plone: https://github.com/plone/helm-charts/

Tips and tricks:

You can use ArgoCD to automatically do a deploy when something changes in a git repo. You need to configure this correctly of course.
Plone can do better on integration with proper observability tools like Grafana and Prometheus. For example, how can such tools get the logs when they are in multiple places.
Use ZODB with PostgreSQL and relstorage. I think it is the most stable implementation today.
Varnish Operator from IBM looks promising when you want to run multiple instances of varnish.
If you run the `test.sh` from Plone help charts this may use a lot of bandwidth downloading docker images. k3d supports import of docker images so they are in a cache.

Rikupekka Oksanen: Building a secure video portal for researchers

Posted by Maurits van Rees on November 29, 2024 01:54 PM

Link to talk information on Plone conference website.

We needed a video platform for our university. Difficult, but we had a dev team of 8 with some architecture experience, so let's try. We did a successful POC in 2023.

We use: Plone 6, VideoJS player, Wowza streaming engine, Keycloak, WhisperX, Camunda 7, Nomad. Most important: open source and license free.

Security. We have it in a dedicated and separate server environment. All data is stored on own servers of the university. All data and files are encrypted. Plone is very secure. All transactions are logged. There is a process for data protection, and it has been reviewed and documented.

There was no definite deadline, which is both good and bad. We could focus on quality, but also needed to focus on other projects.

Now a nice working demo.

[Sorry, I was distracted by client work.]

Melissa Weber Mendonça: Writing and maintaining friendly (and interesting) docs

Posted by Maurits van Rees on November 29, 2024 01:15 PM

Link to talk information on Plone conference website.

I do open source for a living now. No Plone, but scipy and other scientific libraries. I was at the Python/Plone Brasil conference in 2013. At that percent 10 percent of women at the conference. First Python conference with a code of conduct. I am very happy with that. This year at Python Brasil we had 30 percent women, so very different.

I was using scipy, numpy, and I like teaching, so the first thing I had to do at my job was documentation.

Who likes to write docs? Who writes the docs in your company/project? How often do you read the docs? How valuable is documentation for you?

I want to talk about information, tools, open source, and empathy and anti patterns.

Part 1: information

Some people like to say we are not living in the information age anymore, but in the content age.

Documentation can be narrative: tutorials, how-to guides, books. Also doc strings, code comments, code review comments. Video, talks, slides. Communications, social media. Internap processes, handbooks.

How do we use documentation? We can have various goals. Document technical decisions, explain code logic, a concept, a workflow, a process, structure. Help onboard new people. "You can set the tone for the project. Reach other people, communicate news, events, announcements. What binds it together is that we try to reach other people. This is a very human interaction.

Can you find what you are looking for? Curating information is important, the writer has a responsibility to signpost what is important. Search can be very useful. Keep in mind that people like to skim documentation pages, so help them quickly jump somewhere.

There can be a sea of information. Help people navigate this. We have more information than we can process as humans. Danger is that people find old information on the internet and then file bugs because it does not work. Having official documentation helps.

For organising your content, have a look at https://diataxis.fr. Four types of docs: tutorials, how-to, explanation, reference. This is not definitive, explore and choose one taxonomy that works for you.

Reader profiles: organise your target audience. Are you writing for beginner, experienced, sighted, student, non-dominant culture/language, a child, artist, scientist? Think about who your audience will most likely be. Take care of accessibility. Identify who your reader is and organize around this. When your docs talk about a person let it not always be a white male, be inclusive.

Part 2: tools define the docs that we write.

If you use Restructured text, you probably try to avoid adding links, because it is a difficult syntax.

Often nowadays projects have "docs as code". So if you use GitHub for your code, you will use issues and PRs for your documentation as well. Often this comes with docs-driven development: if it is not documented, it is broken or does not exist. Technical writers (if they are relevant in your context) may not be used to GitHub and have their own tools. Be careful with credit, authorship, and the limitations of your tools: people who contribute to docs deserve just as much credit as someone who contributes to the code.

Media: where are your docs? On a blog, web, book, in docstrings? Be mindful of how you present your information: audience, goals, voice.

Write/build the docs. As markup language you should look at MyST Markdown (the Plone documentation uses this too). Documentation engines; Sphinx, MkDocs, AsciiDoc, Quarto. More traditional approaches Confluence, Notion, plain text, docusaurus, GitBook/MdBook.

Share it. Is it help for a command line tool? A PDF (probably less links needed, as people may print it)? Web, a book, video, etc. Does your audience know where to find you?

Update the documentation. Your docs may be perfect, but then the code changes and the docs don't match anymore. This is especially hard for images and videos. If you can generate those automatically, that would be a very good idea. You may just need to do frequent reviews. And you may need to version your docs, that old and new versions can be found.

Invite. Interactive documentation draws people in. JupyterLite, PyScript, MyST JS. Translations help for people who are not so good in English. Create docs on how to contribute to the docs.

Part 3: open source documentation

What makes this different from other kinds of docs? So great, there is no boss, so no one can tell us what to do. Oh, but there is no one to tell us what to do, so I don't do know what to do.

You want to lead by example, write docs when you are a project lead. Not RTFM, but WTFM: write the fine manual. Make sure contributors are recognised, valued. You don't need to be an expert to start writing docs, preferably not even!

Onboarding: welcome new contributors. Docs are usually the first point of contact between users and your work. Be clear about prerequisites, unwritten rules push people away. In your project it may be perfectly fine to tell potential new documentation contributors that they need a decent knowledge of C code.

The practice: When there are meetings, committees, other in-person decisions, record this somewhere, make it available for the people who were not there.

The elephant in the room: should we use AI for docs, just let ChatGPT write them?

No: LLMs are frequently wrong, poisoned or non-sensical, telling you to call functions that have never existed.
Yes: LLMs can be useful to summarise information for a quick starting point. They can help you index information and identify patterns in your corpus.

Dont't rely entirely on AI tools. If you use them, be transparent about it. Agree on a policy, for example: don't accept PRs that were generated by AI.

Avoid ephemeral media like Discord. Don't say "this is easy", that just creates unnecessary anxiety. Don't use foo, bar, a, b: you can use long variable names, it is allowed.

Celebrate human intelligence. Writing documentation is an exercise in empathy. You are not your audience. Docs are living things, they will never be finished, it is okay to deliver something now, and update later.

"You can't teach people everything they need to know. The best you can do is position them where they can find what they need to know when they need to know it." Seymour Papert

https://github.com/melissawm

Lightning talks Thursday

Posted by Maurits van Rees on November 28, 2024 09:54 PM

Martin Peeters: Plone Foundation membership

You can apply to become a member of the Plone Foundation membership. Use this form. Ask for help, especially if you are not so good in English. Don't be shy, we all suffer from imposter syndrome. If you are in this room, you probably should be a foundation member.

Paul: Sprints

In the weekend there are sprints. You don't need to be a programmer, there are lots of other things you can do. So join us this weekend.

Astrid: Democracy

There were lots of elections in democracies this year. Plone is also a democracy. Join a Plone team.

Subscribe to the Plone newsletter.

Donate

Alexander Loechel: Alpine City Sprint

The Alpine City sprint will happen in Innsbruck Austria, February 10-14.

See https://alpinecity.tirol/. Join us, and go skiing if you want.

Dante: Axolote sprint

Join us for this sprint in Mexico, June 23-27.

Erico: "Axolote was the best sprint this year by far."

Antoine: collective AI

This is a collection of packages that brings artificial intelligence in Plone. You can use it for generating summaries of articles. At IMIO plan a release in January. Contact us if you are interested.

Fred van Dijk: Site go live tips, tricks, checks, caveats

https://vmm.vlaanderen.be/ went live this morning. Went fine mostly, but of course there were issues.

The domain had to change from vmm.be to vmm.vlaanderen.be. There were absolute urls to this or a pre-production domain in some of the texts, so those need to be fixed. I did a search and replace to the blocks with a script.

Check settings in Analytics consoles, update the allowed domains.

We have iframe blocks. But where: we use collective.tileindex, meant for Classic UI tiles but reused for Volto blocks, so we know where they are.

Traefik: you can install a file provider.

Renan: electro magnetics and screws

I have a bachelor's degree in physics. I learned something about electro magnetism. An electric current going upwards, generates a magnetic field counter clockwise. Now you know which way to turn a screw driver.

Mohammed: GSoC student

This is my first Plone conference as a GSoC student. It is a global online program connecting students with open source programs. We get a mentor. It teaches us how to communicate and engage. You gave us a warm welcome at the conference, thank you. What I learned here, I did not learn in college. I'll be back. I will give a talk tomorrow about Hydra. I won't bore you, I promise.

Kim: Plone funding

You can sponsor Plone, donate to Plone.

I have been helping with sponsorships. It is 100 percent of our income. Here are our sponsors: https://plone.org/foundation/sponsorship/sponsors

You can also donate individually via GitHub: https://github.com/sponsors/plone

It is not only about us getting money. It is also about how it looks on GitHub: it helps if people there see that people are sponsoring us.

The money we have as foundation is stable. But what are we not doing? We could advertise more. Sponsor local sprints. Have some paid positions. What is the effect of not doing these things? We have a marketing team creating material, but it is not reaching a lot of people. Let's find creative way to raise funds. I will buy you a cerveza.

David Glick: Cookieplone

Posted by Maurits van Rees on November 28, 2024 08:44 PM

Link to talk information on Plone conference website.

Cookieplone is a project generator for Plone. It has several templates to choose from: backend add-on, frontend add-on, project with both.

It is recommended for Volto 18 with Plone 6.0 or 6.1.

See the Plone documentation.

A brief history of Plone installers. What makes a good installer? Minimal: you don't want to need to install a bunch of things before you can use the installer. Reliable: works each time. Portable: works on your laptop and the server, Linux and Mac (and Windows hopefully). Easy to use, batteries included, flexible. Both easy to use and flexible can be hard to get right.

Why can't you just run `pip install Plone`? You can, then you have the Python packages. But then you need scripts, configuration, add-ons, databases, maintenance jobs, a front-end, etc.

Long time ago: download a tarball and unpack it. No dependency management.

Buildout came along, this handled dependency management, could create configuration and scripts using recipes. It has served us well. It has a bit of a learning curve. The reliability and portability could be a problem, especially when the server would be different from your laptop, or you did not pin all versions, or a package could no longer be downloaded.

Then we had the Unified Installer and Windows Installer. It was a wrapper around buildout, including all the needed packages and some scripts to make sure you had the right prerequisites.

plonecli / bobtemplates were more in the area of project scaffolds.

Somewhere in the past decade it become possible to install Plone with pip.

You can use Docker to install Plone. Less useful for development.

cookiecutter-plone-starter was the way to start with Volto and Plone 6. It would give you a Plone backend package and run a separate command to generate a Volto frontend.

And this year cookieplone. It was born in May this year during the Axolote sprint in Mexico City.

What is new compared to cookiecutter-plone-starter?

It is a mono repo for frontend and backend.
There are now minimal differences between an add-on and a project.
We now use pnpm-workspaces for the frontend, which helps if you want to try an unreleased version of a package like Volto.

How does it work? It is a wrapper around Cookiecutter, running several templates, and then some actions at the end: modifying some files or deleting files that you don't want. It has custom filters.

What if I prefer Buildout, or don't want Volto, prefer unittest over pytest? Partial answer: cookieplone is opinionated, so no. But we can talk about it. You can contribute. You can also fork it if your company has different opinions.

It is expected that the templates will evolve over time. We will fix bugs or do things differently. How can I update an existing project? You could generate a new one and then copy in the custom parts that you already have.

What's in progress:

GitLab pipeline
Support for bob templates has recently landed, with some commands to extend an already generated project.
Add a documentation section in the generated project.
Option to create a Plone distribution.
Use hatch for the backend.
Run via web-based UI or GitHub code spaces.

Plone Foundation membership meeting

Posted by Maurits van Rees on November 28, 2024 03:33 PM

The Plone Foundation is a non-profit organization that exists to protect and promote Plone. We have an open annual meeting of the membership during the Plone conference. Some links:

membership meeting 2024
board of directors
The annual report should be available there too in the future.

Currently 92 active members from 22 countries. This year 45 new contributors really contributed to the Plone code repositories. We had Daniel, Mohammad and Ziming as Google Summer of Code students.

This week we had an election for board members, all four candidates have been chosen for a period of two years: Brian Davis, Gouveia, João Henrique, Martin Peeters, Rafahela Bazzanella. Thank you to Paul Roeland and Kim Paulissen for their service on the board. The other existing board members stay for another year.

Mikel Larreategi: Multilingual Plone & Volto

Posted by Maurits van Rees on November 28, 2024 03:00 PM

Link to talk information on Plone conference website.

In our region we have Basque, Spanish and French as languages, with different official rights. So a multilingual approach is important.

Basic options that you have in Plone are to set a default language and other available languages. then you get a language switcher at the top of the page to choose the language of the UI. This is useful when you have basic needs.

You can also activate the multilingual add-on. This gives more options for multilingual content. For each language, a folder is created automatically and the user interface is shown in the language of that folder. This is useful when you have most content in the main language and then have secondary languages.

You can force using an admin language for the UI using the https://github.com/miohtama/silvuple add-on.

You can also choose real multilingual content: link all translations, for example go to a Spanish news item, select Basque as language, and you are redirected to the Basque version of this same news item.

Of course an editor needs to create the translations of those pages. In the toolbar (Classic UI) are extra options for adding a translation or linking existing content as a translation. When editing, you can use the "babel view" to see the original language on the left and the editable language on the right.

The translated pages are not special items: a translated news item is still a news item, not something special. You can move it, rename it, workflow it, etc.

You can link to the same content in all languages: there is a multilingual link that is shown in the toolbar. In the html head of the page, there are links to the other languages. This is useful for search engines.

If as a visitor you go to a Plone multilingual site for the first time, Plone accesses the language-switcher view. This looks at the Accept-Language header, at an I18N_LANGUAGE cookie, or falls back to the default language.

There are some glitches and problems for SEO. Language change urls are not direct. Sometimes Plone keeps showing a `?set_language=XX` parameter in the url, leading to duplicately indexed content.

How about Volto? You need to configure the backend as you would do in Classic UI. In the Volto config you need to add some settings, like `isMultilingual = true`.

Can we use automatic translation? Yes, there is an integration with Google Translate. You need to get an API key for that from Google, then it is available in the babel translate view. And in Volto? You can use ecityclic.google_translate.

This can work for Spanish, but in Basque the automatic translations are not so good. There are some external services you could use. We made a pull request to improve this.

There is an add-on with which you can force an admin language in case you deal with some languages that you really can't read: https://github.com/miohtama/silvuple

Lightning talks Wednesday

Posted by Maurits van Rees on November 27, 2024 09:59 PM

Érico Andrei: PyTour 2024

There were five Python conferences in Brazil this year. Six with this one. If you visited all of them, that is more than 12 thousand kilometers. Two heroines visited them all!

Alexander Loechel: Secure your keys

You should only use key-based authentication for accessing servers via ssh. Use ssh-fido or ssh-resident keys. Don't store them on your machines. Get hardware keys. You can check with a command if your server supports these keys.

Alin Voinea: EEA and Plone

The EEA, European Environment Agency, is using Plone since at least 2006. In 2019 we jumped into Volto, Plone 6 in 2023, this year Volto 17.

We have 74 PyPI add-ons and 119 NPM add-ons, around 400 Docker images and 900 repositories on GitHub.

EEA has a design system, see https://eea.github.io/volto-eea-design-system/. That makes them not always work with the Volto Light Theme, but talk with us and we can look.

We use generative AI with Volto-chatbot to generate answers from all our Plone sites.

Kim: Plone user guide

Plone's user guide was written for Plone Classic. We wanted an updated version for the training: https://plone.org/userguide

This was done with https://scribehow.com/, a paid service. Can export to PDF, html.

Breno Brito: Speed run coding with AI

Demo. Prompt: make a beautiful to do app with Python and Flask.

Belisa: Eu te amo, pandas

I love you, pandas. I am working with some Brazilian data in an Excel file with 188 thousand rows. I import them using pandas and polars and compare how they do. With polars it takes 12 seconds to import. With pandas... still busy, so let's sing some karaoke. So I love you, pandas, but...

Mikel Larreategi: Translations

Plone translations are in the plone.app.locales package: https://github.com/collective/plone.app.locales/

Most translations in Volto also come from this package. Some specific translations are in Volto itself.

In the readme of plone.app.locales you can see a graph that shows how much in your language is translated. You can clone the repo and edit, or use online Weblate: https://hosted.weblate.org/projects/plone/

The documentation is here: https://6.docs.plone.org/i18n-l10n/contributing-translations.html

Piero Nicolli: Bring your own front-end

Posted by Maurits van Rees on November 27, 2024 08:52 PM

Link to talk information on Plone conference website.

Volto is already almost ten years old. Things are moving in the frontend world.

We had a client request for something specific for tourists, with native look and feel, PWA support, multilingual, good performance and UX, accessible. Of course limited budget. It did not really need dynamic routes or the whole that Plone has to offer. But they did like to use Plone for this. But did not really look like a Volto project.

Bring your own frontend. There is Vite, but we already knew NextJS and had only short time so we went with it.

I felt it was faster for us to really start from scratch with a new layout, for this project. We used framer-motion for animated route changes. And used @plone/client, @plone/providers, @plone/components, Tailwind CSS, TypeScript.

react-aria actively helps you with accessibility, it logs warnings.

After a while they wanted dynamic routes anyway, so being able to create pages in a section of the site and they could get dynamically loaded, next to the static routes. Sure. We used @plone/registry.

What if... we would show the whole site? With the dynamic routes we were almost there anyway, we could show pages with blocks just fine.

Promise: we can run Plone in Next.JS. I am working on a template for that. Features/plan: config registry and add-ons, blocks, slots, fully dynamic routes, dynamic theme. Ideas and feedback are welcome here: https://github.com/plone/volto/pull/6442.

Thijs Feryn: Varnish caching in Plone

Posted by Maurits van Rees on November 27, 2024 08:20 PM

Link to talk information on Plone conference website.

If your website is slow or is down, you lose money. How do you solve this. Some say: throw servers/money at the problem. Yes, this can work in the short term, but mo' money, mo' servers, mo' problems.

Optimize: make it faster: your Python, javascript, css, html. At some point you hit a limit.

Use caching. That is what we will talk about. Basic idea: why would you recompute if the data has not changed.

I am tech evangelist at Varnish software. I have a colleague here at the conference who speaks Portuguese, and a German colleague who speaks Spanish. So talk to them as well if you want.

New book by me: Varnish 6 by example.

The visitor of your site first gets sent to Varnish, and this either sends the request on to Plone, or serves a previous answer from its cache. That is basically how it works.

Everything that is static is easy to cache. We listen to any cache headers that are available in the request, or set by Plone in the response.

But by default, if the request has a cookie, we do not cache. But these days every website uses cookies! So we just have to deal with them properly.

You do this using the VCL language. This is a domain-specific language for Varnish only. It is not a top-down language, it builds on other settings, and you can write extra routines for this.

Let's talk about caching in problem. Disclaimer: I am not a Plone expert.

I have setup a Plone backend and Volto frontend locally with varnish, in Docker containers.

In the cache settings:

I see good things in here, you can enable caching.
Very good: you can set a url for purge requests.
A bit strange: you need to select which content types should be purged, I selected them all.
For various sorts of contents and files you can choose strong caching, moderate, weak or terse, which sets all kinds of cache headers, even some e-tags, which is cool.

E-tags are sent on again by the browser if it requests this again. Plone then checks this e-tag and if nothing has changed, it sends back a 304 Not Modified response.

What if we could store the e-tag somewhere, then you could return very fast maybe even without contacting the framework. Maybe this happens already.

Some of the rules use a Vary header, but I wonder if the Accept header is the best here. Also, watch out for filling the cache when an attacker makes request after request with a different Accept header. In Varnish we have an `accept` module that can help here.

Cache purging: on a PURGE request we clear the cache for a url. You allow this from a white list of IP addresses. Plone purges a lot of paths, but maybe not enough. I think the `/Plone` is missing from some urls. And some urls with query parameters are not purged.

I have written a VCL for this, but not sure if that is the way to go. What about tag-based cache invalidation instead? You can do that in the VCL with `add_key`. You can also strip off query parameters.

Also, you can mark things as required, instead of actively purging it. Note that you can also serve stale content while Varnish gets a fresh answer from Plone.

Can we set the stale-while-revalidate keyword in Plone, for the Cache-Control header? That could help.

Question: should you use the varnish load balancer or a specialised load balancer like haproxy? Answer: I am biased of course, but if you are already using Varnish, just use that. We have some advanced options.

Plone is already very advanced here in how it does caching. But look at the Vary Accept header, and at tag-based caching.

Guido Stevens: Public Sector & Open Source: Natural Allies

Posted by Maurits van Rees on November 27, 2024 06:35 PM

Link to talk information on Plone conference website.

The worldwide Crowdstrike outage on Windows computers this year: crazy how such a thing can happen. I have better quality control than that! How resilient is our tech infrastructure?

What is the point of writing code if the world is going to pieces?

A short history of the internet revolution. The Berkeley software was started in the sixties. It grew a lot, in the nineties Web 1.0 started. Metcalfe's law: the value of each new connection increases.

Mandelbrot wrote The (mis)behavior of markets. From bell curve to power law or the long tail: a few people get very rich and most people get very poor. That happens in networked connections. People are more geared towards linear connections.

Web 2.0, in the 2000s, went from a read-only web, to a read-write web.

Trump, Brexit, Covid, they all rocked the world.

"Post-truth is pre-fascism", writes Timothy Snyder in On tyranny. You should read this.

Web 3.0 is skipped and we just have AI. AI will swamp the web with nonsense, and no one can find your content anymore.

I had higher hopes, too. I did not sign up for this dystopia. I have an MBA in environmental economics, done OSS web development for decades, am on the Plone Foundation, and founder of Quaive.

I wrote a book Systems of intent: Open source (Plone) plus knowledge management = Quaive. Quaive changes the Plone UI. It shows an activity stream, documents, news, personally configurable dashboard. It enables a shift in the company from top-down to bottom-up. Technology can also cause problems. But is technology the problem? Or is capitalism the problem? Resistance is not futile.

Chaos theory, butterfly effect: small acts can have huge effects. You and I are all complex individuals, complex systems. It is a shift from a clockwork universe to an alive universe. Emergence: on a higher level something more happens than what you can imagine looking at the lower levels.

Adrienne Maree Brown: Emergent strategy. Adaptive: constant change. Move at the speed of trust.

Principles:

Local autonomy. Have control over your technology.
Strive for technological excellence. Our stuff needs to be good and secure.
Networked agility: how we cooperate and coordinate effectively.
Community sharing: celebrate that we are not alone.

The clients that use Quaive may not be interested in our GPK license, but they do care about those principles.

Asterix: the whole of France is occupied? No, one corner not. Same now: Half of Belgium (the Walloon region) uses Plone in basically all their towns, supported by IMIO. We got into an agreement when I visited personally.

Technical detail: using Docker, IMIO was able to host 8 sites on a server where we previously could only host one site.

Think global, act local. Open source is not only free as in beer, but free as in freedom. The community makes this possible: Free as in community. Public sector and open source are natural allies.

Fred van Dijk: Volto Information Architecture

Posted by Maurits van Rees on November 27, 2024 01:57 PM

Link to talk information on Plone conference website.

I am using Plone since 2002, but am not a frontend expert. So currently I still have a bit of an outsider view on Volto, but I am working at Kitconcept now so will gain more experience.

Central thesis in this talk:

We need to rethink and upgrade or extend our content types and blocks system for Plone 7 and beyond.

For a big project at Zest we needed to migrate a website from Plone 4 to Plone 6 with Volto. I and colleague Maurits are mediocre frontenders at best, so we had help from two frontenders, though they had no experience with Plone or Volto.

We have almost all the building blocks already: distributions, the most secure CMS, the composite page editor, dexterity behaviours.

To do: generalised and connected frontend-backend configurations, devcontainers, version 3 of the block model, more documentation.

The big project: upgrade of the main website of the Flemish environmental agency, kind of the little sister of the European environmental agency. A communication agency came up with 20+ content types and a functionality geared towards Drupal. But the existing site was Plone 4. There were far too many inconsistencies in their design. Let's just start.

Old site is still online: https://www.vmm.be. The new site will go live tomorrow: https://vmm.vlaanderen.be. Not the greatest timing, but at least I have no talks tomorrow.

Versions used in this project: Plone 6.0 backend, Volto 18 alphas at first, now 18.1.1, Volto Light Theme. Still a Cookiecutter template as basis, if we would start now we would use Cookieplone.

Main assumptions in the Volto information architecture:

Composite pages are now built-in, with blocks layout almost everywhere.
Pages are folders.

Challenge: Where do you store images that belong to a page? If you are still adding the page, there is no place yet for storing the image. Store it in the block? No: use references. Put the images in a folder and reference it from a page.

Metadata images: we have lead images for listings and header images for the top of a page. The design should limit the aspect ratios and sizes, preferably just two are three sizes. That is more pleasing to the eyes than 10+ differently sized images on a page.

Media and data (for example graphs) belong in data content types, and their presentation should be in blocks.

One block to rule them all: we have the teaser block in core Volto now. Then we define adaptations/variations for presenting the different content types.

With Volto the editors can go wild on landing pages. Danger: they also do that in detail pages, which may not be what you want. Templates to the rescue: you can create an initialBlocks configuration. We added metadata (subtypes) with which an editor could say what kind of page it is.

"Listy" blocks: you can either use a grid block with teasers, or a listing/search block. They can look the same. But the design had lots of combinations that should be supported, and then all kinds of exceptions came up. To the rescue: default variants. The default teaser block variant can adapt itself to the listing.

Responsiveness: we sometimes have 2 blocks in a listing or grid, sometimes 3 or 4, needed to deal with that. Needed to fix some restrictions with the search facets too, needing to be able to choose different vocabularies.

We have three layers: the visual side of Volto, then information architecture concepts, then the backend. Idea: connect frontend blocks with backend instance behaviours. Then you have no explosion on content types.

Image scales wrongly regenerating

Posted by PloneExpanse on February 07, 2024 06:06 AM

I had a problem with my Frankenstein stack of Plone 4 with various bits (core libraries) upgraded on it. Here’s a description of my bug: When I upload an image and try to use it in a Volto block that referenced its image scales download url (such as @@images/<random-uuid4>.jpg) the image URL didn’t work, it resulted in 404 error. When I reindexed the image in the catalog, then it worked. Now, the funky part is that I could reproduce the problem not only on my “doomed” Plone 4 stack, but also in the modern Plone 6 stack that we use for our main customer.

Cleanup zc async

Posted by PloneExpanse on January 29, 2024 01:11 PM

For my own reference, as I had to do a cleanup of zc.async tasks. The interface was too slow. # bin/zeo_client debug >>> queue = app._p_jar.root()['zc.async'][''] >>> from zc.async.queue import Queue >>> Queue.__init__(queue) >>> import transaction >>> transaction.commit() #for i in range(len(queue)): # queue.pull(0)

Plone Conference 2023 - Eibar

Posted by kitconcept GmbH on October 18, 2023 03:00 PM

The 2023 editon of the anual Plone conference happend from October 2nd to 8th in Eibar, Basque Country

The kitconcept team was present with 8 developers. Our team members gave plenty talks, keynotes and trainings.

kitconcept and friends having team dinner

Trainings

Two days filled with trainings. Free for all conference attendees. This offer is unique in the Plone community and kitconcept team members were the backbone of the trainings about how to use Plone 6. From deployment to development to deep insides into how Volto and Plone 6 works.

Alok Kumar, Jakob Kahl: Volto and React

Alok Kumar and Jakob Kahl did a two day training to help developers get started with Volto and React:

https://2023.ploneconf.org/schedule/training/volto-and-react

Check out their trainings online if you want to catch up:

Érico Andrei : Installing Plone

Our colleague Érico Andrei gave a training about installing Plone on Day 2, the 3rd of October

https://2023.ploneconf.org/schedule/training/installing-plone

Víctor Fernandez de Alba, Tiberiu Ichim: Effective Volto

Víctor Fernandez de Alba is kitconcept CTO and the Volto Release Manager. Tiberiu Ichim is a Volto team member and one of the key contributors to Volto. They gave key insights into how to work effectively with Volto. If you have experience with Volto and you want to become a real pro, this is the training you should go to.

https://2023.ploneconf.org/schedule/training/effective-volto

https://training.plone.org/effective-volto/index.html

Day One

On day one, kitconcept team members presented two talks, including the main keynote of the day.

Keynote State of Plone

Team members Érico Andrei, Víctor Fernández de Alba and Timo Stollenwerk together with Maurits van Rees of Zest Software and Eric Steele of Salesforce presented the very first Keynote of the Conference titled “State of Plone”.

Breaking boundaries: Plone as headless CMS by Víctor Fernández de Alba

Our colleague Víctor Fernández de Alba gave a presentation about the challenges faced by the Plone content management system in keeping up with modern frontend developments and the growing popularity of headless CMSs.

Breaking boundaries: Plone as headless CMS

https://2023.ploneconf.org/schedule/breaking-boundaries-plone-as-headless-cms

Day Two

Day Two was a informative Day, packed with interesting Talks, Panels and Presentations.

Volto Team Meeting

Panel: The Future of Search in Plone, 2023 Edition

Timo Stollenwerk, Sally Kleinfeldt, Tiberiu Ichim,, Eric Steele, Eric Bréhault, Rikupekka Oksanen, Érico Andrei and Guido Stevens hosted a very interesting Panel about the Future of Search Algorithms in Plone.

This panel provided a brief history and modern examples of Plone search, followed by a discussion of what improvements are needed - both from a marketing and technical perspective. This topic was first discussed at the 2011 conference and it was interesting to see how opinions had changed.

https://2023.ploneconf.org/schedule/the-future-of-search-in-plone-2023-edition

Alok Kumar : Is your Volto addon developer friendly ?

Meanwhile, kitconcept frontend developer Alok Kumar held a Presentation about what makes a developer friendly Volto Addon, and how we as a developer ourselfes can improve on the way we develop addons for Volto.

https://2023.ploneconf.org/schedule/is-your-volto-addon-developer-friendly

Rob Gietema : How to build a site using Nick

Later in the afternoon kitconcept developer Rob Gietema held an intriguing Talk about Nick, a headless CMS written in Node.js and how easy it is to build a website with it.

https://2023.ploneconf.org/schedule/how-to-build-a-site-using-nick

David Glick : Tales from a production Plone Cluster

Following Rob, kitconcept Employee David Glick shared some Details and Stories on hosting large Plone sites in a Docker Swarm Cluster.

https://2023.ploneconf.org/schedule/tales-from-a-production-plone-cluster

Érico Andrei : Unlocking the Power of plone.distribution : A Hands-On Guide

In this talk, Érico Andrei guided us through the feature-rich world of Plone Distributions.

https://2023.ploneconf.org/schedule/unlocking-the-power-of-plone-distribution-a-hands-on-guide

Local sport showcase and party

In the evening CodeSyntax organized a showcase of different local sports, including stone lifting, wood chopping and wood sawing. Timo represented kitconcept in this together with Phillip Bauer of Starzel. After that we concluded the day with cold drinks and Pinxos at the conference party.

Day 3

Day 3 was filled with quite technical presentations, providing Information on the Cutting Edge Technology Plone has to offer.

Fred van Dijk : How the Plone Foundation Ai.team manages its websites with CI/CD

On the third Day of the Plone Conference, kitconcept Employee Fred van Dijk shared the News on automating a plone Release and how to host and operate a small Docker Swarm cluster using Plone.

https://2023.ploneconf.org/schedule/how-the-plone-foundation-ai-team-manages-its-websites-with-ci-cd

Víctor Fernández de Alba : volto-light-theme: Volto Theming, Reimagined

After a quick coffee break Víctor Fernández de Alba shared the progress on the Volto-Light-Theme and its inner workings.

https://2023.ploneconf.org/schedule/volto-light-theme-volto-theming-in-2023

Timo Stollenwerk : How we built the Website for the German Aerospace Center (DLR) in less than six months

CEO Timo Stollenwerk indulged us in the Story of the Challenges of migrating large, goverment-owned websites into a Plone project.

https://2023.ploneconf.org/schedule/how-we-built-the-website-for-the-german-aerospace-center-dlr-in-less-than-six-months

Érico Andrei : Testing your Plone codebase with Pytest

A little later in the afternoon, Érico Andrei presented us with a better, improved way to test Plone codebases.

https://2023.ploneconf.org/schedule/testing-your-plone-codebase-with-pytest

Rohit Kumar : Workflow Manager with Volto

In his presentation Rohit Kumar shared the progress regarding implementing a visual workflow Manager in Volto.

https://2023.ploneconf.org/schedule/workflow-manager-in-volto

Summary

The kitconcept team continues to drive innovation in the Plone community. Volto is the default frontend for Plone 6 and dominated the topics during the conference. We are happy to be part of such an amazing community.

Lightning talks Friday

Posted by Maurits van Rees on October 06, 2023 04:52 PM

Jakob Kahl: Beethoven Sprint

We will again host the Beethoven Sprint in Bonn, 13-17 May 2024.

Lukas: PloneGov-BR

Plone portal from Brasil. https://github.com/plonegovbr

1.2 billion page view per year, 28 sites, 800k+ content items.

Plone conference Brasilia 18-24th November 2024.

Victor Fernandez de Alba: Volto PLIPs

PLone Improvement Proposals for Volto. See on GitHub.

Mikel Larreategi: Version inspection

What to do when Maurits tells you to update five packages with security fixes? We made a product for this. DigitalOcean API to get all the project lists, scp to get the instance files (bin.instance, yarn.lock), and search.

CodeSyntax: #PrettyEibar awards ceremony

The winner is Kim Nguyen with a picture of a blue house.

Michael McFadden: Have you heard about Tau?

The Tau manifesto. Tau is 2 times pie. Pie is only half the story. 1 tau is 1 turn. Much easier to teach to children. See https://tauday.com/tau-manifesto.

Kim Nguyen: Do you want customers?

Do you want glory? Do you want to help Plone? Help Plone help you. Get your Plone provider listing today!

Go to https://plone.org/providers and register.

Dylan Jay: Python meetups

In 2014 I moved from Australia to Bangkok. I started a meetup for Python. We are at meetup 94 now. There is a PyCon in Thailand December 13-15 this year.

Do you live in a town that does not have Python meetups? Create one! Build it and they will come. You need a venue. Anyone who is interested in developers, can help you. How do you get people to talk: twist their arm, get them drunk. Have two short talks rather than one big one.

The Python foundation now pays for meetups. You can use meetup.com, but you do not have to. You need a code of conduct. I needed a page, I used pyscript, Python in the browser.

Philip Bauer: Erico, I want a beer

Erico promised me a beer if I did something useful for him. Open an Plone site, do exportimport, you can now export it to one file per item.

Mikel Larreategi: Some random things

I had ideas for talks, but did not do it, so quickly they are here.

pas.plugins.oidc. Created by Mauro Amico. OpenID Connect is a layer of identification. You can install the PAS plugin in Plone and talk to such a server [works in a test setup for me as well, Maurits]. Works with various identity providers, like Google Workspace, Keycloak, EU Login. Click a link, redirect to the provider and identify there, callback, internal connection to provider, user is created in Plone and group management granted. We use it in production in two scenarios. See https://github.com/collective/pas.plugins.oidc.

Lichess: Open source chess server. Free, free, free, no ads, no tracking. 93M games played last month. I wanted to translate it into Basque. They use CrowdIn. Can we use something similar for Plone? Weblate maybe? We use this at CodeSyntax. Perhaps I will work on this the coming months for the core Plone translations.

This Plone conference started at Beethovensprint 2022. Last of of the sprint, after last dinner, with last beer in our hands, a few guys were there and approached us: "You are going to organise the conference, right?"

Thank you everyone!

Plone Foundation annual meeting

Posted by Maurits van Rees on October 06, 2023 03:47 PM

This year the Foundation membership voted in favour of a change to the bylaws, making it more inclusive, and moving to two cohorts for the Plone Foundation board, each cohort serving for two years.

New Foundation members this year: Mauro, Tanya, Jan, Karel, André, Brian, Joao, Martin. We have 100 active members from 21 countries. It you are an emeritus member and want to reactivate, contact the board.

There is a new contributor agreement process, more digital now, not fully automated yet, but easier to handle. There were 43 real new contributors, not including some hopeful GSoc students that never did anything. There were 5 real active GSoc students, thank you for your work, and thank you Google for sponsoring this. Others see that we are handling this well as organisation. And we try to bring the students here to the conference to present their work. Unfortunately there are strict rules in the EU making it hard to get visa for everyone.

Election for the next Plone Foundation Board of Directors. The result of the vote is in. For a one year term: Brian Davis, Kim Paulissen, Martin Peeters, Paul Roeland. For a two year term: Eric Brehault, Guido Stevens, Mikel Larreategi. Congratulations. Thank you outgoing directors: William, Jens, T. Kim and Erico.

Meeting adjourned.

Plone Conference 2022 - Namur

Posted by kitconcept GmbH on October 17, 2022 03:00 PM

The 2022 editon of the anual Plone conference happend from October 10th to 16th in Namur, Belgium.

The kitconcept team was present with nine developers. Our team members gave three trainings, two keynotes, and six talks.

The kitconcept team in action at #ploneconf2022. pic.twitter.com/wQV4qeVCxB
— kitconcept (@kitconcept_gmbh) October 13, 2022

Trainings

Alok Kumar, Jakob Kahl: Volto and React

Alok Kumar and Jakob Kahl did a two day training to help developers get started with Volto and React:

https://2022.ploneconf.org/schedule/training/volto-and-react

Check out their trainings online if you want to catch up:

Érico Andrei, Jens Klein: Installing Plone

Our colleague Érico Andrei gave a training about installing Plone with Jens Klein on day one about how to install Plone:

https://2022.ploneconf.org/schedule/training/installing-plone

Victor Fernandez de Alba, Tiberiu Ichim: Effective Volto

Victor Fernandez de Alba is kitconcept CTO and the Volto Release Manager. Tiberiu Ichim is a Volto team member and one of the key contributors to Volto. They gave key insights into how to work effectively with Volto. If you have experience with Volto and you want to become a real pro, this is the training you should go to.

https://2022.ploneconf.org/schedule/training/effective-volto

https://training.plone.org/2022/effective-volto/index.html

Day One

On day one, kitconcept team members presented three talks, including the main keynote of the day.

Keynote Érico Andrei

Plone Foundation president, and our teammate, Érico Andrei kicked off the first day of the conference with a keynote about the Plone community.

Case Study by Timo Stollenwerk

kitconcept CEO Timo Stollenwerk presented a case study about how kitconcept relaunched hundreds of websites with the Plone 6 stack.

Plone at Scale: How Plone Powers Hundreds of Websites at one of the Largest Research Institutions in Europe

Alok Kumar

Alok Kumar presented “A Deep Dive Into Internals Of Volto” where he explained the different configuration options in Volto.

Day Two

Plone 6 beyond 2022

Volto Release Manager Victor Fernandez de Alba presented the roadmap for Plone 6 and Volto beyond 2022.

https://2022.ploneconf.org/schedule/talks/plone-6-beyond-2022

David Glick: The past, present, and future of how we test Plone

https://2022.ploneconf.org/schedule/talks/the-past-present-and-future-of-how-we-test-plone

Érico Andrei: Brazil still loves Plone

30 Million visitors on a single day. 670k content objects. If you are running a Plone site at scale you definitely want Érico Andrei on your team. He helped the Electoral Justice department in Brazil to run and scale Plone for the upcoming elections.

https://2022.ploneconf.org/schedule/talks/brazil-still-loves-plone

Rob Gietema: Nick: A Nearly Headless CMS

Rob Gietema started plone-react together with Roel Bruggink at the Beethoven Sprint 2017 in Bonn at the kitconcept office. This first became Volto and later Plone 6.

For Rob, this isn’t enough. Check out his latest “pet project” Nick CMS. A NodeJS-based CMS headless CMS backend that works flawlessly with Volto.

https://2022.ploneconf.org/schedule/talks/nick-a-nearly-headless-cms

Day 3

Timo Stollenwerk, Victor Fernandez de Alba: State of Plone

Plone Release Managers Maurits van Rees, Eric Steele, Victor Fernández de Alba, and Timo Stollenwerk started day three of the Plone Conference with the third keynote about “The State of Plone”:

https://2022.ploneconf.org/schedule/talks/state-of-plone

Summary

The kitconcept team continues to drive innovation in the Plone community. Volto is the default frontend for Plone 6 and dominates the topics during the conference. We are happy to be part of such an amazing community.

Volto recipe for footer actions managed as site content

Posted by PloneExpanse on July 18, 2022 02:10 PM

Managing the Footer as content is one of the common tasks on a Plone / Volto website. One typical approach is to designate some root folder, let’s say footer-links as a container for Link instances, and use those links as shortcuts to dedicated pages. So, a footer component may look like this: import React from 'react'; import { getContent } from '@plone/volto/actions'; import { useSelector } from 'react-redux'; import {UniversalLink} from '@plone/volto/components'; const Footer = () => { const footerLinks = useSelector((state) => state.

Custom Volto configuration to fix Babel problems with react-leaflet

Posted by PloneExpanse on February 06, 2022 04:05 PM

I’ve started working on a new Leaflet-powered Volto map block and the first thing that happened while loading react-leaftlet was an error reported by the browser: Module parse failed: Unexpected token (10:41) in @react-leaflet/core/esm/path.js ... const options = props.pathOptions ?? {}; ... The problem is that is, for some reasons, the transpiled JS bundle includes code using the nulish coalescing operator This is already a problem reported in react-leaflet and it happens because the distributed transpiled library includes that code.

On The Road to Plone 6 - Volto 14 Released

Posted by kitconcept GmbH on December 22, 2021 09:24 PM

Volto continues to innovate at a fast pace towards Plone 6. Today we released another major milestone on our road to Plone 6: Volto 14.

Volto 14 was in the making since September 2021 and it is in active use in various projects at Eau de Web, Red Turtle, Rohberg, kitconcept, and others.

Volto 14 comes with a set of new exiting features: a new search block that supports faceted search, locking support, a new seamless mode that makes deploying Volto easier, a new mobile navigation and support for Node 16.

Faceted Search Block

The new search block allows editor to create sophisticated faceted searches through the web without writing a single line of code.

Editors can define criteria for the content that is listed, like in the existing listing block in Volto.

Then editors can then choose arbitrary facets that are displayed to the users to choose from to narrow down the search.

Locking

Locking in a Content Management System is a mechanism to prevent users from accidentially overriding each others changes.

When a user edits a content object in Plone, the object is locked until the user hits the save or cancel button. If a second user tries to edit the object at the same time, she will see a message that this object is locked.

Content object in Volto 14 that is locked by another user

Locking requires at least plone.restapi 8.9.0 or plone.restapi 7.4.0 to be installed.

Seamless Mode

The new “seamless mode” allows zero configuration deployment by avoiding hardcoded environment variables in builds involved, and establishing good sensible defaults when setting up deployments (and also in development). So the developer/devOps doesn’t have to overthink their setups.

These are its main features:

Runtime environment variables
Unified traversal ++api++
Use Host header to auto-configure the API_PATH

and these immediate benefits:

Avoid having to expose and publish the classic UI if you don’t really need it
If possible, avoid having to rewrite all API responses, since it returns paths that do not correspond to the original object handled and “seen” from Volto, so you have to adjust them (via a code helper) in a lot of call responses.
Simplify Docker builds, making all the configuration via the runtime environment variables

Seamless Mode requires at least plone.rest 2.0.0a1 to be installed.

New Mobile Navigation

We polished the mobile navigation for Volto 14 to improve the user experience on mobile.

New mobile navigation in action on the Plone Conference 2021 website

Node 16

Volto 16 will support Node 16 in addition to Node 14 and Node 12. Node 16 will receive security updates until April 2024.

Node 16 release schedule from nodejs.org

Plone 6

Volto 14 is an important step towards Plone 6. Volto 15 will switch the default editor from DraftJS to Slate editor and it is planned to be ready in Q1/Q2 2022. This is the last big step for Volto before Plone 6 can be released.

David Ichim: What's new in Plone 6 frontend for developers

Posted by Maurits van Rees on November 08, 2021 11:18 AM

Distilled from the latest work done in Volto, we're showcasing some patterns, features, or enhancements that have landed in Volto from the last year to the present. We will also have a glimpse of what is ahead in the future of Volto with the new features roadmap.

In the past year we had four major releases, 40 minor releases, 36 alpha releases, 25 patch releases, for a total of 105 releases. Plus some new tooling and tool releases, like plone i18n and plone generator.

New Volto config, dubbed as Volto's Configuration Registry, introduced to fix circular import dependency problems. Hot Module Reloader was fixed. Read the upgrade guide.

New i18n (internationalisation) infrastructure. This is now a separate package. Same is used for generation of i18n in add-ons.Read the upgrade guide.

Forms as schema. Forms should be constructed from schemas instead of by hand. InlineForm allows us to create forms for blocks from a schema. Blocks can have variations, or we can extend it. Read the edit component documentation.

New widgets:

Object List Widget. Similar to the original DataGridField. Used in core by the Search Block facets.
Object Browser Widget is now a separate widget, instead of part of a block, and now allows the addition of external content.
Querystring Widget. Behaves like its counterpart from plone.app.querystring. Allows to create search criteria, used by the search block.
URL Widget. Used on text inputs, it knows to validate their value as a url, both internal and external.
Vocabulary Terms Widget for a JSONField, acts as a source for a SimpleVocabulary or Choice field. Play with it in the storybook.

New core blocks:

Search block
Teaser block (ongoing work).

Pluggable framework, similar to React's Portal component. See the talk by Tiberiu and see the Pluggables development recipe.

Storybook provides a sandbox to build and test visual components in isolation. Currently it is only setup to be used by Volto core. We need help with work to have storybook setup with adding. See the Storybook talk held by Victor, and see the storybook itself.

Critical CSS: inline the critical CSS for improved performance. Run critical-cli to output critical.css. This is then inlined in the headers, while regular CSS is moved to the bottom of the body. See the deployment documentation.

Lazy Loading utilities. Introduced injectLazyLibs HOC wrapper to inject lazy-loaded libraries as properties to your components. These components are only loaded once all your main libs are loaded.

Express.js middleware. Volto uses this for SSR and static resources serving. You can now write custom middleware and add it to settings.expressMiddleware.

API expanders allow the expansion of different API endpoints from Volto with calls from your custom endpoints. Avoid adding too many expanders if they are not critical to the initial page.

External routes: useful when another application is published under the same top domain as Volto. If Volto would fetch the content for that path, it will break. You can disable this path in config.settings.externalRoutes. You can also use regular expressions.

Seamless mode, introduced in Volto 13, enhanced in Volto 14, which has already seen a lot of alpha releases. Originally, we tried to unify both frontend and backend servers under the same path, but this was tricky, causing various problems. We settled on a new ++api++ traversal for getting information from the backend. Also, to come closer to zero config, you can now pass environment variables at runtime instead of build time. This means you can generate one build, and use this in all environments (testing, production). Read the deployment documentation.

Context navigation component. This is a navigation portlet component, similar to Classic Plone. The view is there, but you need to enable it. See the development recipe.

There is some work in progress:

Slots are Volto's answer to portlets, see the Volto Slots talk by Tiberiu Ichim for more details.
Image proxy: image scale generation done by a middleware instead of plone.scale.
Authentication from backend.
Replace Draft.js editor with Volto Slate. What is missing is a migration tool from one to the other. But work has started on a block conversion tool.
Async blocks that work with SSR.
Defaults in blocks form.

Future work:

Defaults in all widgets
Enable blocks enhancers in all blocks
Storybook in add-ons
Use newest react-intl package
Refactor the folder contents component
Form editing text enhancements, making it easier to modify text inputs.
A "group block" included with Volto
Quanta toolbar

I did nothing, I just brag about what others have done. So thank you Volto early adopter community!

Tiberiu Ichim: Volto Pluggables

Posted by Maurits van Rees on November 08, 2021 09:55 AM

This presentation is an introduction to the new Volto developer-targeted feature, the Pluggables framework. It is more an argument for extensibility in CMS UI and in Volto.

Basically, with Pluggable a central component provides a pluggable slot that other components can fill, like this:

<div className="toolbar">
  <Pluggable name="toolbar-main" />
</div>
// ...
<Plug id="quicklinks" pluggable="toobar-main">
  <Button />
</Plug>

This is a Volto port of https://github.com/robik/react-view-slot.

The big picture:

I work with Eaudeweb Romania
Our client EEA is an early adopter of Volto
The strategy is: move everything to Volto
But the EEA sites are less brochure, the CMS side is really strong.
We build powerful UIs for power users.
The EEA already has 91 Volto repositories on GitHub. How can we scale that? Can we write an add-on to make it easier to write an add-on?

In React, data flow is top to bottom. A parent component passes properties to children and children communicate with the parent by emitting events. This makes sense and works well. For "out of tree" data you need Redux. There is no protocol for add-hoc communication between components.

UI state is fluid. Extensibility means reusability and scalability. This is hard. You need to design upfront. Plone backend uses the Zope Component Architecture, which means pluggability is baked in, it is very natural. You can view Pluggables as viewlets-on-demand, but they are really not. But yes, you can think about a Pluggable as a viewletmanager and a Plug as a viewlet.

You can overwrite a Plug with a Plug, by registering it with the same id. So if the original Plug gives you color blue, you can overwrite it with color red.
You can do custom rendering of Plugs within your Pluggable, by iterating over all Plugs and for example wrapping each in a div with a class name.

Showcase: volto-workflow-progress (main toolbar plugin) and volto-editing-progress ("sidebar" for plugin).

Limitations:

No Server Side Rendering
Watch out for dependency lists.
Limited adoption for now.

Implementation in pseudocode:

First the context:

<PluggablesProvider>
  <Pluggable name="top" />
  <Plug name="delete" />
</PluggablesProvider>

Then the dumb version of the Plug:

const Plug = ({id, children}) => {
  const { register } = useContext(PluggablesProvider.Context);

  React.useEffect(() => {
    register(id, () => children);
  });

  return null;
}

Takes a bit of study, but in the end it is not so hard.
The Pluggable becomes a bit simpler:

const Pluggable = (name) => {
  const { getPlugs } => useContext(PluggablesProvider.Context);
  return getPlugs(name).map(f => f());
}

Site updated from Plone 2.5 to Plone 6.0

Posted by Maurits van Rees on November 04, 2021 11:34 AM

I am one of the Plone Release Managers, and have been working on Plone 6, which is now in alpha stage. But my personal website was still using the ancient Plone 2.5:

Often I have made plans to update my site to:

Plone 3
Grok
Plone 4
Plone 5
Plone 5.2
finally Plone 6

Long ago it was clear to me that an inline migration would not be practical. It would take too many steps: update the code to Plone 3.3, migrate the data, to Plone 4.3, migrate data, to Plone 5.2 Python 2.7, migrate data, Plone 5.2 Python 3, migrate data, Plone 6, migrate data.

Additionally, the question was how to handle the weblog, which is the main content. This was using Products.Quills, a Plone add-on for blogs. Updating to Plone 3 could have worked at the time, but this was made harder by some some custom code I added. This enabled me to use it as a podcast. I used this to enrich some of my summaries of sermons from my church with the actual audio of the sermon. I doubted whether to even include this content in Plone 6, as the last sermon was from 2008. I hate breaking links, so I kept it, although a bit hidden.

Another point that needed some extra attention, was that most, if not all, blog entries were not written in html, but in ReStructuredText. I make a lot of summaries of talks when there is a Plone Conference. The html editor on Plone 2.5 did not work anymore, or I disabled it to a simple textarea. I always open up the same text editor that I use for programming (previously Emacs, currently VSCode), and type the summary there. I much prefer writing ReStructuredText there, especially when I simply need text without any markup. I then paste it in Plone, without fear of losing all my work when my internet connection dies.

Lastly, I have an RSS/atom feed which is used by planet.plone.org and maybe others to stay updated when I add a new blog entry. I did not want this url to change.

Anyway, about six years ago I decided that I would use collective.jsonify to export my site, and then import it using transmogrifier. But time passed without any progress. This year, collective.exportimport was shaping up to be the preferred way to import the data. For export you can use it in Plone 4.3 as well, but definitely not in Plone 2.5.

At the beginning of this week I looked at jsonify. Didn't I have a local copy of my website on my laptop, with collective.jsonify installed? No! And it was not installed on the live site either. And installation would be hard, because the site uses Python 2.4, and currently you cannot even reach PyPI with older versions of Python 2.7.

Mildly panicked, I started on a custom script to export the content, still as json. Some details aside, this actually was not so hard. I looked at what collective.exportimport expected, and created the Python list of dictionaries accordingly. Then do a simple json.dumps() call and you are done. Except that this gave an ImportError: the json module is not available in Python 2.4. Okay, install simplejson package. But you need PyPI access for that, which does not work. Workaround:

Manually download an egg of Python 2.4-compatible simplejson 1.7 and save it in the buildout directory.
cp bin/instance bin/instance-json
Edit the new script and add the simplejson egg to the system path.
bin/instance-json run export_mvrsite25.py

After that, it was not too hard anymore. I used plonecli to create a new add-on with Plone 6.0.0a1. I actually do not yet use the add-on code, except for loading a minor patch that I added, but this gave a reasonable, modern Plone 6 buildout. Create a Classic Plone site, tweak a few settings (let Folder use folder_workflow, allow English and Dutch, navigation depth 1, enable syndication, configure caching and mail), import the content with collective.exportimport, edit a bit, and done.

The weblog now consists of standard Folders and Pages. To improve the view, I added a Collection, showing the latest pages first, and showing all content of the last seven blogs. I enabled syndication here, so it has an RSS/atom feed.

The weblog has always advertised two atom feeds:

One for all weblog entries, at https://maurits.vanrees.org/weblog/atom.xml
One for weblog entries with keyword 'plone' at https://maurits.vanrees.org/weblog/topics/plone/@@atom.xml

In the new site, the first one kind-of worked out of the box, but it only showed the items that were directly in the weblog folder, and this is not where my weblog entries are. I solved this with a patch to Products.CMFPlone.browser.syndication.views.FeedView: when atom.xml is viewed on a folder, check if it has a default page, and show the atom.xml of this default page instead. In this case, the default page is a Collection with the correct settings. So the general feed will keep working.

For the second one, my first idea was to create a folder 'topics' and within it a Collection with id 'plone'. Problem: 'plone' is a reserved word and cannot be used as id of a content item, so the id became 'plone-1'. Solution here: create the 'plone-1' collection directly in the weblog folder, and do a redirect in the frontend server (nginx):

 rewrite ^/weblog/topics/plone/@@atom.xml /weblog/plone-1/atom.xml last;

And that's it! My website is now on Plone 6.0.0a1:

There are some more details that I could go into, like splitting up the buildout into multiple parts, with tox as main way to build everything, in preparation for moving more and more parts to pip-only. But that will have to be for another time.

Meanwhile: have a look around, and enjoy the fresh look!

Steering Circle

Posted by Maurits van Rees on October 31, 2021 02:39 PM

Volto 14 alpha 23 is out. So still in alpha, but companies are using it in production. Should be final soon. Some plans for Volto 15. Created plone.volto integration package, where we try to give an easy transition from earlier company-specific versions. plone.restapi as always is pretty boring, stable. Erico worked on Docker integration.

Plone 6 alpha 1 is out. Eric sent an email for some coordination, like docs, training, accessibility, installers. If you want to be involved, let me know.

Franco has stepped out of the Framework Team, thank you for all your work. There is discussion about the role of the Framework Team. Plan is to keep it running, some more people have been asked.

Membership team: we have some people in sight as new members. Erico is stepping down as team lead, Victor is stepping up.

Security: Plone 6 will have 5 years security support. Synchronizing with Zope team. Some new members may be coming.

Marketing: busy with conference, also after the conference. Busy with plone.org renewal.

Installers: see the talk by Jens Klein earlier. Plone 6: no more installer, but we do have tooling. There are Docker images. We may want to reduce the role of buildout, and focus more on pip.

Plone Conference: you are looking at it. Some tasks to do afterwards. If anyone is interested in getting a certificate for following a training, ask us, and we can send it.

Internationalization: new branches for Plone 6, so Plone 5 uses different branch. New releases for 5 and 6. Updating po files, looking at i18n in Mockup.

Admin/Intrastructure: servers are still running. Cat herding sysadmins for doing stuff, keeping things up to date.

Trainings: relaunch is complete. We have three new trainings: Theming Plone Classic, Deploying Plone 6, Getting Started with Plone 6 (that one only in video). Various have seen major updates. Need to work on landing pages (we have two), remove the number 5 from the url, update some more trainings. Maybe Mastering Plone Classic training, but hard with navigation due to duplicate section targets when copying. Migration training would be good. We need to prune and tag some.

Plone Classic: We did polishing on Barcelonate, it is pretty ready. Focussing on bobtemplates before the trainings, making theming easier. JavaScript/ES6 is the remaining big issue. Plan is to finish it this year, we are quite far. We need other people helping us out.

Documentation: will be releasing a new Plone 5 branch today. For the new stuff, the tech stack is ready. New version of automated screen shot is about ready. We don't want a duplicate of the training, but we can automatically include code of it, so there is only one source of truth. The style guide is not always followed, seeing what we can do about that. Biggest point is missing documentation. There are now branches where the various teams can add and edit their content. We may change things, but we take this as input for the final structure.

Fred van Dijk: 7 things that can surprise you when you start customising or developing for Plone

Posted by Maurits van Rees on October 29, 2021 09:06 AM

This is a basic rundown and summary of our beloved subjects like ZODB persistence. Traversal. The view/viewlet/portlet trinity. How is a call handled in Plone. The differences between zcml and generic setup. Utilities and the ZCA. restrictedTraverse. The Plone catalog.

These are surprises that I have encountered myself, or that I have seen on faces of people I have trained or worked with during the years.

Everything can or could be done through the web (TTW).

Zope vision from the nineties. Why don't we use this dynamic language called Python so we can change things TTW? It's so easy.

The learning curve.

It starts easy, but then you hit what we call a Z-shaped learning curve. Dynamic Python makes things easy, and then it makes things hard, at least when you put Zope on it, then CMF, then Plone. Plone the product on top of a CMS on top of a framework on top of a language. We have a product, a framework, a stack, so it is hard.

Five levels of conceptual complexity.

It helps to teach all the levels. Give new users a drawer for each level, so they have a box that they can put some info in.

You have:

browser: html, css, js
frontend: transforms, templates, zpt, metal, diazo
application logic: views, viewlets, portlets, adapters, Zope Component Architecture
dynamism: GenericSetup, zcml, xml, zope schema
programming language: Python, buildout, pip
package WorkManager (OS): apt, rpm

4 Same language/formats on different levels:

XML is used for the ZCA, GS, zope.schema, Diazo rules
package manager: buildout, pip, setuptools, GS

But: there is no magic. It is just advanced technology.

Startup:

Python uses sys.path modules to start bin/instance
ZCA loads site.zcml, package includes, other zcml to change configuration.
Then we have a runtime environment with objects and classes, the ZODB. GenericSetup is then some XML that you can use to change the ZODB.

So the ZCA overrides components in runtime. The alternative is to edit core files, maybe compile them, and restart. Much less nice and not sustainable.

So now we have a Plone process running.

Zope is not that complicated.

Over HTTP Zope gets a request, does traversal, finds an object in the ZODB, loads it in memory. Then on top of this object we show a browser view / template. The template hooks into a main template, maybe does some sub requests, some Diazo, and we end up with some html and we are done.

This is all still 'lies to children'. It is simplified until we are able to understand more. With increment exposure to these concepts, it will stick more. It is complicated, but there is no magic.

Acquisition.

It is traversal in the wrong direction.

When you try to explain things, you improve your own understanding.

There is so much Plone content online: training, docs, youtube, github, discourse. We all learn in different ways, with own preferences, reading, viewing. There are so many repositories on github that you can explore for new ideas. Just yesterday Philip did a talk about exportimport and afterwards I did it, but from a different angle. It helps.

The community is our biggest asset.

On The Road to Plone 6 - Plone REST API 7 and Volto 12 Released!

Posted by kitconcept GmbH on April 12, 2021 09:24 AM

Two weeks ago we cut two important releases on the road to Plone 6. Plone REST API 7 introduces a new link integrity feature for blocks-based pages. Volto 12 improves the add-on ecosystem by introducing a new configuration registry to avoid circular dependencies.

Photo by Stephen Leonardi on Unsplash

Plone REST API 7

Linking pages is one of the core idea of the world wide web. Keeping links within a website intact is therefore one of the core features that any Content Management System needs to provide.

One of the core features of Plone has always been that editors can copy and move single pages as well as large content trees without breaking internal links to other parts of the site. This is accomplished by using unigue IDs (UUIDs) instead of relative or absolute paths when adding a link to another page. Plone uses “portal transforms” internally to rewrite those links in RichText fields on save operations.

Plone 6 (aka Volto) introduces blocks-based page layouts, that store a JSON structure internally instead of HTML. This more structured way of storing page content and layout allows more complex page layouts. Though, because of this change, the existing portal transforms mechanism that rewrites links to UUIDs did not work any longer.

Blocks-based Volto page in edit mode (with kitconcept-blocks-grid)

Werkbank, a Plone agency from Bochum, Germany, stepped up to sponsor the development of link integrity in 2020, since they needed that for a client project.

I started to draft a possible solution and wrote a first prototype. Thomas Buchenberger from 4teamwork picked up that work at the Plone Conference sprint in Ferrara, Italy.

Andrea Cecchi from RedTurtle joined our efforts and refactored the resolveUID algorithm into a blocks transformer, that we started to use for other use cases and that made the resolveUID transformer more generic and flexible.

After that, we cut a first plone.restapi 7 alpha release and entered a longer period of quality assurance and testing.

Two weeks ago, after we tested the new feature in multiple projects at kitconcept, I cut a final release of Plone REST API 7.

After the first alpha release, the ResolveUID transformation was added to links and images. In addition plone.restapi 7 comes with a new blocks serialization mechanism and an important fix that makes sure files are opened directly by Plone for anonymous users.

We also added a “smart fields” concept that allows integrators to mark a blocks field as searchableText field.

A new @contextnavigation endpoint was added that allows for local navigations. We enhanced the navigation endpoint to expose an optional navigation title (“nav_title”) field.

Since the final 7 release, we cut six more releases and REST API 7 was included in Plone 5.2.4 release.

Volto 12

Photo by Gaelle Marcel on Unsplash

Tiberiu and Victor worked on a new Volto configuration registry which is the new central point to store and retrieve Volto configurations. The configuration registry is a singleton that ensures a setting only exists once. With more an more Volto add-ons that started to depend on each other, we started to run into circular dependencies issues that we can avoid now with the new configuration registry.

You can find more details about it in the Volto docs and in the Volto 12 upgrade guide.

Plone 6

Plone REST API 7 and Volto 12 were two very important releases on our road to Plone 6.

We plan to cut another Plone REST API 8 branch and release for Plone 6, which will support Python 3 only.

Volto will continue to move at very high pace towards Plone 6. We started to organize a series of Plone 6 “Micro-Sprints” to push things further.

Check out the Volto Roadmap on github for more details.

A Volto gotcha when dealing with async calls

Posted by PloneExpanse on December 11, 2019 08:35 PM

Just some quick notes, in case this might help someone. After quite a bit of time and tests in trying to use asyncConnect to get data in a Volto component view (strictly focusing on the SSR side), I’ve realized that what I’m trying to do is not supported by the redux-connect library. In Volto, right now there are two components that use asyncConnect: App.jsx and Search.jsx. The purpose of asyncConnect is to have the server side rendered page “dynamic”, depending on the input from the originating request.

Speedup volto razzle builds

Posted by PloneExpanse on November 17, 2019 12:58 PM

I’ve been looking for a way to speedup Volto razzle/webpack builds, both while developing and for “production” mode, when building the final bundle. Fortunately, this solution exists and it’s extremely easy to integrate. Let’s define the problem, to see how to approach it: what is Volto actually? What do you get when you open, in your browser, a Volto frontend Plone website? To greatly simplify (and I hope I didn’t get anything wrong as I am not a Volto core developer):

Plone Beethoven Sprint 2019

Posted by kitconcept GmbH on September 25, 2019 09:00 AM

21 developers from nine different countries gathered in Bonn, Germany between June 20th and 24th to work on implementing the upcoming Plone 6. The sprint at the office of the kitconcept GmbH has been declared a “strategic sprint” by the Plone framework team. Sprint topics included working on Volto, the Plone REST API, and Guillotina.

Volto

Volto is a new ReactJS-based frontend for Plone. It was started by Rob Gietema in 2017 and it is actively developed since then. Volto will become the default frontend for Plone 6. It implements a complete new user interface called Pastanaga UI, which was developed by Albert Casado.

Victor Fernandez de Alba and Rob Gietema led the efforts to further enhance Volto together with Paul Roeland, Nicola Zambello, Piero Nicolli, Janina Hard, Jakob Kahl, Thomas Kindermann, Maurizio Delmonte, Stefania Trabucchi, Rodrigo Ferreira de Souza, Fred van Dijk, and Steffen Lindner.

Pastanaga UI Toolbar

The new editing toolbar has been part of Volto right from the start. Though, so far we did not fully implement every detail that Albert has imagined for the Pastanaga UI toolbar. Victor has been working on this for quite a while already. During the sprint, he integrated his work and polished it. Victor and Rob also worked on a new sidebar that shows up on the right side of the Volto user interface that holds additional information like page metadata and controls the settings of the individual tiles.

volto toolbar The new Volto edit toolbar

New Tiles: Collection, Slider and Table

At the Plone conference in Tokyo, we came up with a list of tiles for the new Volto composite page editor that we would like to implement. On top of that list were the listing/collection tile, the slider tile and the table tile. Piero, Rodrigo, and Rob started to work on the listing/collection tile that is supposed to become a replacement for the Collection Type in Plone.

Jakob and Janina worked on copying over the code of a slider tile that was developed by kitconcept. Rob implemented the table tile. Before you could ask Rob about the progress, he already finished his work, like always…

The new Volto table tile

Accessibility

Accessibility has always been a first class citizen in the Plone community. Plone is used by many government and public websites where accessibility is a basic requirement. Paul, Thomas, Timo, and Victor worked on improving the accessibility of Volto. Our goal is to fully support the WCAG 2.1 standard with the “AA” level of conformance.

Timo set up ESlint-based static code analysis checks and Cypress-based axe accessibility tests. Paul, Thomas and Victor then started to fix the reported accessibility issues. At the end of the sprint, we were able to fix all the reported violations. Our CI build now fails if new accessibility violations are committed.

Documentation

Sven Strack, the Plone documentation team leader, joined our sprint remotely from Amsterdam to discuss how to maintain and structure the Volto documentation in the future. We all agreed that we will use Markdown as format for our docs, since this is the de-facto standard in the JavaScript community as well as in most other Open Source communities. As much as we like Restructured Text, it seems that Markdown won that battle.

We currently use MKdocs for the Volto documentation and Styleguidist to document our React components. There are lots of different tools to choose from, Docz, MKDocs, MDX, Gatsby to just name a few. We agreed that we need to do more research to make an educated decision regarding our doc toolchain.

There will be a Google Season of Docs project this year where a technical writer will help us to enhance the Volto docs further. More Volto Features Nicola added a feature to display the currently used Volto version in the Plone control panel, he added animations for page transitions and added an Italian translation for Volto. Together with Victor, he worked on a toast component for user notifications.

More Volto Features

Piero worked on improving and fixing the event type view and added styling for the listing tile.

Victor and Steffen worked on the image sidebar.

Rodrigo added a feature that allows the user to use SHIFT+RETURN to create a new line in a Volto text. By default Volto creates a new text tile on RETURN. Rodrigo also looked into the Add-ons control panel work that Eric Steele started some time ago.

Nilesh joined us remotely from India and continued his work on the users and groups control panel for Volto.

Steffen fixed a bug in the subjects search. Rodrigo and Thomas looked into our Cypress-based acceptance tests.

Fred looked into what consequences it would have for us to get rid of the description tile and how we could customize the domain in Plone to allow to send portal emails via the Volto frontend.

Stefania and Maurizio worked on a product definition of Volto and an elevator pitch for clients and users. Together with Paul, Fred and Timo, they had a longer discussion about how to position Volto and Plone 6 in the market.

Plone REST API

The Plone REST API builds the foundation for the new Volto frontend and Plone 6. It has been under active development in the last four years and seen more than 80 releases in that time period. The REST API is stable, battle-tested and production-ready and part of the Plone core since the release of Plone 5.2.

Thomas Buchberger and Lukas Graf led the REST API efforts during the sprint and worked with Carsten Senger, Roel Bruggink, Janina Hard and Timo Stollenwerk to further enhance the REST API, fix bugs, improve the documentation, and the error handling.

Plone REST API discussion in the garden

Querystring Endpoint for the Collection Type

One of the most important new features, that has been developed during the sprint, is the new querystring endpoint. This endpoint will allow us to implement the Collection type for the Volto frontend. Lukas implemented this essential feature with help from Rob, who took care of the frontend implementation in Volto.

New Features, Bugfixes, and first time contributions

Thomas added support for the retrieval of additional metadata fields in summaries in the same way as in search results. Lukas made the @types endpoint expandable and the @users endpoint easier to customize. Janina fixed setting the effective date on workflow transitions. This was her first contribution to the Plone core. On the same day she signed the contributor agreement, got her fix merged and released, which is quite an accomplishment for a single day. Way to go Janina!

In addition to that, the team fixed lots of smaller bugs (#780, #729, #764, #755, #777, #732, #738) during the sprint. REST API Error handling Carsten, Nathan, Thomas, Lukas and Timo had a longer discussion about error handling in the Plone REST API and about how we can improve things. We considered and discussed implementing RFC7807 and decided against adopting it.

Sprint discussions…

The RFC did not see much adoption and it only slightly differs from what we are already doing. A breaking change does not seem to be worth the effort. We decided to create a separate error reporting component unifying the error reporting and ensuring consistency. We also started to fix a few inconsistencies right at the sprint.

API Documentation

Guillotina is using Swagger to document its API. Nathan explained the approach he took for that and Carsten started to integrate Swagger to generate dynamic api docs in plone.restapi.

REST API Releases

Most of the work we did on Plone REST API during the sprint has been released in Plone REST API 4.1.4, 4.2.0, 4.3.0, and 4.4.0.

Dexterity Site Root

Roel Bruggink joined us on Saturday and Sunday to continue his work on turning the Plone site root into a Dexterity content object. He fixed some complex recursion and acquisition problem when migrating the site root to Dexterity. His Plone Improvement Proposal (PLIP #2454) has been accepted by the Plone framework team and it will build the foundation for a new Plone 6 branch that we plan to cut this year.

Guillotina

Guillotina is a new AsyncIO REST data API. It has been written from the scratch by Ramon Navarro Bosch and Nathan van Gheem. It can easily scale billions of objects via its REST API. With the CMS addon, it is compatible with the Plone REST API. Therefore you can run Volto on a Guillotina backend and Guillotina might be able to replace our existing Plone backend in the future.

The Guillotina master thinking hard. :)

Nathan and Ramon worked on Guillotina 5. The Guillotina 5 release brought Python 3.7 support and the use of the context vars APIs. They refactored quite a bit, moved parts of addons into core(swagger, caching, pg catalog), implemented OOTB PostgreSQL catalog, added a Helm/Kubernetes/Docker configurations for the Guillotina CMS addon that can be configured to act as a Volto backend, worked on improving caching and filestorage.

Websockets for Plone

At the Plone Open Garden 2019, Asko Soukka started to work on bringing websocket support to Plone. Websockets allow bi-directional communication between the server and the browser. With websockets, the server can send messages and notifications to the browser. This can be used to implement notification systems or collaborative editing to just name a few possible applications.

Asko continued his work during the Beethoven sprint and created an example where comments that are added to Plone automatically pop up as toast notifications for other users. He published a Docker image with a Twisted-based ZServer and implemented the foundation to use that feature for gatsby-source-plone. This will allow us to create an extension that will allow editors to live-preview changes they do in Plone in a GatsbyJS site.

Karaoke, Barbecue and Garden Parties

A Plone sprint is much more than just programming. Collaborating, meeting old friends, talk, discuss and having a good time together are the ingredients that make a Plone sprint. We started our sprint on Thursday with a free React and Volto training for newbies. At the evening when everybody arrived, we went to a local Karaoke bar for a revival of the epic Karaoke sessions we had in Tokyo last year.

Karaoke at the “Nyx”

On Friday evening we ordered Pizza and hacked the night away at the office and our garden. On Saturday we went to the “Arithmeum”, a mathematics museum owned by the Research Institute for Discrete Mathematics at the University of Bonn.

Our tour at the “Arithmeum”

Afterwards we had a garden party with a barbecue and we spend the afternoon sitting in the garden and the office and continued to drink and hack on Plone.

Barbecue in the garden of the kitconcept office

Summary

The Beethoven Sprint 2019 was a major success. We were able to start working on the main missing pieces for Volto. The new edit toolbar is in place and the foundation for the new tiles/metadata toolbar on the right side of the UI has been finished. We also implemented a new table tile.

We implemented a proof-of-concept for the new listing / collection tile and implemented the required querystring endpoint in Plone REST API. Both Volto and the Plone REST API got lots of enhancements and bugfixes.

Both the Plone REST API and Volto are stable and used in production. We will use the next month to further enhance Volto and implement the missing bits and pieces. Our plan is to present Volto 4 at the Plone conference in Ferrara at the end of this year and cut a Plone 6 branch afterwards that contains the Dexterity Site root, folderish content types and other enhancements that are needed make Volto the default frontend for Plone 6.

Guillotina and the Websockets support for ZServer together with Gatsby as another frontend for Plone are amazing projects by the smartest folks in our community. Our diversity is one of our main strength and we are looking forward what the future will bring!

We had a great time at the sprint. Thank you to everyone who attended! We thank the Plone and Python community for their ongoing support. The Plone Foundation, the Python Software Foundation, and the German Python Software Verband made this event possible with their sponsorship as well as our company sponsors Abstract, Starzel, Werkbank and Zest.

We are looking forward to see you all in Bonn next year!

The “Hofgarten” near the kitconcept office

Help Oshane get to the Tokyo Plone Conference

Posted by David "Pigeonflight" Bain on October 24, 2018 03:38 AM

Summary

Oshane Bailey, a talented Plone developer with loads of Plone experience has been selected as a presenter for the 2018 Tokyo Plone Conference. His Japanese visa was just approved.
He will share a streamlined approach to Plone development that he is applying on a Plone project targeted at Jamaican Developers. At the time of writing he has raised about 17% of the funds he needs to get to Tokyo. You can help him get to Japan by contributing to his crowd-funding campaign.

Oct 29, 2018 update
Thanks to generous contributions, Oshane's trip is now 70% funded. You are welcome to join the crowdfund and cover the rest of his trip.

Oct 30, 2018 update
Oshane's trip is now 89% funded. The plane ticket and conference ticket have been purchased. you can still pitch in by joining the crowdfund to cover the rest of his trip.

Since at least 2015, Oshane has worked on Plone projects for teams around the world and in the process has been exposed to varied approaches to the development and ongoing management of Plone sites. Over recent months he has poured his, hard earned, experience, into a side project -- the Jamaican Developers site. Through this project he has refined a continuous development pipeline based on some of the best techniques used in the Plone community and enhanced with some of his own innovations.

Last year Oshane participated as a Plone Google Summer of Code student and presented his work at the Barcelona conference. He also participated in the after-conference sprints, contributing to efforts to port Plone to Python 3 and also looking into the WebSauna project.

Supporting his trip to Tokyo will serve to enrich PloneConf 2018 in many ways. Here are three that spring immediately to mind:
1) As part of his talk he will share the techniques he is using on the Jamaican Developers site
2) He plans to participate in the after conference sprints.
3) He will bring an important perspective to discussions influenced by constraints common to Jamaican developers.

How to Support Oshane

Appropriately, his crowdfunding campaign is running on the Jamaican Developers site that he built with Plone. His goal is to raise enough to cover his travel and expenses related to the Japan trip.

As we say in Jamaica... "Follow back a me" as I support Oshane's trip to PloneConf2018 in Tokyo.

Pastanaga Editor Status Report - Plone Beethoven Sprint 2018

Posted by kitconcept GmbH on July 20, 2018 11:24 AM

During the Plone Beethoven Sprint in Bonn, we worked hard on creating a first version of a new content editor for Plone-React.

Here is a short demo of what the editor looks like right now:

Demo of the Pastanaga Editor with tiles

We already had a first implementation based on DraftJS that allows inline styles (e.g. bold, italic), block styles (headlines, (un)ordered lists), and links to remote URLs.

The new version of the editor is based on a “tiles” backend, that is build by Victor Fernandez de Alba during the sprint and released with plone.restapi 3.2.0.

This allows us adding more complex content elements such as images, videos, and in the future more complex layout elements.

With the new backend in place, Rob Gietema went ahead and implemented the basic editor that Albert Casado designed as part of the new Pastanaga UI for Plone.

Mobile Pastanaga Editor design by Albert Casado

The user can type in the title, description and the text content of the document without worrying about form fields or be distracted by tabs and fieldsets.

In addition to the standard text editing it is now possible to add an image tile that can be placed on the left or right side, on the center of the page or in full page width.

Rob also added a YouTube tile that allows the editor to add a YouTube video URL and then displays the video within the editor and the page view.

Text, image, and video tiles can be added to a page. They can be deleted and moved up and down to change the order of the elements.

Next Steps

The new editor is a great accomplishment. The tiles endpoint in plone.restapi allows us to further enhance the current version of the editor with more advanced layout variants and tiles.

We plan to continue with our iterative and agile approach of building a useful, fully functional version of the editor with each step, that allows Plone companies to use the editor and Plone-React today in their client projects.

The next steps are polishing the editor and the existing tiles. Work out some UX issues that we found when working with the editor and building more advanced tiles.

Stay tuned for more news and features after the Costa Brava sprint…

Plone Beethoven Sprint 2018 - Sprint Report - Day 3

Posted by kitconcept GmbH on June 30, 2018 06:24 PM

Report of the third day of the Plone Beethoven Sprint 2018 in Bonn, Germany.

The Beethoven Sprint was a “strategic” sprint that took place June 21-25 at the kitconcept office in Bonn, Germany. The focus of the sprint was to work on the Pastanaga editor, Plone-React, plone.restapi, and Guillotina. This is the report of the third day of the sprint.

Day three wrap-up meeting at the kitconcept office

UTC DateTime Discussion

After having breakfast and doing a quick stand-up, we started a discussion about how to store datetimes in Plone. This was a discussion that came up when we implemented plone.restapi. Though, this question is more related to plone.app.event. As seen many times in the Plone community, when it comes to complex technical matters, offline discussions work way better than discussing issues back and forth via github comments.

Since we had all the experts at the sprint, we decided to schedule a discussion to solve the issue. In the end we reached an agreement that dates should be stored as UTC and that the local time zone should be stored separately in an additional field, for instance to calculate recurring dates.

plone.restapi

Thomas Buchberger worked on the workflow endpoint and added a feature to change the workflow state recursively, to set the effective and expiration date and to allow workflow comment.

Lukas Graf finished his work on implementing redirects in plone.rest based on plone.app.redirector. The redirects do not only work for simple GET requests but also redirects POST, PATCH, und DELETE requests.

Sune Brøndum Wøller finished the portlets endpoint for plone.restapi.

Mikel Larreategi made plone.restapi return cachable image resources to improve the caching of images in plone.restapi.

Roel finally wrote a PLIP for the IPloneSiteRoot and fixed edge cases and a few bugs in the current implementation.

Plone-React

Victor completed his work on plone.schema, allowing to use a JSON field in Plone that validates the JSON structure within the field and also allows through-the-web editing. He also finished the backend behavior for tiles and blocks to store the tiles and the tiles layout on a Plone content object.

Eric Steele added a backend implementation for the add-ons control panel that he wrote for Plone-React. Eric presented a fully working add-ons control panel that allows to install add-ons via Plone-React by the end of the day.

Plone-React add-ons control panel presented by Eric Steele

David completed his work on the vocabularies endpoint by implementing the frontend widgets that rely on those vocabularies to make sure the endpoint serves its purpose.

Davi Lima and Victor continued to work on the override mechanism for Plone-React (JBOT) to customize widgets and views in a config.js file using a babel plugin.

Pastanaga Editor

Rob presenting the new video tile for the Pastanaga editor

As always, Rob did not give us any time to breath and added a YouTube video tile to the Pastanaga Editor, that shows a YouTube video in the editor itself and on the content view.

Of course that wasn’t enough for a day, so he fixed a few other smaller issues and worked on exposing the sitemap.xml via Plone-React.

Dinner and some late night hacking

Dinner at “Hans im Glück”

Rob might not admit it, but after claiming he could fix all those issues mentioned above in one day, he continued to work on the sitemap during our dinner at a local burger restaurant in the inner part of the city. We’ve been there ourselves, so we didn’t mind, while we were enjoying our tasteful cocktails. After a good meal, some drinks, and having to say good by to Eric, we went back to the office.

We said good bye again to our fantastic Dutch Plone-React team Rob and Roel and then went on to hack a little bit more on the stuff we were (and still are) so enthousiastic about.

Plone Beethoven Sprint 2018 - Sprint Report - Day 2

Posted by kitconcept GmbH on June 29, 2018 06:24 PM

Report of the second day of the Plone Beethoven Sprint 2018 in Bonn, Germany.

Guillotina

After having breakfast at the office, we started the day with a stand-up/wrap-up meeting with Ramon Navarro Bosch presenting Plone-React running on Guillotina. The user can add new content, edit existing content, and browse the content with Plone-React and Guillotina, which is a huge acomplishment and a promise for the future of Plone. After the stand-up Ramon had to say good bye to the other sprinters and head home to Barcelona.

Second day at #Plone #Beethoven sprint 2018! Standup time!! pic.twitter.com/qmkCP0K6oR
— kitconcept (@kitconcept_gmbh) June 23, 2018

Pastanaga Editor

Rob Gietema continued his work on the Pastanaga editor. At the end of the day, he was able to present a working version where the user can add a title and a description tile as well as a text tile with inline styles such as bold, italic, headlines, links, and lists.

Pastanaga Editor with basic text editing

Rob also added an image tile that allows you to upload images. The uploaded images then can be aligned left, right, in the middle and in full width or just be deleted.

Image tile that can be aligned left, right, center and displayed with full width.

Victor Fernandez de Alba finished his work on the tiles backend for the Pastanaga editor (https://github.com/plone/plone.restapi/pull/547), so Rob could present a fully functional editor that actually stores the content (text and images) in Plone tiles.

Plone-React

Eric Steele continued his work on the add-on control panel and was able to present a first version at the wrap-up at the end of the day.

Add-ons control panel written in React by Eric Steele

Carsten Senger worked on implementing the users and groups control panel in Plone-React which now allows to add and delete users.

Victor worked on being able to use Plone-React as a library that allows developers to override React components. The system is supposed to work the same way as JBOT (just-a-bunch-of-templates) works in Plone.

Johannes Raggam and Andrea Cecchi continued their work on the reference widget and were able to present a first prototype. We agreed that we have to put some effort into the UX/UI of that widget, but that was beyond the scope of a sprint and requires the help of a UX specialist.

plone.restapi

Thomas Buchberger finished the pull request for the object create order and started to to work on the workflow endpoint enhancements.

David Glick worked on enhancing the vocabularies endpoint with batching and filtering.

Sune Brøndum Wøller fixed some nasty bugs with time freezing and transactions errors in the Plone 5.2 of plone.restapi (e.g. https://github.com/plone/plone.restapi/pull/562, https://github.com/plone/plone.restapi/pull/561).

Lukas Graf continued his work on the translations of the REST response data, fixed a few bugs and wrote a script that tests the Sphinx-based documentation for warnings and errors and fails the Travis build if there is a problem.

Mikel Larreategi finished the pull request for the history endpoint and documented the Accept-Language headers in plone.restapi. He also started working on making the image scales that plone.restapi returns cachable.

Roel Bruggink continued his quest on the folderish site root and fixed errors in CMFPlone, plone.app.content, and in plone.app.dexterity.

We were able to release plone.rest 1.1.1 and plone.restapi 2.1.0 as well as plone.schema 1.2.0 with a JSON field that is required for the tiles endpoint in plone.restapi.

Hacking Night

Germany was playing at the soccer world cup in the evening, so we decided to order Pizza for dinner and then split up between people that wanted to code and people that wanted to watch the game. As you might have guessed the latter group was rather small. We ended up in a bar because the public viewing was already full. Though I guess the sprinters got a taste of the German soccer culture. After the game we went back to the office to join forces with the others again to wrap up the day and hack the night away.

Plone Beethoven Sprint 2018 - Sprint Report - Day 1

Posted by kitconcept GmbH on June 28, 2018 06:24 PM

Report of the first day of the Plone Beethoven Sprint 2018 in Bonn, Germany

Stand-up

Stand-up of the first day

We started the day with a stand-up meeting giving people a heads up on the current state of affairs of plone.restapi, plone-react, and Guillotina.

I started with plone.restapi, which is considered stable and battle tested in production.

Victor Fernandez de Alba then gave a brief introduction to our first Plone-React based project VHS-Ehrenamtsportal, that we successfully shipped to our client a few weeks ago and run without any issues since then.

Victor introducing VHS Ehrenamtsportal

After this, Rob Gietema gave a short introduction to the current state of Plone-React.

Last but not least, Ramon Navarro Bosch presented “Guillotina”, an async server written in Python with a Cockroach DB / ElasticSearch backend that adopts some of the core concepts of Zope and Plone.

With a group of 15 sprinters, we decided to split up in four different groups for the main sprint topic. Thomas Buchberger led the plone.restapi group, Rob the Plone-React group, Victor the Pastanaga Editor group, and Ramon the Guillotina group.

Pastanaga Editor

Rob Gietema going though the different approaches we discussed during the Tiles planning meeting

Right after the stand-up, we had a longer discussion about the “tiles” endpoint in plone.restapi and the editor implementation in Plone-React. We already reached an agreement of the API design at the Plone-React sprint a few months ago. Though, it turned out that implementing that on top of the existing plone.tiles implementation was harder than we thought and we did not anticipated all the problems that came along with that.

We decided to keep the API design and to write a simple Dexterity behavior that adds a “tiles_layout” field for the layout information and a “tiles” field that holds the actual data of the tiles. Ramon already wrote a JSON-field in the Guillotina code that we decided to re-use for our implementation.

Rob Gietema already wrote a first prototype of the new editor at the Plone-React sprint and he was waiting for the backend code to be implemented. While we were working on the backend implementation, he focused on the prototype.

plone.restapi

Planning board with plone.restapi issues

Lukas Graf added missing translations in plone.restapi responses and simplified the test setup and did some clean up on the code (https://github.com/plone/plone.restapi/pull/545).

Thomas Buchberger worked on fixing the plone.restapi object creation logic to behave more like through-the-web object creation (https://github.com/plone/plone.restapi/pull/549) and separated the object creation from firing the events.

Sune Brøndum Wøller cleaned up and upgraded multiple Plone versions in plone.restapi (https://github.com/plone/plone.restapi/pull/537), worked on portlets and portletmanager serialization and fixed a ReadConflictError in the plone.restapi tests for the documentation that was bugging us for quite some time (https://github.com/plone/plone.restapi/pull/543).

David Glick and me worked on Zope 4 compatibility for plone.rest and plone.restapi. It turned out that one of my fixes on plone.rest was already sufficient and that the test failures in plone.restapi were caused by a plone.testing issue that David found and fixed (https://github.com/plone/plone.testing/pull/49).

Roel Bruggink continued his efforts on turning the Plone site root into a Dexterity object. He worked on making the IPloneSiteRoot interface / content object behave more like content and he attached behaviours to the IPloneSiteRoot to edit them without relying on a default page.

Mikel Larreategi finished his work on the translation of the content-type names on the @types endpoint (https://github.com/plone/plone.restapi/pull/540) and the translation of the actions and workflow state and transitions on the @history endpoint (https://github.com/plone/plone.restapi/pull/541).

Plone-React

Rob gave an introduction to the Plone-React codebase and explained the basic concepts and libraries that we use in Plone-React.

#Plone #React hands-on by @robgietema pic.twitter.com/PSBQm0zfrN
— kitconcept (@kitconcept_gmbh) June 22, 2018

Eric Steele started to work on creating the add-ons control panel in React. Carsten Senger took over the work that Rob started before the sprint to bring the users and groups control panel to Plone-React.

Andrea Cecchi. and Johannes Raggam worked on the React-based widget for references in Plone.

Guillotina

After giving an introduction to Guillotina to the sprinters, Ramon went ahead and made Plone-React work on top of Guillotina. Since it origins, plone.restapi and Guillotina were supposed to share the same API to allow us to switch the backend for our new frontend at some point in the future. Ramon was also heavily involved in the API design of plone.restapi and wrote the first version of plone.rest before he decided to invent Guillotina. Over the time both APIs differed because of differences in the underlying implementation.

Ramon and Rob worked on this and by the end of the day they could present a working version that allows basic content editing and browsing.

Roadmap / Plone 6

We had a hangout with Philip Bauer from Munich who is leading the efforts to migrate Plone to Python 3 and Zope 4.

Hangout with ⁦@StarzelDe⁩ discussing the roadmap to Plone 6. pic.twitter.com/Wl1H4xVXcx
— kitconcept (@kitconcept_gmbh) June 22, 2018

Hangout with Philip Bauer

Philip and I already had a longer discussion about a possible roadmap for Plone 6 and how to bring our efforts on the frontend together with the efforts of the group that works on Python 3 and Zope 4. We discussed the outlined roadmap and the upcoming sprints where we plan the implementation of the roadmap.

Lunch / Dinner / Evening

We went to have lunch in a Vegan cafe (Black Veg) and to a traditional brewery in the old part of the town for dinner (Bierhaus Machold). After dinner and a few drinks, we decided to head back to the office for some late night hacking. Not without stopping by at a local “Kiosk” for some customary buying of beverages for the evening.

Plone-React Sprint Bonn 2018

Posted by kitconcept GmbH on April 23, 2018 04:46 PM

From March 12th to 15th, we hosted a Plone-React sprint at our office in Bonn. The main goal of this three day sprint was to contribute back the work we did for a recent client project. We also planned to upgrade Plone-React to React 16 and improving the error handling.

A small and very focused sprint with the core contributors of Plone-React was the perfect opportunity to do so. Participants were Rob Gietema, Roel Bruggink, Asko Soukka, Victor Fernandez de Alba, Carsten Senger and me.

Sprint Day 1

We started the first day of the sprint by doing a quick stand up and planning meeting. We used the days before the sprint to outline our main goals and objectives, so we could get started right away.

Planning board after the stand up

React 16

On the first day Rob and Victor worked on upgrading plone-react to use React 16 and Webpack 4.

We postponed the upgrade to React router version 4, because some of the dependencies that we need, are still in alpha phase and staying with the old version does not hurt right now.

Card to upgrade to React 16

Error Boundaries

The main reason why we were eager to upgrade to React 16, was a new feature called “error boundaries”, which allows catching errors in React components and handle them gracefully, without failing the entire app.

Victor implemented error boundaries for client and server side components and for the Redux middleware. It is also possible to pass those errors to Sentry for aggregation and further error handling.

Since we are about to launch our first Plone-React-based project in the next weeks, this was something that was especially important for us.

Error message in plone-react

Token Expiration Middleware

Rob added a token expiration middleware to plone-react that improves the handling of JWT auth token in Plone-React. Working with plone-react on a daily basis revealed a few edge cases where the current authentication failed. The new middleware solves those issues.

plone.restapi

Roel and Carsten focused on plone.restapi. Even though this was not planned to be a major topic for the sprint, we had to fix some issues that were causing troubles in the Plone-React frontend.

Carsten worked on fixing an issue that was preventing to allow the API consumer to reset field values to “None”.

Roel started to work on exposing widgets with tagged values via plone.restapi. This is necessary to support autocomplete or reference widgets in Plone-React.

Test Setup

Asko, who joined the sprint remotely, worked on a brand new end-to-end testing setup with Robot Framework. Because this would be way to easy for a super-smart guy like Asko, he decided to wrap the components into a Docker containers, to make it easier for non-Python devs to set it up. Making Plone PIP-installable is also something that we want to have for quite a while. Asko decided to also mix that in. To make this fun, he decided to add support for Jupyter notebook, which makes it super easy to write Robot Framework tests.

Hangout with Asko

I started the day discussing the acceptance testing setup with Asko, quietly listening to Rob and Victor and discussing the plone.restapi issues with Roel and Carsten.

Pastanaga Toolbar

During the first day, we started a discussion how plone.restapi could support a toolbar that automatically adapts to the permissions of the logged in users and shows only the actions that this particular user has permission for.

Before the sprint Victor worked on implementing the new super fancy adaptable Pastanaga toolbar, and we were eager to build a proper backend implementation for this.

🥕Pastanaga Toolbar Demo (WIP, heating engines in preparation for Plone-React Bonn Sprint 2018) So far, basic behavior, support for lazy load menu components, push-pull navigation. #Plone #Pastanaga #🥕 #React #CSS #UI #LetsMakeItReal pic.twitter.com/t88twlwc9j
— Victor Fernandez de Alba (@sneridagh) March 4, 2018

Victor's tweet with a short demo of the new Pastanaga toolbar

Sprint Day 2

Create-React-App

On the second day of the sprint, Rob and Victor started to look into building a “create-react-app”-like functionality for Plone-React. create-react-app is a widely popular code skeleton generator by the React team at Facebook. It hides lots of complexity from the user (e.g. Webpack, libraries, configuration) and makes it easy to get started with React. This is super important for the adoption of Plone-React because it also allows to use Plone-React as a library and basis for custom client projects.

Rob and Victor created a proof-of-concept that kind of works but it became clear that this requires a lot more effort before this becomes ready-to-use.

Pastanaga Editor

We scheduled a time slot for a discussion and planning session about the new “Pastanaga editor” user experience. We already implemented basic text editing based on the DraftJS editor from Facebook. The current editor allows basic text editing as well as inline (italic, bold, etc.) and block styles (e.g. headlines, bullet points) and external links.

The next step on our agenda to make this editor based on Tiles to make it extendable and allow the user to add images, videos and other media objects.

We agreed on moving forward with an agile approach of building something useful step-by-step, making sure to build a fully functional and useful editor at any point of the development stage, rather than building the full Mosaic-like functionality at once.

Right after this was settled, we started to draft a tiles endpoint that would build the basis for the next iteration of the Pastanaga editor.

Toolbar

Rob started to work on implementing the context-aware toolbar. This was a challenge because the toolbar needs to adapt to the content that is shown in the main column. This means a component deep down the component hierarchy (content) needs to be able to update a component that lives outside of the DOM hierarchy of the parent component. Luckily for us, “Portals” in React support exactly that use case.

Victor worked on the flexbox styling of the toolbar as well as on optimizations of the Webpack configuration.

plone.restapi, Tiles, and Testing

Roel and Carsten continued to work on plone.restapi issues. Roel continued to work on the tagged values representations and Carsten finished the actions endpoint for the context-aware toolbar.

I followed our established documentation-first approach on plone.restapi and wrote the docs for the new tiles endpoint.

With the basic test setup already in place, Asko struggled with the Travis CI setup running different versions of Node and Python at the same time. Problem is that the ZEO version requires the latest Python 2.7.13 which is not shipped by default with Travis CI.

He also worked on a Docker compose option to support an option to run the API server. This would allow faster development of Robot Framework tests since the API server does not need to restart for each test (iteration).

Sprint Day Three

We started the last day of our sprint discussing the create-react-app use case we worked on the day before. Afterwards, we did a hangout with a student who is interested in working on this during the Google Summer of Code 2018.

Hangout with a possible Google Summer of Code student

Right after this call, Asko gave me a tour of the new, Docker compose based, PIP-installable, Jupyter notebook enhanced test setup.

My head was still spinning from Asko’s amazing work, when Rob asked me for a minute to show me a prototype of the new Medium-like Pastanaga editor.

My head is still spinning from the #robotframework based acceptance testing setup walkthrough by @datakurre earlier today. Then @robgietema comes along and shows me a working prototype of the new #Plone Pastanaga editor for plone-react. Mind-blowing! pic.twitter.com/2VLLvIkgQf
— Timo Stollenwerk (@timostollenwerk) March 14, 2018

Summary

The sprint was extremely productive and fun. Having a small and dedicated group of developers with a clear goal and focus really worked well for us.

Group photo in front of the kitconcept office

We contributed back all our re-usable code from our client project, upgraded Plone-React to React 16. We fixed some important issues in plone.restapi and build a super fancy test setup that will allow us to further improve the software quality of Plone-React.

We also laid the groundwork for the next important steps forward: Building the new Pastanaga editor with a tiles-based backend and allowing to use Plone-React as an extensible library with a “create-react-app”-like functionality.

We are already looking forward to the upcoming Beethoven Sprint and the Costa Brava sprint where we will continue to push Plone-React, Pastanaga and plone.restapi.

Pastanaga icon system

Posted by kitconcept GmbH on January 25, 2018 07:24 PM

The way we deal with icons in the web has evolved over the years. Images, images sprites, fonts, SVG, SVG sprites… I’ve been looking lately for the current best practice in order to include it in Pastanaga and I wanted to share with you my results. Please note that it’s not closed and I’m open to suggestions. PRs are welcome too!

Abandon font-based systems

It was clear to me that font-based icon systems are a no longer an option today. For several reasons:

The font is loaded every single time on the page, regardless if we use all the icons or none of them. This bloats the application size and forces an additional request (in the best case scenario).
An existing font is difficult to create, maintain and update. You can use some online (free) services to do that and even you can forge your custom icon font with them but it’s cumbersome and not practical.
Forces you to maintain a parallel CSS that maps the icon name with its actual character in the font (which is obscure). The font creation tool helps you with that, but…
Extending them with new icons is also complex, especially for newbies, and you need access to the source and reload the source in the same tool that was created.

only to name a few.

Time to move on: inlining SVG

The rise of SVG in modern web developing is for a good reason:

SVG is a vector format, so it looks great in HiDPI displays
It’s lightweight and portable, since it’s not a binary file
It can be styled (and animated) easily using CSS (provided they are inlined, not used with the <img /> tag
You can control it via JS

My initial feeling was that using a SVG sprite based system would be the best approach, but I soon was discouraged, after reading to Chris Coyier, CSSTricks: A Pretty Good SVG Icon System

All in to inlining SVGs, then.

So, we need an SVG icon system. Luckily for us, Pastanaga already has a complete set of icons based on SVG organized in one file per icon.

Goals

Our main goal is to provide inline SVGs in our applications, having in mind:

It should be performant and small in size
Only the used icons should be loaded in the given view, compatible with lazy loading
Has to be a no-brainer and clutter-less from the developer point of view
You should be able to extend (or override) the default icon set with your own icons easily
Valid for all modern frameworks, with focus on Angular and React

Harnessing the power of Webpack and modern JS

As developers we want to use the tooling that we have at our hands in the best possible way. So our icon system should use simple ES6/7/* and TypeScript conventions.

import myIcon from './icons/my-nice-icon.svg';
import Icon from './components/Icon';

and the from JSX:

<Icon name={myIcon} />

or angular template:

<Icon [name]="myIcon"></icon>

<div icon [name]="myIcon"></div>

Deconstructing the SVG and put it back together again

According to all the use cases shown in this interesting article by Amelia Bellamy-Royds in CSSTricks: How to Scale SVG the most sensible approach when inlining SVGs is to simply just set the viewBox on your <svg> tag, and set one of height or width to auto. The browser will adjust it so that the overall aspect ratio matches the viewBox. As Amelia points out, that would work for all modern browsers back until 2014. If we have to support older ones, we will need to apply for those the famous padding-bottom hack. Let’s keep things simple for now.

Let’s assume that our SVG is not perfect, and we want to have the all the flexibility that a modern browser can achieve handling SVGs. We will take the existing SVG, deconstruct it and get all the SVG attributes, then the content. We will then put it all together in our components, exactly the way we want it.

The Webpack part

We can accomplish all our goals by using a Webpack loaders combo for loading SVG:

{
    test: /\.svg$/,
    include: path.join(paths.appSrc, 'icons'),
    use: [
        {
        loader: 'svg-loader',
        },
        {
        loader: 'svgo-loader',
        options: {
            plugins: [
            { removeTitle: true },
            { convertPathData: false },
            { removeUselessStrokeAndFill: true },
            { removeViewBox: false },
            ],
        },
        },
    ],
},

We will use svg-loader a super simple inline svg loader that provides you extra flexibility when handling your SVG. Initially I tried the popular Webpack Team’s svg-inline-loader but it was not that flexible at the end. svg-loader returns an object with the contents and the attributes of the svg separatedly that we can later manipulate in our components. We are also filtering the SVG using the well known SVGO utility svgo-loader, we can extend or add more filtering options to optimize our SVGs thanks to it.

We are also restricting this loader to the icons folder, just in case we are handling the other SVGs in our app differently, but of course, you can use it for all SVGs removing the include key.

React

Make it work in React is very straight forward. We need to add the loader to our Webpack config, then add an icons folder and the Icon component.

import React from 'react';
import PropTypes from 'prop-types';

const defaultSize = '100%';

const Icon = ({ name, size, color }) => (
  <svg
    xmlns={name.attributes.xmlns}
    viewBox={name.attributes.viewBox}
    style={{ height: size, width: 'auto', fill: color }}
    dangerouslySetInnerHTML={{ __html: name.content }}
  />
);

Icon.propTypes = {
  name: PropTypes.shape({
    xmlns: PropTypes.string,
    viewBox: PropTypes.string,
    content: PropTypes.string,
  }).isRequired,
  size: PropTypes.string,
  color: PropTypes.string,
};

Icon.defaultProps = {
  size: defaultSize,
  color: null,
};

export default Icon;

That’s it. Our React component takes as props: the name of the imported module of the SVG, the size, and the color. If not given, the SVG will inherit the fill color set in the parent element (or itself). Also, if not specified, the SVG will scale to the parent container height.

Take a look into the JSX of the example

<div style={{ height: '100px' }}>
    <Icon name={Add} />
</div>
<Icon name={Add} size="45px" />
<Icon name={Add} size="45px" color="red" />
<Icon name={Plone} size="60px" color="#1782BE" />
<Icon name={Guillotina} size="60px" color="#EC5528" />

Angular

For the angular icon component we needed the same recipe for the Webpack config and this icon component.

import {
    Component,
    Input,
    HostBinding,
    ViewEncapsulation,
    ChangeDetectionStrategy } from '@angular/core';

import { DomSanitizer, SafeHtml } from '@angular/platform-browser';
import { OnInit } from '@angular/core';

const defaultSize = '100%';

@Component({
  // tslint:disable-next-line:component-selector
  selector: '[icon], icon',
  template: `
  <svg
    [attr.viewBox]="name.attributes.viewBox"
    [attr.xmlns]="name.attributes.xmlns"
    [innerHTML]="svgContent"
    [style.height]="height"
    [style.width]="'auto'"
    [style.fill]="color"
    >
  </svg>
  `,
  encapsulation: ViewEncapsulation.None,
  changeDetection: ChangeDetectionStrategy.OnPush
})
export class IconComponent implements OnInit {

  constructor(private sanitizer: DomSanitizer) {}

  svgContent: SafeHtml;
  defaultSize = defaultSize;
  height: string;

  @Input() color: string;
  @Input() size: string;
  @Input() name;

  ngOnInit() {
    this.svgContent = this.sanitizer.bypassSecurityTrustHtml(this.name.content);
    this.height = this.size ? this.size : defaultSize;
  }

}

We also use the same approach using the component, the Angular template way:

<div icon [name]="Add"></div>
<div icon [name]="Add" color="green"></div>
<icon [name]="Add" color="red" size="45px"></icon>
<icon [name]="Plone" color="#1782BE" size="60px"></icon>
<icon [name]="Guillotina" color="#EC5528" size="60px"></icon>

Our Angular component takes the same three properties as the React one.

In addition, Typescript forces us to overcome some tiny things.

Typings

In order to be able to import the SVG as a module, we need to add this typing to our app:

declare module "*.svg" {
  const content: any;
  export default content;
}

Add the imported SVG object as a Class member

The Angular template won’t be able to use it if the imported SVG object is not a Class member, like:

import { Component } from '@angular/core';
import Add from '../icons/add.svg';
import Plone from '../icons/plone.svg';
import Guillotina from '../icons/guillotina.svg';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  Add = Add;
  Plone = Plone;
  Guillotina = Guillotina;
}

Conclusion

While there are other approaches out there like the Icon component that @angular/material has, they all feel to me like too much and all of them are bloated with lots of options that we don’t really need. I’d like to use a more lightweight and approachable solution like the exposed here that only does what we really need. At the end, it’s not rocket science.

If you have any suggestion, please contact me or open an issue on Github. PRs are welcome!

plone.restapi 1.0.0 released - A Story of Successful Open Source Collaboration

Posted by kitconcept GmbH on January 19, 2018 07:24 PM

After more than three years of development and 25 alpha and one beta release, we are very happy and proud to announce the release of plone.restapi 1.0.0.

plone.restapi is a RESTful hypermedia API for the Plone Open Source Content Management System. It exposes the unique and powerful features of Plone, including the core content management features as well as dynamic content type creation, workflows, permissions, versioning and more.

plone.restapi builds a bridge between a stable and mature Open Source CMS that has been around for more than 15 years and modern state-of-the-art JavaScript-based solutions like React, Angular, Vue and others.

A Little Bit of History

PLOG 2014

The development of plone.restapi started in beautiful Sorrento, Italy at the Plone Open Garden in 2014 after I gave a talk about building an AngularJS application on top of Plone.

A long discussion with Simone Deponti under the Italian sun, about REST API design principles and hypermedia (of course), led to the first commit and the development of a first proof-of-concept implementation.

plone.rest and PLOG 2015

One year later we gathered in Sorrento again. Laurence Rowe, Ramon Navarro Bosch and I spent our days and nights discussing the details of the REST API design and drafted multiple endpoints.

One of the main obstacles to building a RESTful API on top of Plone was the missing ZPublisher support for HTTP verbs such as PATCH, PUT or DELETE. In 2015, I sat together with Ramon Navarro Bosch in Sorrento (again) and we (he really did all the heavy lifting) started to build plone.rest, a small package that adds support for HTTP verbs to Plone.

Plog 2015 from Abstract on Vimeo.

Archetypes and Serializers

We never planned to support Archetypes in plone.restapi. Though, when Thomas Buchberger and Lukas Graf came along and offered to build it, we did not object (of course not, this is Open Source). Their company 4teamwork planned to build a REST api on top of Plone for their OneGov GEVER platform.

Instead of building something on their own, they decided to join forces and share their work and code with the community. Along the way, they heavily refactored the code, added tons of adapters for loose coupling and the ability to customize the JSON serialization.

After this, we were confident to do a first alpha release of plone.restapi on June 14th 2016.

Beethoven Sprint

In March 2017, fourteen Plone developers from eight different countries gathered in Bonn, at the kitconcept office, for the Beethoven Sprint to work on plone.restapi and related topics. In addition to sorting out the last remaining design decision, many exciting new projects were started and announced.

Angular

At the Beethoven sprint, Eric Brehault started to work on an Angular SDK for plone.restapi. A release followed soon and Eric gave a very successful and crowded training at the Plone Conference 2017 in Barcelona.

Today, Angular SDK is a mature package for Angular 2 that makes it really easy for front-end developers to interact with Plone and a fantastic starting point for newbies.

Eric and I mentored Noel Varghese during last year’s Google Summer of Code to build a Progressive Web App for Plone in Angular 2. Noel gave a nice presentation of his successful project at the Plone Conference in Barcelona.

React

Rob Gietema and Roel Bruggink started to build a React-based front-end on top of plone.restapi at the Beethoven sprint in Bonn. Later that year, they went to Toulouse in September 2017 to implement the Pastanaga CSS together with the Plone Angular team.

In November they visited Bonn again for the Pastanaga Sprint where we started to implement the new Pastanaga UI for plone-react.

At kitconcept, we started to use plone-react with Pastanaga for an ongoing project. We can’t wait to release our work and contribute it back to the community.

Vue.JS

Inspired by the Angular SDK and plone-react, Kevin Bieri started to build a VueJS plone-vuejs implementation on top of Plone at the Plone Conference 2017 in Bareclona.

Guillotina

Ramon Navarro Bosch and Nathan van Gheem revelead the name of “Guillotina”, a blazing fast async Python framework that shares the public API with plone.restapi at the Beethoven sprint in Bonn.

Successful Open Source Collaboration

plone.restapi started with an idea and discussions. People and companies jumped in and contributed in many ways that haven’t been dreaming about at first.

Simone, Laurence, Ramon and other helped to shape the initial idea. Lukas, Thomas, Roel, Carsten, Victor, Mikel and many others contributed new endpoints, bugfixes, etc.

Eric, Rob, Noel, Kevin, and others started to build frameworks and solutions on top of plone.restapi.

Many companies such as 4teamwork, Code Syntax, Markina Corpus, VNC invested and contributed to plone.restapi.

The Plone Foundation always supported our efforts by funding sprints.

plone.restapi is a true community effort and the joy that we feel when collaborating with wonderful people pays us back for the countless hours we spend on hacking on code.

Future Plans

A Plone Improvement Proportal (PLIP) to ship Plone 5.2 with plone.restapi has been accepted by the Plone Framework Team:

https://github.com/plone/Products.CMFPlone/issues/2177

With plone.restapi considered stable and close to being feature complete, we will continue working on what could become the next Plone…stay tuned.

Continuous Performance Analysis with Lighthouse and Jenkins

Posted by kitconcept GmbH on December 22, 2017 06:11 AM

Lighthouse is an open-source, automated tool for improving the quality of web pages by Google. It measures the performance of a website and provides metrics for accessibility, best practices for modern web apps, search engine optimization, and assess web applications for adherence to Progressive Web App standards.

Lighthouse Logo

Together with WebPageTest and Google Page Speed Insights it is an indispensable tool to optimize your website performance.

Installation

Lighthouse can be installed in any JavaScript-based project by just running ‘npm install’:

$ npm install lighthouse -g

If you don’t have a package.json in your project, just install npm and run ‘npm init’ before installing.

Running Lighthouse

You can check the performance of any website by calling the ‘lighthouse’ command with the URL of the website you want to test. Append the --view parameter to show the HTML report, right after the command has finished:

$ lighthouse https://kitconcept.com --view

The report will give you five different ratings about PWA, performance, accessibility, performance best practices, and SEO.

Lighthouse Results

Continuous Performance Measurements

If you run your performance test every now and then, you always risk to hurt your website performance without noticing. If a performance regression happens unnoticed, it is usually very hard and time consuming to figure out which change caused the performance regression.

You can easily fix this and save lots of time when you run your performance tests and analysis continuously.

Unfortunately Lighthouse does not allow you to set performance test specifications that your CI system can test against, like WebPageTest or Google Page Speed Insights do (we will cover those tools in later blog posts). Though, it is still very convenient to run the performance test on a regular basis for each commit and include them into your CI report.

Install Lighthouse locally for CI

When it comes to a Continuous Integration, a local installation is prefered over a global one, which is usually harder to manage and to maintain. Especially if you have multiple projects with different sets of package versions on your CI.

Therefore we install Lighthouse locally in our project directory:

$ npm install lighthouse --save-dev

This command will install Lighthouse to your local package.json file. We recommend to use yarn or npm package-lock.json to lock down the package version you are using for a repeatable and stable project build.

For convenience, we add a “lighthouse” script to our package.json:

"scripts": {
  "lighthouse:ci": "node_modules/lighthouse/lighthouse-cli/index.js \
  --output-path=./lighthouse-report.html --quiet \
  --chrome-flags='--headless' https://kitconcept.com"
}

We call the locally installed lighthouse binary and set a static output path (by default, Lighthouse creates a file with the current date/time in the filename which makes it harder to publish on your CI).

We also include the --quiet option and run it on headless chrome, so we don’t need to install and run an X server on our CI system.

At the end, we hard-code our project URL into the command so we do not have to type it manually each time we run this command.

Now we can just run:

$ npm run lighthouse:ci

and it will create a nice HTML report that we can publish in our CI.

Configure Lighthouse for your local development environment

For convenience, we also add a command that you can run locally:

"scripts": {
  "lighthouse": "node_modules/lighthouse/lighthouse-cli/index.js \
  --output-path=./lighthouse-report.html --quiet \
  --chrome-flags='--headless' https://kitconcept.com/blog"
}

The --view parameter will fire up a browser with the report at the end of the performance analysis. This is something we clearly don’t want on our CI system.

Publish Lighthouse Reports in Jenkins CI

Travis and other lightweight CI system usually lack the option to publish any reports except the command line output. Though, if you are using Jenkins CI, you can use the HTML publisher plugin to publish your Lighthouse report.

sh 'npm install'
sh 'npm run lighthouse'
publishHTML (target: [
  allowMissing: false,
  alwaysLinkToLastBuild: false,
  keepAll: true,
  reportDir: '.',
  reportFiles: 'lighthouse-report.html',
  reportName: "Lighthouse"
])

After adding publishHTML to your Jenkins pipeline, you will see a “Lighthouse” link under the ‘Artifacts’ tab:

Link to Lighthouse report in Jenkins

There is a caveat though. Jenkins 1.641 / 1652.3 introduce the Content-Security-Policy header to static files served by Jenkins. The default header is set to a very restrictive set of permissions to protect Jenkins users from malicious HTML/JS files in workspaces.

To allow Jenkins to display the Lighthouse reports, we have to add the following JAVA_ARGS to the Jenkins startup (for instance by adding the following line to your /etc/default/jenkins file):

JAVA_ARGS="-Dhudson.model.DirectoryBrowserSupport.CSP=\"sandbox
allow-scripts; default-src 'unsafe-inline'; img-src * data:\""

For more details see the Content Security Policy Reference and the Jenkins docs on configuring Content Security Policy.

After you fixed the Content Security Policy of your Jenkins you will see the full report when clicking on the ‘Lighthouse’ link on the ‘Artifacts’ tab on your Jenkins build:

Lighthouse Report in Jenkins

Jenkins Declarative Pipeline Stage for Performance Tests

A full declarative pipeline stage for lighthouse looks like this:

stage('Performance Tests') {
  agent {
    label 'master'
  }
  when {
    branch 'master'
  }
  steps {
    deleteDir()
    checkout scm
    sh 'npm install'
    sh 'npm run lighthouse'
  }
  post {
    always {
      publishHTML (target: [
        allowMissing: false,
        alwaysLinkToLastBuild: false,
        keepAll: true,
        reportDir: '.',
        reportFiles: 'lighthouse-report.html',
        reportName: "Lighthouse"
      ])
    }
  }
}

We run the performance test stage on ‘master’ agents and only on the master branch. The steps performed are a simple “npm install” to set up the project build and then we run ‘npm run lighthouse’ to produce the HTML report. If you already have an npm build from a previous step you can of course just unstash the build artifact.

Jenkins pipeline with Lighthouse performance tests stage

Summary

Lighthouse is a valuable and indispensable tool if you want to deliver a fast and user friendly website. Running the analysis on a continuous basis on your CI is a good idea if you take performance seriously. Setting it up is fast and easy. Maybe in the future Lighthouse will also provide a testspec feature that will allow us to fail a CI build (or mark it as unstable) on performance regressions. Though, if you run WebPageTest or Google Page Speed Insights additionally, this is not really needed.

Pastanaga Sprint Bonn 2017

Posted by kitconcept GmbH on November 23, 2017 05:02 PM

Pastanaga is a new user experience framework for the web, designed by Albert Casado.

Pastanaga was first presented in March 2017, at the Plone Open Garden in Sorrento. In July, we started with an initial implementation during the Midsummer Sprint in Jyväskylä, Finnland.

Pastanaga was also present at the recently held Plone Conference in Barcelona, where Albert gave a presentation on it. In addition, Eric Steele, the Plone release manager, gave us the opportunity to present Pastanaga to the audience during his keynote on the first day of the conference.

With all the positive feedback and energy we took from the Plone Conference, we wanted to push things further and we just couldn’t wait until our “Beethoven Sprint”, which is planned for early 2018. Therefore we decided to organize a small and focused sprint at our office in Bonn to work on the implementation of Pastanaga.

The Pastanaga Minimal Viable Product

As an Open Source community (and software engineers) with many years of experience in designing and building complex Content Management System applications, we sometimes have the tendency to try to solve all problems at once.

Over the years we encountered and solved many complex problems and when we build something new, this can be both a source of wisdom as well as a baggage that you carry around.

This sometimes led to a situation where we were over-engineering solutions, to solve all the problems that we encountered over the years at once. Enhancements sometimes stayed around for years without really becoming production ready and usable in real-world projects.

To avoid this from happening when working on implementing Pastanaga, we decided in Jyväskylä to focus on a Minimal Viable Product.

A Minimum Viable Product (MVP) is a product with just enough features to satisfy early customers, and to provide feedback for future product development. The Pastanaga MVP needs to provide what we consider the essentials of a Content Management System:

A site administrator can and add, edit, and delete a page
A user can view the created pages and navigate the site structure

In order to be usable for public facing website projects, we added two additional technical requirements:

The page should be fully rendered within 500 milliseconds
Google should be able to crawl the contents of the website

Those requirements might sound very simple, but they are actually not.

Pastanaga aims to leverage the editing experience and reduce the complexity that we took for granted over the years. We aim to simplify the user experience for the editors by getting rid of things that we got used to. For instance, adding an image to a page should be as simple as just dragging and dropping an image to the page and Plone will take care about the heavy lifting of automatically uploading and resizing the image.

You can find a list of all the user stories that we plan to implement as part of the MVP here:

https://github.com/plone/pastanaga#minimal-viable-product

Having the goals and scope for this set the only thing that was needed was a bunch of Plone devs and three days and nights of coding.

Sprint Day One

After the sprinters arrived, we started with our sprint planning session. We decided to focus on the implementation of the Pastanaga MVP and work on the other issues (e.g. plone.restapi) only if we need them for the MVP.

.@robgietema giving us an introduction to plone-react. #plone #sprint pic.twitter.com/Ga6Bsd3l2J
— kitconcept (@kitconcept_gmbh) November 15, 2017

After the planning meeting, Rob gave us an introduction to plone-react, a ReactJS-based implementation of the Plone UI that he and Roel worked on over the past months and that we decided to use as a basis for our MVP.

Pastanaga sprint planning meeting at our office in Bonn. #plone #Sprint pic.twitter.com/FDpjv5wx5t
— kitconcept (@kitconcept_gmbh) November 15, 2017

We went through all components, reducers, bells and whistles of the application and discussed best practices, developer environments and developer approachability.

After that session, Rob and Victor started with the implementation of Pastanaga. Davi created a pull request that adds an uninstall profile for plone.restapi and started to learn about React. Roel started to look into a way to turn the Plone site root into a Dexterity object, something that we would need to simplify the Plone editing experience. I worked on the basic Robot Framework acceptance test setup and updated the contents of the Pastanaga github repository, which is supposed to be just an entry point for all our initiatives around Pastanaga:

https://github.com/plone/pastanaga

Day Two

On the second day, Victor finished the login form and made the error messages work.

Rob implemented the document edit accordion menu, fixed the button styling, made plone-react use the Pastanaga icons and started to work on the toolbar.

Davi added a search widget to the header, implemented the breadcrumbs navigation and added styles for the document heading and description.

Right before the wrap-up meeting of day two, Roel showed us a Plone site with a “containerish” Dexterity-based site root. We did not really expect that much progress and went to bed (some of us a lot later) still very impressed by his accomplishment.

Day Three

On day three, Rob started to work on the new Pastanaga document edit view. He made the new edit view to show multiple content items (e.g text, image, video) and allowed to change the order of those content items via drag and drop.

Davi continued to work on the header and breadcrumbs styling. Victor looked into the mobile views of our responsive design, fixed some issues with the status messages and briefly started to look into GatsbyJS (which we plan to use to implement pastanaga.io).

Summary

After three days (and nights) of hacking, we had:

A fully functional login form with error messages and password forgotten functionality:

A fully functional Pastanaga Toolbar that can be collapsed or expanded. With all the menu items present and the personal toolbar functionality available:

A view to add and edit pages with all the existing functionality:

In three sprint days, we accomplished our main goals and were able to create the first iteration of a Minimal Viable Product that we can use to build things upon. We plan to continue to work on this, use it in our current and upcoming projects, and of course: contribute back as much as we can.

Stay tuned for more updates on this soon!

Plone Conf 2017 Day 2: Timo Stollenwerk: Building Bridges - The Headless Future of Plone

Posted by David "Pigeonflight" Bain on October 19, 2017 08:48 AM

I decided to try a Maurits van Rees and live blog a conference talk.
Talk by Timo Stollenwerk on Building Bridges - The Headless Future of Plone

Plone's headless future

Working on what we call headless these days started in 2014
You already heard a part of this from the Keynote (about Pastanaga UI etc..) on the first day so I won't repeat that.
My ultimate goal is to bring the vision to reality.

A few observations

Mobile is overtaking Desktop (Plone is mobile ready but Pastanaga aims to have the best experience on every device)
Open Source is Mainstream (Plone is different, today large open source projects are coming from large players like Facebook and Google, this helps to make open source more mainstream). Github looked at contributions last year and visual studio code was the project that had the most contributions... Microsoft!!)
Javascript is taking over (Javascript is becoming more important, if you are a web developer in 2017 you have to learn modern Javascript)
The Web is everywhere (I visited my Uncle who is a Doctor, 5 years ago and noted that he was using a web app on his desktop for viewing scans of the body)

In recent studies they discovered that swift is losing popularity because web technologies are taking over. The web is coming back with technologies like Electron (Desktop) and Cordova (mobile)

Isn't it a great time to be a Web, Javascript, Open Source developer in 2017?

We're hearing that the CMS market is dead

If we see it in other sectors we say it is more efficient but when it happens to us we don't want to transform ourselves for the better. I think we are living in exciting times...

If JS is so great why don't we just go with it and build a CMS with Javascript?

Why do we keep using Python and Plone?

I love Python (wasn't my first language, but the first one I loved, I still miss Python with every line of Javascript I write). I can live with Javascript for the tooling and the community but would prefer to keep Python. I can't imagine using Node on the backend because I think Python is doing a way better job on the backend.
Plone the community. In the last year I've been to Jenkins, CI and testing conferences but there's no place like Plone.

I went to a JS conference alone. Usually when you go to a conference alone you need to make an effort to talk to persons. Then I went to the sprint but out of 1000 persons there were only 20 or 30 persons at the Sprint. When I speak to Python conference attendees they ask me, how do you get people to come and even pay for a flight to Plone conferences, it's like magic!

Plone the Software is still unique (permissions, traversal, workflows)
Plone the CMS (as Eric says, Plone is doing Breadcrumbs since 2001) Go out and try all the Javascript CMSes, they all have awful user interfaces, they have nice libraries and everything you can imagine but lack the basic functionality of a CMS. I couldn't just jump and move to another system because I'd only have half or 10% of the current functionailty I have now. We don't want to become Grandpas and isolate the new JS communities who have lots of energy

What do we have now?

Stabilising JS frameworks, it's not too hard to switch between VueJS, React, Angular. Which one you use depends on if you like a library vs a framework. How do we handle this? We want to give our clients something that can be supported over the next 5 to 10 years. That's a lot of time. If you look at Plone we are able to provide that. How do you handle that? The answer is plone.restapi (restful hypermedia for Plone).

Our idea with plone.restapi is to use it as a bridge. Stability on the backend with flexibility on the frontend. In two or three years the JS ecosystem will change further.

Status of plone.restapi

It is stable and used for 3 years, it is used in production by several companies. We consider plonerest.api to be feature complete.
I asked Eric what is plonerest.api missing, he said "nothing".

I'm just lazy about releasing a 1.0 release.

We can get back to our vision with plone.restapi being stable.

So we can get back to our vision of bridging...? One of our ideas about building with Plone 5

You can use React in the core today, if you want to go with a full framework like Angular you can too. We have 3 branches and the plonerest.api allows us to build bridges between
standard plone and the other branches.

How do we make this happen?

Regarding stories... Victor sometime says to me "Should we really say that, going on stage and telling people about these things? They will expect it to happen."

So who here would like to have Pastanaga UI today and use it. With projects in the Plone community we've building like that...

when we should be building like this..

I believe that if we want to have that we need to start with the Minimal viable product. Something, not just for users but for companies that can give them value right away. I want that skateboard.

What do we need to get that skateboard?

Login
Content Editing
Image upload

That might sound easy but it is not. We're aiming for making Plone stand out, we want people to be like "wow, that's the greatest editing experience I've ever had". I want us to iterate over that and focus on that user story.
I want to make image upload really easy. One thing we need to solve on a technical level, we currently have created a Medium-like editor but we don't have image scaling (something we take for granted in Plone). I want us to have the ability to add the image and have scaling done "magically".

There are things that are essential for an MVP

Performance
Image uploading
SEO

If we don't have this, users will abandon pages. Modern page builders focus on this (e.g. gatsby js is a modern page builder built in ReactJS). If you want to compete we have to provide users with a great out of the box performance. We'll need to use all the tooling, webpack etc...
We will also need server-side rendering. We saw that, without server-side rendering at kitconcept we can't do good SEO.

Visit https://github.com/plone/pastanaga#minimal-viable-product for details on our projections of a MVP.

We have an open space at the conference and there will be sprints.
Real world projects (if you have any projects and want to use plone.restapi, angular sdk, plone.react, please talk to us)
At kticoncept we have a few projects where will do that
Sponsorship (we may be able to do something on that front)

Summary

We have a stable platform in plone.restapi for building bridges
I think Plone's future is bright is we combine our knowledge and experience with the new things
Pastanaga UI is really greate
I hope we can provide you with a roadmap
The great thing about the Plone community is tha tyou start with an idea and...
let's get together and do Plone magic together!

Help two Plonistas get from Jamaica to Barcelona (Catalonia) for the 2017 Plone Conference

Posted by David "Pigeonflight" Bain on September 07, 2017 01:23 AM

Jamaica to Catalonia for Plone Conference 2017

TL;DR - David Bain and Oshane Bailey are looking to attend the 2017 Plone Conference via crowd funding.

Sept 20, 2017 Update: We will mostly be walking so this reduces our transportation costs, we've adjusted our target to reflect this. We've extended the campaign until September 27.
Sept 17, 2017 Update: We're finding some cheaper fares, adjusting our target to reflect this
Sept 16, 2017 Update: Oshane has been offered a free room, this will lower the overall target further
Update: It looks like there are more cost effective accommodation options, as a result, we've further adjusted our estimates.
Update: We have found some cheaper flights via Google Flights so we're adjusting our estimates down by $2,000.

(David's the one on the right).

This is a manually managed crowdfunding tracker updated by David (no AI was harmed in the creation of this tracker)

We are trying to get from Jamaica to Catalonia for the 2017 Plone conference. Our target is to raise a significant part of the roughly ~~USD$7,000~~ ~~USD$5000~~ ~~USD$4,400~~ ~~USD$3,700~~ USD$2,900 needed to cover airfare, accommodation etc.

How to support us

You can contribute to our travels via Paypal (see the button below), funds go to my Paypal account.

Why support us?

Support us so that we can deliver training, talks and participate in the sprints*.

Getting us there will allow David to deliver training and a talk or two, Oshane will be able to share his Google Summer of Code experiences and participate in his first face to face community sprint after the conference. Oshane worked this summer on improving the theme editor experience, here are some links with more information about what he did...
https://community.plone.org/t/gsoc-2017-improving-the-theme-editor-experience/3906
https://community.plone.org/t/thank-you-for-the-support-during-gsoc-2017/4792
We're hoping he'll be able to present his experience as a talk at the conference.

David has been an active part of the community for many years. He delivered training and two talks at the last conference and has been invited to be part of the training team at the 2017 conference as well.

Both of us are really excited to participate this year, however the cost of airfare is prohibitive.

* While Oshane will stay for the sprints, due to family commitments, David won't be able to stay for the sprints.

Rough Breakdown of expenses

Identifying and fixing broken objects in a Plone website

Posted by PloneExpanse on September 06, 2017 05:30 PM

I’ve removed plone.app.stagingbehavior from a website because the new plone.app.iterate has the same functionality. In addition, the p.a.s package was overriding adapters that I wanted to write. Now, my problem was that I could no longer save any related items, I would get an error: Module ZPublisher.Publish, line 138, in publish Module ZPublisher.mapply, line 77, in mapply Module ZPublisher.Publish, line 48, in call_object Module plone.z3cform.layout, line 66, in __call__ Module plone.

Why you had problems figuring out Plone (the webinar)

Posted by David "Pigeonflight" Bain on August 03, 2017 10:32 PM

Presenting... Why you had problems figuring out Plone (the webinar). Okay, that's not the actual name of the webinar. Instead, we went with the more descriptive but slightly less clever Plone for Newbies - The Big Picture.

The Big Picture is about understanding the model.

If you're a developer about to begin your journey of Plone development, The Big Picture aims to fill out your understanding of how the pieces of Plone fit together. Thinking of it as a purpose built system lays a strong foundation for success.

Only smart persons use Plone

I've heard someone suggest that you have to be really smart to use Plone (implying that it is hard to use). I call it the "this helicopter is harder to use than my bicycle" problem. Every time I benefit from Plone's link integrity support, flexible access control model or use cut and paste to move content around I'm glad I'm not using a "bicycle". I like to point out Plone's comprehensive suite of tools which you'll be glad exist when you need them. For developers, once you accept that you're looking at a "helicopter" you need to spend a little time "understanding the model". As you understand the purpose of the major controls you'll find it easier to use the system to solve problems.

Why Plone?

Now is the right time, with an increase of cybersecurity related issues, organizations should be looking to adopt secure platforms. I've been using Plone for more than 15 years, and I can confidently recommend it as a secure platform. In case you took your eye of Plone for a few years, now is a great time to give it a second look, it has kept up with modern development practices and remains an excellent choice for your content management needs.

Why a webinar?

My target audience isn't in one geographic location, the most effective way to reach them is a virtual medium and webinars provide a well-known, tried and tested approach.
Additionally, I have run one or two webinars before, if you count online training courses. Of course, it is different when you are doing more than just showing up, reaching out to a "less captive" audience and convincing them to commit 90 minutes to a webinar. So this is new ground for me. I am learning a lot from this experience and have had a fleeting thought, maybe I'll take all this webinar and Plone stuff and do the "meta" thing, build a webinar management tool on top of Plone. You never know.

Restore missing blobs from blob cache

Posted by PloneExpanse on August 03, 2017 01:40 PM

I had a curious case of missing-but-present blobs in an old Plone service, configured with a Zeo server and 2 Zope instances. The root of the problem (I think) was that the blob folder configuration was broken: the Zope client instances were configured with shared blobs to “off”, but they were really sharing the same caching folder. In the end, the blobs were loaded by the Zope services and everything appeared to be working, but when I’ve tried to move the blobstorage folder to a new machine, I ended up with missing blobs.