Planet Plone - Where Developers And Integrators Write

PLOG 2020 Registration Now Open

Posted by PLONE.ORG on November 27, 2019 04:47 PM

Sign up now!

Registration deadline is February 28, 2020

Training classes on several topics will be held in the mornings, sprinting will happen in the afternoons and discussions will go on all day. The Hotel Mediterraneo is in a beautiful location overlooking the Bay of Naples, with wonderful food and a lovely garden where the sprinting and training will be held.

We have reserved a limited number of rooms and they are available on a first come first served basis. Breakfast and dinner are included. The rooms for three and four persons are the most economical and are expected to sell out early.

We are looking for women to share a triple!

Questions? Email us at plog@plone.org

Event details ~~ Read About PLOG ~~ Watch the Video ~~ Register

Python, the most popular programming language of the year

Posted by CodeSyntax on November 19, 2019 06:56 AM
IEEE Spectrum has published its sixth annual list with the most popular programming languages of the year across multiple platforms, and, once again Python repeats in 2019 as the undisputed leader, as it will happen in 2017 and 2018.

Speedup volto razzle builds

Posted by PloneExpanse on November 17, 2019 12:58 PM
I’ve been looking for a way to speedup Volto razzle/webpack builds, both while developing and for “production” mode, when building the final bundle. Fortunately, this solution exists and it’s extremely easy to integrate. Let’s define the problem, to see how to approach it: what is Volto actually? What do you get when you open, in your browser, a Volto frontend Plone website? To greatly simplify (and I hope I didn’t get anything wrong as I am not a Volto core developer):

Four Members Join the Plone Foundation

Posted by PLONE.ORG on November 13, 2019 09:47 PM

The Plone Foundation welcomes four new members, after unanimous confirmation by the Foundation's Board of Directors on October 10, 2019.

Membership in the Foundation is conferred for significant and enduring contributions to the Plone project and community. The Plone Foundation Membership Committee overwhelmingly recommended each applicant for their ongoing contributions to Plone.

Fulvio Casali

Fulvio Casali

Fulvio started working with Plone in 2008 and since 2012 he has been operating a consultancy business, Soliton Consulting, specializing almost exclusively on Plone. Fulvio organized Emerald Sprints (2013, 2014), Plone Open Garden (2016, 2017, 2019, 2020), participated in every Plone Conference since 2010 (except one!), has given two conference talks, and attended the Plone Symposium East 2010 and the Plone Konferenz in 2012. Fulvio has provided Plone training for Mastering Plone, Diazo theming, Rapido, Angular, and TTW Dexterity.

Stefania Trabucchi

Stefania Trabucchi

Stefania is the co-founder of Abstract Technology in Germany. She was the organizer of Plone Meet Up Berlin 2004-2014, of World Plone Day Berlin 2009-2015, of Plone Social Sprint Berlin 2014, and of Plone's Europython 2014 presence and associated marketing. She has participated in the Plone Konferenz 2012 as a speaker, the Plone Beethoven Sprint 2019, the Plone Conference 2014 sprint, and the Plone Open Garden sprints in 2014 and 2015. Stefania represented Plone as a member of the CMS Garden 2014-2017 and is active in the Plone Intranet Consortium with Quaive.

Thomas Buchberger

Thomas Buchberger

Thomas is the CTO of 4teamwork AG, has been working with Plone since 2006 and became a code contributor in 2011. He has attended many Plone conferences, beginning with Naples in 2007. He attended many sprints, including the Barcelona Strategic Sprint (2016), the Bonn Beethoven Sprint (2017 and 2018), the Sorrento Sprint on Frontend Modernization and Python 3 Porting (2019), and the Beethoven Sprint (2019). Thomas' main code contributions have been in the plone.rest and plone.restapi modules.

Andrea Cecchi

Andrea Cecchi

Andrea was the lead organizer of the Plone Conference 2019 in Ferrara. He participated at earlier Plone conferences and sprints (Sorrento 2019, Tokyo 2018, Barcelona 2017, Bristol 2014). As part of the RedTurtle team, he has organized and participated in World Plone Day events since its inception. Andrea is the maintainer of over 100 packages on PyPI.org.

 

 

The Foundation is the trustee for Plone's intellectual property, works to protect and promote Plone, and has over 80 active members.

Essential Plone Add-ons

Posted by Jazkarta Blog on November 11, 2019 03:45 PM

If it’s fall, it must be time for the Plone Conference. This year the annual gathering took place in Ferrara, a beautiful small city in the north of Italy. The weather was perfect, the streets medieval, the party was in a real castle, and the food – well! The food was amazing. This is the traditional dish cappellacci di zucca al ragù, pasta stuffed with pumpkin in a meat sauce. Yes it tastes as good as it looks.

Cappellacci di zucca al ragù

Following the tradition begun at the Barcelona conference and continued in Tokyo, we held a popularity contest to identify the best add-ons for Plone, Python’s open source CMS. Plone comes with tons of features out-of-the-box – like workflows, search, a multilingual UI, conformance to accessibility standards, and granular user roles and permissions – but it also offers an extensible platform for building new add-ons. Attendees nominated their favorites and the results were posted in the conference venue where people voted their top 5 using sticky dots.

Add-on voting sheets

Thirty-three add-ons were nominated, and the voting revealed a few that are particularly popular – notably for form generation and faceted search. Others included add-ons for document generation (Word, PDF, etc.), image cropping, taxonomies, authentication, and lazy loading. The full results can be found at the 2019 essential Plone add-ons page.

Plone Foundation Board Officers Selected for 2019-2020

Posted by PLONE.ORG on November 08, 2019 09:22 PM

Following the election held in October 2019, the 2019-2020 Plone Foundation Board held its first meeting on November 7, 2019, and its first order of business was to select its officers:

  • President: Chrissy Wainwright
  • Vice President: Paul Roeland
  • Secretary: Andy Leeb
  • Treasurer (non-voting): Jen Myers

The Board also selected the following committee chairs:

  • Marketing: Érico Andrei (assisted by Kim Nguyen)
  • Membership Co-Chairs: Érico Andrei, Kim Nguyen

The following (unofficial) team liaisons were selected:

  • Framework: Chrissy Wainwright
  • Frontend: Victor Fernández de Alba
  • Education: to be determined
  • Security: Jens Klein
  • Guillotina: Andy Leeb

The new Board is determined to reflect the changing landscape of the Plone Foundation, where it is not just about the Plone CMS but a host of technologies and communities that reflect the same spirit, including Volto, Guillotina, Zope and others.


About the Plone Foundation

About the Plone Foundation Board

New Foundation Board for 2019-2020

Posted by PLONE.ORG on November 07, 2019 05:12 PM

The Plone Foundation Board is now in its 2019-2020 term. 

The results of the Foundation Membership vote were announced at the Annual General Meeting held in Ferrara on October 25, 2019.

The 2019-2020 Board members are:

William Fennie was not elected although he received over 30% of votes.

Our outgoing Board members chose not to run this year. We thank them for their many years of service!

  • Alexander Loechel
  • Carol Ganz
  • T. Kim Nguyen

Lightning talks Friday

Posted by Maurits van Rees on October 31, 2019 07:36 AM

Wolfgang Thomas: OIRA

Demo of tool for online risk assessment. Export to Word (with python-docx). Training module. Export to Powerpoint presentation with python-pptx.

Alex, Guido: Quaive

State of Plone Intranet in 2019. Dozens of packages. Social stream, search, auditing, real-time document collaboration. Classifieds app makes it nice to share info, legal app showing contracts. PAS graph optimizations, see talk by Alessandro. SVG rendering. Progressive web app, we did not want to create a special app, but it does really give a mobile experience.

Busy supporting Plone 5.2 and Python 3, we will manage that. We will use crossbar.io for push notifications. Encryption via CWS. Johannes is joining Syslab to work on Quaive fulltime. iMio is helping too.

Michele Finelli: vini delle sabie

The last of my three easy pieces on Ferrara.

vini delle sabie is a wine. Sand is influencing the grapes. The grapes are actually immune to some diseases. Most typical is de fortana grape. Give it a try. Perfect companion to the dishes I talked about.

Fred van Dijk: collective.ldapsetup

Example package with LDAP setup. Uses pas.plugins.ldap, which is a replacement for plone.app.ldap. The core package now supports wildcard search, with help from Asko Soukka. Robert Niederreiter did a lot of work to get it working on Python 3. LDAPS support.

Code: https://github.com/collective/collective.ldapsetup

Gauthier Bastien collective.documentgenerator

Desktop document generation (.odt, .pdf, .doc, ...) based on appy framework and OpenOffice/LibreOffice. Templating in LibreOffice.

Code: https://github.com/collective/collective.documentgenerator

Maik Derstappen: EasyNewsletter

Create news letters in Plone. Send mails to subscribers. Let email server do this, or use an external delivery service. On a collection you can define how items are rendered in the email. You can customize the output and aggregation template. Or fully write your own. Works now on Plone 5.1/5.2 Python 2.7/3.7.

Planned: integrate Mosaico editor.

Code: https://github.com/collective/Products.EasyNewsletter

Paul Grunewald: Plone Tagung

9 to 11 March 2020 in Dresden. Talks and sprints. Come!

Philip Bauer: Training Plone 6

We plan to create quickstart Plone trainings for Python devs, one for frontend devs, one for users. Mastering Plone 5.2 talks about a lot, you should follow it.

But what about mastering Plone 6. It would need to be about backend and frontend. I would love feedback about what would be required. We do not want to overwhelm people.

Maurits van Rees: 3 authentication add-ons

I present three PAS plugins:

  • collective.denyroles
  • pas.plugins.headers
  • collective.contentgroups

See the full presentation.

Treasure hunt solutions

There was a treasure hunt this week, where every day you had to find three items in the city and make pictures. We show the solutions.

Timo Stollenwerk: Cypress

We use this Acceptance testing framework to test Volto. It has gained traction in the JavaScript world.

I use robotframework and love it. But Cypress is written in JavaScript, so an intern could write extra plugin library for it.

See https://www.cypress.io

Timo Stollenwerk: Sprints

The next two days we will have Plone sprints. Everyone is welcome. It is a great chance to share code, get to know the Plone code, ask questions to coders, and generally work together. We will find someone to pair up with you.

See the list of sprint topics. You can work on other stuff, you can add ideas, also when you will not work on it. You can add your name to topics when you are interested, also multiple topics.

Sally Kleinfeldt: Essential Plone 5 add-ons

People could suggest add-ons, and then people voted for them. Clear winners:

  • collective.easyform
  • plone.restapi (in core now actually)
  • eea.facetednavigation

Others:

  • collective.documentgenerator
  • collective.z3cform.datagridfield
  • collective.taxonomy
  • and more

Plone Conference 2020

The Plone conference next year will be in... Namur, Belgium. November 16-22.

Thank you

Thank you Red Turtle for organising such a lovely conference!

Panel: Frameworks comparison

Posted by Maurits van Rees on October 25, 2019 04:04 PM

Plan:

  • Brief introduction to each framework
  • Followed by discussion focused on use cases
  • Maurits will live blog (thanks!)
  • Framework vs. Product - let’s not worry about that!

For each system:

  • What use cases is it well suited to
  • What use cases is it poorly suited to

Discussion:

  • Focus on client use cases and what frameworks would be a good or poor fit
  • You can also ask questions!

Live blogging.

Use cases:

  • Non-profit with custom forms and import/export
  • Concert venue with strict event listing and advanced settings.
  • NGO needs a database application for their highly confidential data.
  • Generate a map out of data, and store extra information, like images for the data.
  • Three-person company with basically static pages, non-technical users.
  • Real-time application with websockets.
  • A large company wants to monitor how well their customer service staff are doing. Strict form with statistics on it.

Pyramid:

  • Best for APIs, not really CMS or websites, but applications
  • Start small, grow
  • Complex non-profit use case: you would have to build everything yourself.
  • Event listing: we have a site that manages stuff for music bands as example, you just need some endpoints. Maybe websauna on top.
  • NGO privacy: security really good, and simple enough to understand the full system. You can fully build your own fine grained security system. UI is a lot of work.
  • 100 person company wants intranet with various content, installable by local IT team.
  • Intranet: try to create a good solution for multiple clients, not tailored to one in a consulting project. Pyramid is good for building such products.
  • Map: pyramid does not care which database you use. In a project we are mapping electical grids. In Guillotina they think they are nice with AsyncIO, but it makes their whole codebase harder to understand.
  • Real-time: we had websockets for a year, but it was too slow, so we rewrote it in Go-Lang.
  • Form statistics: you can create a form, store it in a database, export it, fine.

Django:

  • Aimed at 80% of use cases, simple auth system
  • Good for CRUD apps, smooth beginning
  • Non-profit: export/import is good, we can do forms. Sharing access gets tricky
  • Event listing: Django can do it, but just use Wordpress. But the technical user can do this in the admin interface.
  • NGO privacy: permissions are too simple OOTB, not per object. You may want end-to-end encryption, which no framework offers. Limited built-in audit log, nice start.
  • Intranet: not good fit for Plone for the people who do not often work with it. Django has easier CMSes for this. The expenses claim form could be easier to integrate into Django. Deployment just needs Postgres. DjangoCMS is Plone Light. Wagtail is WordPress Plus.
  • Map: any of the frameworks except Plone can do the geo stuff in postgres. Images just on the file system. Static and uploaded files are handled fine by Django, can also be in the cloud.
  • Real-time: not many Django users will care about websockets, will never be the focus.
  • Form statistics: you can but I would not do it. This feels like a custom web application. Do it in any framework.

Guillotina:

  • Small framework, scale from small to big
  • Non-profit with forms: decent fit, use JSON schema
  • Event listing: go to wix.com or something, it seems too simple
  • NGO privacy: Good fit, permissions are no problem for this. You need to build the full UI. Okay, you may be able to start with Volto as UI, but that needs integration in the backend.
  • Intranet: backend can handle it, need to build UI, deployment can be with docker and kubernetes.
  • Map: for the large files, Guillotina wins because of AsyncIO.
  • Real-time: we use websockets just fine.
  • Form statistics: you can build an app, depending on your UI skills.

Plone:

  • Good when you have different use groups with complex requirements for security.
  • Edit interface is doable, but less so on mobile.
  • Import/export complicated, but you can hire Jens.
  • Difficult to find skilled people.
  • Non-profit with forms: really good fit, end-users can create the forms. The import/export would need help from a provider.
  • Event listing: do not bother with Plone. It sounds more relational, Plone would be overkill.
  • NGO privacy: you need deep understanding of Plone security, hierarchy for storing data. Audit logging with add-on.
  • Intranet: Plone is fine. You may need to help with installing at first, but it will keep running once installed.
  • Small almost static site: none of us.
  • Real-time: once Asko is done with his ZServer improvements, we can do it.
  • Form statistics: do not use Plone. Use sed, awk, perl. Technically you could use some add-ons.

Maurits van Rees: 3 authentication add-ons

Posted by Maurits van Rees on October 25, 2019 03:28 PM

Three PAS plugins:

  • collective.denyroles
  • pas.plugins.headers
  • collective.contentgroups

collective.denyroles

Deny access to roles like Manager and Editor

  • Use case: Manager only logs in to edit-domain, not live site.
  • By default deny access to Manager, Editor, etc.
  • env DENY_ROLES=0 to disable
  • or Apache/nginx header X_DONT_CHECK_ROLES
  • Actually not a plugin, but a patch.

Code: https://github.com/collective/collective.denyroles/

pas.plugins.headers

PAS plugin for authentication based on request headers.

  • Use case: Apache/nginx adds SAML headers to requests.

  • Configuration in ZMI or profiles/default/pas.plugins.headers.json:

    {
        "userid_header": "uid",
        "required_headers": ["uid"],
        "roles_header": "roles",
        "allowed_roles": ["student", "teacher"],
        "deny_unauthorized": true,
        "redirect_url": "https://maurits.vanrees.org",
        "memberdata_to_header": [
            "fullname|HEADER_firstname HEADER_lastname"
        ]
    }
    

Code: https://github.com/collective/pas.plugins.headers/

collective.contentgroups

Plone PAS plugin for content as groups.

  • Use case: create content item that works as a group.
  • dexterity behavior
  • No Products.membrane, no Products.remember, no dexterity.membrane.
  • No separate membrane_catalog.
  • Only groups, not users.
  • No multiple inheritance, just AccessControl.users.BasicUser.

Code: https://github.com/collective/collective.contentgroups

Alessandro Pisa: PAS adventures

Posted by Maurits van Rees on October 25, 2019 01:36 PM

What is PAS? It is the Pluggable Authentication Service from Zope and Plone. It manages everything related to users and groups, like authentication, permissions, searching. It can get or set information from your site or the request or an external service like LDAP.

It uses plugins, so you can register your own. You can see a lot of those in the ZMI at the Zope root, at http://localhost:8080/acl_users/plugins/manage_active.

There are about twenty different plugin types, and they interact with each other.

In Plone we have Products.PlonePAS. It adds Plone-specific plugins and types in PAS, for example the local roles plugins. Local roles are regular parts of Zope, but not pluggable there. So this is at http://localhost:8080/Plone/acl_users/plugins/manage_active

So this is the basis to manage security, users and groups, using plugins that can be activated, deactivated, and ordered, and these plugins interact with each other.

Now let's go to the kitchen and make a plugin. We want our Plone Intranet to use an LDAP plugin. At the time, we started using Products.PloneLDAP, which builds on a few other Zope packages. (Currently pas.plugins.ldap is the more modern choice.)

We also wanted Products.membrane: this allows to create content that works as user or group. It uses a separate membrane_catalog, a sort-of copy of the portal_catalog, for better or worse.

We wanted collective.workspace to manage roles through groups instead of sharing.

And Plone Intranet itself had some extra sauce (plugins) on top. And important: caching, for performance.

It worked fine! But after a while it slowed down. We had many calls to LDAP, in the wrong way, even with the cache in place. We were using the membrane catalog too much, causing the catalog queue to flush, causing a slowdown in the standard portal_catalog search. We were also writing on SOLR, again external, which did not help for performance.

So what was the reason of the bottleneck? The big deal is when PAS does not find what it is looking for. It then tries all relevant plugins, which are intertwined. For example it finds a workspace group, and needlessly goes to LDAP to search for this group there, where it will not be found, etcetera.

So maybe we could store the LDAP information in Plone. We decided to sync LDAP in the Plone objects.

And then we also patched PAS. Finding a group results in trying to get the properties, parent groups, roles. Our patched PAS skipped plugins that looked unrelated.

There was another problem: a custom local role manager. There were users that belonged to lots of workspaces. So also to lots of groups. So role assignment was slow: you need to get all roles of all groups. We replaced Plone's local role manager with a custom one.

Another plugin is the recursive groups plugin. Standard in Plone. This can be really expensive. For each of those thousands of groups you would check if they have a parent group. We replaced this with a utility that could handle this fast, using graphs with networkx. With this, we could replace all our group plugins, especially the recursive groups plugin.

Possible problem: in the graph we had 20.000 nodes, 25.000 edges, 25 relations. But creating the graph took 50 milliseconds, so that was okay. The nodes are strings like principal_type:UID, for example collective.workspaces:Admins. We stored the information in OOBTrees: uid2path, path2name, name2uid, and an OOTreeSet called edges. And then nx.DiGraph(uid2path) to create the graph. This helped speed things up a lot.

To keep the data structure up to date, we use events.

To recap:

  • remove external connections when possible
  • know the limitations of your plugins
  • you can patch PAS to avoid some plugins
  • you can use a custom group plugin

Further ideas:

  • lazily create collective.workspace groups
  • lazily fetch group and user properties and roles

Takeaways:

  • PAS is great, but mixing and abusing plugins can be deadly
  • Sometimes replace plugins with a new one that does exactly what you need.

Rodrigo Ferreira de Souza: Data migration to Plone 5.2 and Volto

Posted by Maurits van Rees on October 25, 2019 12:26 PM

At KitConcept we have some websites that need to be deployed in Plone 5.2. We could have used collective.transmogrifier to migrate from 4.3 to 5+. And from 5.1 to 5.2 migrate the ZODB from Python 2.7 to Python 3. Transmogrifier is also an option for that part.

Why would we use transmogrifier? There are many generic pipelines available for common cases. Flexibility to deal with different use cases. And it is actually a brilliant wayto use the iterator design pattern. Every piece of the pipeline changes a small thing, and hands the information over to the next piece.

Clients:

  • Large university site
  • High-profile government client
  • Large research institution client website

Challenge: go to Python 3, Plone 5.2 and Volto. With Volto, they spare a migration from Plone 5 to 6, at least partially. Volto is a way to sell the client a Python 3 upgrade: it is the right time to use Volto.

We use Jenkins to test the migration.

General backend things:

  • Old ATTopics to core Collections. Use collective.jsonify to export it, in a way that already looks more like a Collection.
  • Rich text: migrate to Volto blocks.
  • Post migration: collective.cover needed special handling.

What we polish to enter in Volto land:

  • We use collective folderish types.
  • deal with default pages
  • Convert RichText to Volto DraftJS
  • easily point to old website when content is not imported
  • fix some urls (we want to use resolveuid)
  • simple folders we should turn info page with listing block
  • simple collections we should turn info page with collection block

Question: can you open source the module that handles the RichText to Volto blocks migration?

Answer: yes, it is actually just 38 lines with React in Node. I tried to do it in Python, but could not get it to work. And we can sprint on it.

Open Plone Board Meeting

Posted by Maurits van Rees on October 25, 2019 11:46 AM

See the annual report.

Jen Myers has graciously offered to stay on next year as treasurer.

The Zope Foundation has been added to the Plone Foundation. Still ongoing, complex process.

With the Pylons community, those talks have stalled at the moment. Might happen in the future.

Financially we lost more money than we usually do. There was a trademark conflict that cost a lot more than we thought.

Some money should come from the Zope Foundation by the way.

Results of the vote for the board.

48 valid votes, one invalid. 2 went to the spam folder, but we found them. 30 percent of votes were in paper.

We had a vote by the foundation membership. Voted into the Plone board have been: Victor, Erico, Chrissy, Andy, Paul, Jens, Fulvio.

Meeting closed by Paul.

Thanks to Alexander, Kim and Carol who are leaving the board.

Riccardo Lemmi: Deployment Automation

Posted by Maurits van Rees on October 25, 2019 10:34 AM

We wanted to find an easy way to reproduce the installation process. We use:

  • Vagrant
  • Fabric
  • AWS
  • Boto 3 / awscli

Vagrant manages virtual machines and containers, for example with VirtualBox. I use the init command to do some more configuration on a default box.

Then I use Fabric to make an ssh connection to the machine and do remote actions. It uses invoke and paramiko for this. You can let it run the same actions on different machines. You can also use it to transfer files to and from the server, or use sudo to restart Apache.

More libraries with Fabric:

  • fabtools to make sure that for example Python is installed, or a user is created.
  • Cuisine: update files. You can also use this tool to ensure packages and users, so parts are very similar to fabtools.

Next as AWS, Amazon Web Services. With this we deploy production and test machines in a simple and replicable way. You can choose to add more CPUs, bigger disks, more memory, etcetara. I use EC2 (elastic compute cloud), EBS (elastic block storage) and EIP (elastic IP) for the most. Snapshots as simple backup tool. Security group rules as a firewall.

I create a machine with Boto 3 or awscli, both available with pip install. Why would I script this? To have "infrastructure as code". When you manually try to replicate a server, you can easily forget things.

Philip Bauer: Migrations! Migrations! Migrations!

Posted by Maurits van Rees on October 25, 2019 10:02 AM

I have some upgrade code that you can use, or for some parts copy and adapt for your use case.

Code: https://github.com/collective/collective.migrationhelpers

Upgrade steps:

  • Do imports in the function.
  • Do them conditional, so it does not fail if some package does not exist.
  • Do not let a step fail when run a second time.

You may want to disable LDAP temporarily in an upgrade step.

Get the birds eye view, with some code that reports:

  • how many items are there
  • which portal types
  • how big
  • local roles
  • etcetera
  • How many items are there that need to be replaced or removed, like PloneFormGen or Collage.

Divide and conquer:

  • Deal with one problem at a time.
  • Ignore problems that don't block you. You may try to solve something in the old Plone 4.3 site which is already fixed just by completing the migration to Plone 5.

You can register upgrade steps conditionally in zcml if needed, for example with zcml:condition="installed plone-52".

Make big problems small:

  • Write something that removes 98 percent of your content, for testing. Keep the structure in place though: Folders may have portlets or local roles that give problems that you want to know of.
  • Do not migrate blobs. The PDF that lives on the filesystem will not change. Move the blobstorage out of the way, use experimental.gracefulblobmissing during migration, and move blobstorage in again.
  • Copy the Data.fs.index too, if it is a really big site.

Forget the past:

  • remove all revisions
  • maybe manually use collective.revisionmanager for this.
  • pack the database with zero days (bin/zeopack keeps the last day by default).
  • Remove no longer needed portlets.
  • You can use uninstall and upgrade profiles. I like having the upgrade code in Python, but sometimes a profile is much easier.
  • Remove utilities and adapters from add-ons that will be removed.
  • Sometimes you cannot easily remove a package. You can make an alias for it. plone.app.upgrade has code for this, for example to not crash on an old site that still expects the kupu editor package somewhere.

Migrating LinguaPlone:

  • Content editors may have done crazy things, combining folders and content from different languages.
  • We have code to migrate this to plone.app.multilingual in migration.plonehelpers.

Update to Plone 5.2:

  • Use Python 2.7 at first.
  • Include Archetypes if needed for migration.
  • Run the migration to migrate to dexterity.

Archetypes to dexterity:

  • Use the methods from pac_migration from plone.app.contenttypes.
  • Start with the containers/folders.
  • Do one type at a time.
  • Especially for blob-like items there are options to speed this up, like disabling updating SearcheableText.
  • You can write custom migrators, which might just need a few lines.

Alternative: inplace migrator from ftw.upgrade. Interesting for large folders. Might be a nice PR to get this part into core.

To Python 3:

  • We need to remove some archetypes tools.
  • Then remove the Archetypes eggs from buildout and use Python 3.
  • Run the zodbupdate script.
  • When using RelStorage, you may want to switch to filestorage temporarily.
  • Read the documentation please, especially: - upgrade to Python 3 - upgrade zodb to Python 3

Why do it this way? I don't want every Plone company to have their own migration code stashed away somewhere. Please contribute to the core.

Upgrade to Plone 6? Install plone.restapi, use Volto, done.

Asko Soukka: ZServer reloaded with HTTP/2 and WebSocket support

Posted by Maurits van Rees on October 25, 2019 08:46 AM

A year ago, ZServer was one of the last Zope parts that did not run on Python 3. The idea was to use WSGI instead. I wondered why Python 3 would not be possible. So I tried porting it. It worked. And I added websocket support. You can use this to automatically push pages from Plone to Gatsby. Or show a notification when a page needs review, via content rules.

I replaced the old Medusa server with Twisted. ZServer was optionally using Twisted years ago, but this was ripped out, and I don't know why. But I got it working again. So for HTTP and webdav we have a Twisted http server and thread pool with an async FileSender, an AsyncIO/uvloop when avilable, standard ZConfig configuration and logging, And HTTP/2 works as well.

Bonus: websockets with PubSub messaging. It uses the Autobahn Twisted WebSocket protocol. ZeroMQ PubSub cockets using IPC-socket. The connection autosubscribes to local events. The server publishes a guarded event, for example indicating that an event is restricted to Editors.

One problem at this moment with the wsgi setup using waitress, is that the requests are limited to the number of wsgi threads, so server large files can become problematic. There are probably ways around this, but ZServer does not have this problem.

Status:

  • My ZServer fork is at https://github.com/datakurre/ZServer. See mostly branch datakurre/master.
  • You will need a special branch of plone.recipe.zope2instance as well.
  • Plus collective.wsevents, plonectl, collective.taskqueue.

Upstream:

  • In Zope 4, WebDav was made dependent on ZServer, by mistake?
  • Zope 5 assumes ZServer and WebDav no longer exist.

Remaining steps:

  • Get it upstream or not?
  • Eliminate dead code.
  • QA: tests, documentation
  • Release it.
  • Restore wsgi=off on Python 3.

I would be interested to hear when others are interested in using this. It is better if more people use this.

I am an introvert from Finland, so I did not ask the Zope people, or visited the Zope sprints.

Jens: There was no real reason to drop ZServer, except that no one has tried it because the code was so very old. Now you got it working, let's get this into Zope.

See the slides.

Paolo Perrotta: A Deep Learning Adventure

Posted by Maurits van Rees on October 25, 2019 08:14 AM

It is hard to see what part of machine learning is over hyped, and what part is actually useful.

Basis of ML (Machine Learning) is to get input and transform it to output. The ML gets a picture of a duck, and it gives as answer "duck". That is image recognition. You train the ML with images that you already label. And then you give it an image, and hope it gives a good answer. Same: from English to Japanese text. From an fMRI scan to an image visualized.

Simpler example: solar panel. Input: time of day, output: how much generated power. You would train this with data. The ML would turn this into a function that gives an approximately good answer.

Simplest model: linear regression. Turn the data into a function like this:

a * X + b

No line through the training points will be perfect. You find the line that minimizes the average error. So the ML uses linear regression to find a and b.

In our example, we try to guess the amount of mojitos we sell, based on the number of passengers that a daily boat brings to our beach bar.

But this may also depend on the temperature, or the number of sharks. With two variables you would not have a line, but a plane. With n, you would get an n-dimensional shape. You have n inputs, and give a weight to each input, and add them.

This is about numbers. For image recognition, you don't get a number as output. But we can apply a function to the result, and get a number between 0 and 1 that gives a likelyhood. For an image we may have 0.02 certainty that it is a cat, and 0.9 that it is a duck, so our answer will be the highest: a duck. Translated to the mojitos: what is the likelyhood that my bar breaks even?

This system with weights and a reducer function is called a perceptron. With a short Python program I got 90 percent accuracy on the standard NDIST test. We can do better.

We look at neural networks. This is basically: mash two perceptrons together. Or more. We are finding a function that approximates the data, and reduce the error iteratively.

What would I say is deep learning? Why has it grown?

  1. Neural networks with many layers...
  2. ... trained with a lot of data...
  3. ... with specialized architectures.

We helped Facebook with neural networks by tagging our photos. We gave them training data!

A lot of engineering is going into deep learning.

Generative Adversarial Networks: GANs. Example: a horse discriminator. Train the system with images of horses and others, and it should answer with: yes or no horse.

Other part: horse generator. Randomly generate images for feeding to the horse discriminator. You train this by saying: you did or did not manage to trick the discriminator. And you try to get better. I trained this a night, and after about a million iterations, I got pictures that are not quite horses, there is something wrong, but they are amazingly close.

Lightning talks Wednesday

Posted by Maurits van Rees on October 24, 2019 09:34 PM

Michele Finelli: Salama

This week I will do three easy pieces on Ferrara. And a rant.

You need to read this book: La Salama da Sugo. Salama = ninino (pig). This food is cooked, for a very long time, so the fat goes away. You do NOT eat it with bread. Eat it with purè, like mashed potatoes but better.

PyConWeb

A conference about web tools in Python. In Munich Germanu in May this year. We have many Plone talks and a training. See you next year?

Asko Soukka: plonetheme.webpacktemplate

One more way to theme your Plone. Webpack development server. Live demo. This is how we themed all our Plone 5 sites.

Code: https://github.com/collective/plonetheme.webpacktemplate

Sally and Kim: Plone Open Garden 2020

April 19-26 2020 in Sorrento, Italy. Registration is open soon.

See https://plone.org/events/sprints/plog-2020

Armin Stross-Radschinski: Search in Docs matter

(I lost a camera, if you find it, please give it to me.)

The default search in Sphinx really sucks. Local JavaScript, ugly, not translation proof.

Pretty live search with Algolia. Commercial SAAS engine. Free for open source docs.

It is already implemented for https://docs.plone.org. It uses Algolia, but not properly setup yet. Join my in a sprint on this!

See https://www.algolia.com

Maik Derstappen: plonecli

We did a lot of work continuing on plonecli, the Plone command line client. Talk in detail tomorrow. If you are new to backend development in Plone, you should have a look.

Code: https://github.com/plone/plonecli

Nejc Zupan: PyCon Balkan

We will do a Python conference in Slovenia. 18-20 April 2020 in Ljubljana, and three days of sprints afterwards. A track for web and data science. Also a workshop track. Come with the family. Everybody speaks English. We will do some excursions before.

Alexander Pilz and Johannes Raggam: py-spy

Debugging performance issue in production. We usually use Products.LongRequestLogger, it gives you the backtraces of long-taking requests on your server. But it is not compatible with WSGI and Python 3.

py-spy is non-intrusive. Just pip install py-spy. And then py-spy top and other commands. You can make flame graphs. You can attach it to a running process and get information from it.

Code: https://github.com/benfred/py-spy

Michael Töpf: React

Video with Manuel Bieh on the hype of React and why it is taking over the UI development. https://www.youtube.com/watch?v=ykaWoBDzuYA&feature=youtu.be

Lightning talks Thursday

Posted by Maurits van Rees on October 24, 2019 09:29 PM

Michele Finelli: cappellacci

My second of three easy pieces on Ferrara.

Now about cappellacci, or caplaz in the local dialect. No, it is not tortelloni or tortellini, please.

Pasta filled with pumpkin is a tradition of manu parts or Northern Italy, but easily butter and grated cheese. With vegetables? Not if you want to avoid getting arrested.

And remember: spaghetti a la bolognese does not exist.

Jens Klein: yafowil, declarative forms

Yet Another Form Widget Library. It is now a drop in replacement for z3c.form, by activating the yafowil behavior per portal type.

For example usage, we have an example package. And documentation.

Federico Campoli: Postgres carbonara

Spaggheti carbonara, the PostgreSQL way. Lots of SQL code and demo.

Code is in a gist.

Eric Brehault: PLIPs

PLIPs are PLone Improvement Proposals. There are no PLIPs at the moment. A lot is happening in Volto, but that is outside core.

If you submit, should you do the work yourself? No! You can, but it is not needed. Just give ideas to people who can develop it. Please do not hesitate.

There is a PLIP about the Dublin Core metadata behavior. This is a meta behavior for four others. This may change. If you have a concern about this, please add your thoughts to the PLIP.

Asko Soukka: robot tests

Robot framework is a way to automate tests. You can combine this with Jupyter notebooks. It can help you write better tests, including auto completion. See https://robots-from-jupyter.github.io/public/

Sven Strack and Erico Andrei: Jekyll and Hyde

How to rank Plone events. So many years, so many pictures. How do you compare pants and pools?

There can only be one answer. Food!

  • Bronze medal: Awesome Tokyo, Plone conference 2018.
  • Runner up: Somni Català, Barcelona Plone conference 2017.
  • Honorable mention: The Plone Cake, Boston Plone Conference 2016.
  • The winner is: Red Turtle Tiramisu, Ferrara Plone conference 2019.

Paul Grünewald: Digital Signage and Plone

[Note from Maurits: I expected this to go about digital signatures, but it is about showing signs on monitors, for example to inform your visitors.]

University Dresden. Content types for monitors, slide sets and slides. Contents: text, images, timetables, fullscreen video, ticker. Editing WYSIWYG inline using CKEditor, preview, scheduling

Code: tud.addons.monitor

Sven Strack: Docs analytics

Margot Bloomstein: "If we don't know if our documentation is successful, how will we know what we need to do to improve it?"

For the Plone docs we use an overwatch dashboard. Alerts when files on docs.plone.org have not been updated in a year. Open issues and PRs. Accessitility, performance, speed. Graphana, Prometeus, Matomo.

Christine Baumgartner and Ilvy: Alpine City Strategic Sprint 2020

Side note: German symposium "Plone Tagung" March next year in Dresden. See https://plonetagung.de/2020/

11 tot 14 februari in Innsbruck, Austria, come sprint with us (and enjoy beautiful and snowy Austria):

https://alpinecity.tirol

Rob Gietema: Volto form editor

I said there wasn't a form editor. But actually there is a schema editor. Saved to JSON schema. No backend implementation, but maybe we can sprint on this.

Panel: Ask Me Anything on Volto

Posted by Maurits van Rees on October 24, 2019 03:05 PM

Is Volto compatible with Guillotina? Not 100 percent. Question is how we keep the APIs in sync. If people figure out a shared generic API that works with both Plone and Guillotina as backend, this can work.

How do you migrate? You can migrate from Plone 4.3 to Volto. We did that with transmogrifier. Biggest problem is moving composite pages, like cover, to the Volto blocks. In a post migration step, you can fix things up.

Why Semantic UI? Rob researched all the popular ones, concentrating on how easy it is to theme or override stuff. Semantic UI makes this easy. You derive from a theme, and only override some parts.

For add-ons, how do you keep the backend part in sync with the frontent part? You don't. You use Python packages in the backend and node packages in the frontend. There will probably be a lot of packages that are only in the backend or only in the frontend. You may need to be careful in the frontend so that it can work with several versions of the backend of this add-on.

TypeScript has won the JavaScript wars. Will you support that? We don't want to support two ways of doing the same thing. So if we switch to TypeScript, we want to stop using ES6, also in the documentation. Same for class based versus function based.

Did you check accessibility? Yes. We have a static code checker ALM that helped us fix issues. Also the Cyprus tool. But we also do manual checks. Plone Foundation is trying to get funding for an audit.

Alok Kumar: Gatsby Preview with Plone

Posted by Maurits van Rees on October 24, 2019 01:54 PM

Gatsby Preview gives you hot reloading of content. So you change something in Plone CMS, and it immediately is visible on your Gatsby site.

Plone CMS -> Web hook -> Gatsby source plugin. When content is created, edited, deleted, we will fire a websocket event.

When you change the title of a folder, you should update the breadcrumbs of the children too, and also navigation in several places. It took me a long time, but I have implemented this.

I have created a Gatsby theme for Plone: https://github.com/collective/gatsby-theme-plone

Timo Stollenwerk: On the Road - Plone 6 and Beyond

Posted by Maurits van Rees on October 24, 2019 01:54 PM

Collections are hard to explain in user trainings. In most cases you have to explain default pages first. Can we make this simpler? In Volto we do this by having folderish pages.

What if we kill all content types, and rely on one content type to rule them all? Confession: I actually voted against folderish pages in the Framework Team twice. So I was sceptical. With Volto we thought, let's give it a try, and we got overwhelmingly positive feedback. It avoids cognitive overhead.

Same with the Save button. Pastanaga puts this in the top left corner of the page, and I hated it. Then I actually tried it for 30 seconds and loved it.

Composite pages. The idea has been around for ages, and there have been lots of add-ons that do this, but it was never really good enough for core. Always a problem for migration.

Naming things. Some names are open for discussion. We renamed tiles to blocks. We renamed proxy to teaser. Topics, Smart Folders, Collections: in Volto we simply call this Listings.

I rediscovered a blog by Alexander Limi, one of the original creators of Plone: https://limi.net/things-plone Still an interesting list after all these years. And I recognize parts in the Pastanaga UI.

Plone in 2019 is tiny. This means we cannot afford to reinvent the wheel over and over again. So Volto builds on React, which has far more contributors. We tried Angular and Vue too, but React seems the best system. And we use the standard stack with React, Redux, Semantic UI. This will be very familiar to most React developers.

We have trained people with zero experience in html and css, and in three days they could basically theme a Plone Site, even though we only started talking about Plone and Volto in the last two hours of the last day. It was really amazing.

With Plone 6 we aim to simplify the stack. There are several PLIPs (PLone Improvement Proposals), like having the Plone Site root be a dexterity item. Most of these are optional, they would be nice to have for cleaning things up, but they are not strictly necessary. You can use "Plone 6" today:

  • Plone 5.2
  • Python 3
  • Volto 4

Carrot and stick: selling a Plone upgrade just because you get Python 3, and not the unsupported Python 2 (stick), may not be enough to convince your customer. As extra carrot you can say: you can have Volto.

You can try Volto out live: https://volto.kitconcept.com (Still Volto 3 at this point.) This weekend, come join the code sprint at the conference.

Panel: Future of Plone

Posted by Maurits van Rees on October 24, 2019 01:54 PM

Panel with Paul Roeland, Philip Bauer, Timo Stollenwerk, Asko Soukka, Kim Paulissen. Panel chaired by Tim Nguyen.

This panel is not about Plone 6, but about what happens after: what could be in Plone 7, without thinking about limits.

What would you like to see as the biggest changes in Plone 7?

  • Some sort of machine learning, for example to provide tags for all images. Important for accessibility. The same for giving keywords.
  • Scheduling that works. Press release to go out at 9.00 on Monday should not be visible even when you know the url, which is what happens now.
  • Reliable intelligent pasting from Word and email.
  • Get rid of lots of old cruft in Plone. Not have so many ways to do the same thing. No more browser views, because we have a REST API and front end.
  • Get rid of the server rendered interface. Move all Plone UI to the ZMI, for admins who know what they are doing.
  • Make it easy for users to customize things without knowing Python.
  • Get rid of ZODB, ZMI, pickles, GenericSetup.
  • More Python, not too many different technologies.
  • Make Plone friendlier for smaller websites. Custom easy themes.
  • A form editor that can easily be reused as schema editor. Export as json schemata, and the Python schemas.
  • Customization is really hard. We had Products.Gloworm a long time ago. Click on a part of the site, and it shows you where it comes from, and allows you to customize it. This would make the first step of customizing easier.

What are the main pain points you have with Plone today?

  • When you create a new content type, and you add it to your site, it does not show up in the navigation. We should fix this. It bothers me that no one created an issue for that. We have a smart community, but people think that things have been decided by a secret committee, and there must be a good reason for it, so they never question it. Usually decisions are made five minutes before a final release, so it may not be the best solution.
  • The byline: I don't want the author to show, and you can switch this off, but it always shows somewhere.
  • Language and date/time notation should not be linked.
  • Accessibility should be more enforceable.
  • The entire front end stack: that is why we wrote Volto.
  • Behaviors: how we structure behaviors is done differently in Guillotina. We have lots of layers of flexibility, which is good, but also too much. At some point we have to take a step back.
  • I hate our TinyMCE link dialog.
  • Sane key bindings please, like CTRL-Enter to save the page.

What makes you envious when you look at other systems?

  • I regularly use the web forms from Drupal. Immensely powerful and user friendly.
  • Castle CMS has content quality checks.
  • Doing migrations is hard, so I am envious about systems that don't care about their users. I am joking.
  • Speed of GatsbyJS
  • Stability of Guillotina
  • Quick-start documentation. Short screen casts.

How can we remain approachable for enthusiasts?

  • Django had a perfect introduction. We need people that grow up with Plone. Easy tools to get your content in and out: we have those ways, but can be improved and documented more clearly.
  • For beginners it is hard to install add-ons. Buildout is new to them. Installing add-ons in a live site would be helpful.
  • Our developer team got energy from using the latest front end technologies. Keep up with the pace.
  • The feeling of the Plone community is very good. We try to do that for React as well. Building a community is incredibly important. Plone can give this to the JavaScript communities.

What will our theming story be? Will everything need to be done through Volto, and where will that leave Diazo.

  • Technically you can still use Diazo. With or without Diazo, theming has basically taken the same amount of time. Diazo gets you in the right direction faster, but you still need to do more.
  • Keep things simple: don't create and maintain our own difficult tooling. Volto builds on React and Semantic UI, who did the heavy lifting.
  • Plone is a do-ocracy: do what you want, you can use and improve Diazo.
  • For non-programmers, Diazo is still hard.

When will Plone drop server side rendering?

  • We have server side rendering of JavaScript. But I guess the question means Page Templates.
  • Plone 6 will keep the classic UI fully functional, is the plan.
  • After that, we can decide what to do. We could reduce two thirds of our code if we drop it.
  • It hurt when we moved to Python 3, because it was a lot of code. Currently it does not really hurt. But we can propose to remove it.
  • For Plone 6 we can say that PLIPs do not need to have traditional Page Templates.

How much of the REST API would Django CMS need to implement before we can call it Plone?

  • If the API contract is implemented 100 percent, then I don't care whether it is Django or Plone.
  • 180 percent, because Plone is also add-ons. Or 240 percent, because Plone is also the community.
  • Guillotina could replace Plone eventually. Django is just really different, making it hard to match the API.

How will Plone run 30 percent of sites on the Internet. In other words, how would Plone recapture mind share?

  • I refuse to discuss marketing in a bigger group than two.
  • Be better at introducing new types of developers to Plone. One for Python developers, one for React developers, for tinkerers, for users, etcetera. Make it easier to get into, and to grow.
  • Have plone.io website where people can try out Plone, deploy small sites.
  • Reaction to Volto was overwhelmingly positive.
  • Creating custom content types is really important for a CMS now. If we can provide that in an easy way, that really helps. People create things of which I think: Why are you doing this, Plone does that out of the box.

If you have improvement ideas, please do not suffer in silence. Talk with people, create a PLIP issue.

Rob Gietema: How to create your own Volto site!

Posted by Maurits van Rees on October 24, 2019 01:54 PM

We will go through the Volto training in about forty minutes.

To create a new Volto project, you need nvm and yarn, and then type the following:

$ npx @plone/create-volto-app my-volto-app
  • You can override public resources like the logo, styling, etc, from Volto.
  • Volto uses Semantic UI. For example, it uses the BreadCrumb component.
  • Theming: in theme.config you use the Pastanaga theme by default, but can pick and choose other themes for some elements. You can do a lot of theming just by changing some variables in less. less instead of sass is needed by Semantic UI for conditional imports.
  • You can override a component by copying the folder structure of that component from Volto to a customizations folder.
  • We use yarn i18n to extract all translatable strings into .po and then .json files.
  • Volto uses the DraftJS rich text editor. You can configure the buttons that are shown.
  • As is usual in React, Volto uses actions and reducers based on Redux.

If you want to learn about Volto, see the full Volto training.

Question: why don't you use React hooks?

Answer: This was introduced one and a half year ago in React, allowing to use more functions instead of classes. Quite cool. But then we would have two ways to do the same thing, and we try not to do that in Volto. We try not to jump on those hooks right away. If we switch after a while, we want to switch the whole code base.

Question: how do React developers experience this?

Answer: we did some presentations in React conferences. Reaction is that it looks really nice, we get positive feedback. I wrote a small reference implementation of the needed backend in node, and was finished in a few days, so people can do that too.

Eric Steele: The State of Plone

Posted by Maurits van Rees on October 24, 2019 01:53 PM

I start with some community updates from the past year.

  • We have six new foundation members: Christine, Thomas, Fulvio, Rikupekka, Kim, Stefania.
  • We have added Zope to the Plone Foundation.
  • 35 new core Plone committers.
  • Nine funded sprints.
  • There was a sprint in Barcelona this year from the Automization and Infrastructure team. They worked dist.plone.org and other core servers, rewriting Ansible scripts. They are looking to do two or three sprints next year, and they would like more people to join. You don't need to be an expert.
  • Google Summer of Code: Gatsby and Guillotina work was done.
  • Google Season of Docs: improved Volto documentation.
  • A lot more people have been following Plone trainings. The trainings are available for free via https://training.plone.org. Right now 17 different courses available.
  • Steve McMahon is stepping down from creating Plone installers. Please step up, and he will gladly help along in a transition period.

Python 3:

  • Python 2.7 will not get security patches anymore from 1 January 2020. So for Plone it was really important to run on Python 3. That now works: first Zope was upgraded, and now Plone 5.2 runs on Python 2 and 3.
  • Archetypes is now optional. It will not work at all in Python 3, so this is your wake-up call to update your code to use Dexterity.

Other changes in Plone 5.2:

  • Plone 5.2 gives us plone.restapi. This means BYOFE: Bring Your Own Front End. So if you want, you can create an own front end to talk to Plone as backend.
  • We have dropdown navigation.
  • The login code was rewritten, to not use the old skin layers anymore, but use browser views. Much better testable.
  • Theme improvements: static resource refactoring, removed old css/js resource registries, first main content then sidebar content.
  • Integrated Products.RedirectionTool so you can manage redirects.

Other:

  • Guillotina was started as a reimagining of Plone. It is really a separate project now. New version 5 has PostgreSQL indexing and search, pub/sub, there is Guillotina CMS.
  • The new Pastanaga UI is evolving. We have a design document for Plone now.
  • Volto: React-based front end that uses plone.restapi to talk to Plone. It uses the Pastanaga UI. Dramatically simplified rich text editor. It uses modern front end development, instead of building our own tools. We are improving the learning curve by removing Plone from the learning curve. That is awkward to say for me, but it brings more people in, front enders who can customize Volto. Volto uses JSX (JavaScript Expressions), Semantic UI. Significantly faster than our current built-in front end. And Volto works on both Plone and Guillotina. It is almost on par with the existing Plone UI, but some features are missing, like content rules and several control panels.

Volto helps with decoupled development. Plone is about keeping your data safe, having migrations. That is all backend stuff. We need a measured pace for this. The front end needs to evolve much faster, which Volto can do.

Challenges and open questions.

  • Can we move to one content type, that can behave like an event, or a page or a folder? Does that solve problems or introduce new ones? Timo will talk about that.
  • UI support: how much of the classic Plone UI do we keep around? Do we put all effort into Volto instead?
  • What is "Plone" now? We have said: Plone is a CMS, a framework, a product, a community.
  • We can say: Plone is the API contract: You can use Plone on top of Zope, or you can use Guillotina, you have an API on top of that, plus a front end. But what is important is this contract: security, flexibility, extensibility, user experience.

Maik Derstappen: State of Plone back-end development today

Posted by Maurits van Rees on October 24, 2019 01:33 PM

Frontend is nice, but it needs a backend, so let's talk about that. I will talk about plonecli, plone.api, Plone snippets for VS Code and plone.restapi.

You should use plonecli. It saves a lot of time for boring stuff. It helps you to create a product and enhance it step by step. It can cleanup and create a fresh virtualenv with a specific Python version and requirements and run buildout.

It has sub templates, for example to add a content type to your package. Now a sub template for a restapi service. Also one for upgrade steps, adding an upgrade profile per upgrade step.

We create a structure and files, and if you don't use some parts you can ignore or remove them. We will check if you have a clean git status before, and ask to commit, and afterwards we automatically commit.

You have good test coverage right from the start. All features added by plonecli have at least basic test coverage. You only write the tests for your own code. We make a tox environment to test different Plone and Python versions.

You can configure plonecli/mr.bob to your taste, changing the default answers or ignoring questions in a .mrbob file.

It is extensible, you can write your own custom bobtemplate packages and register them for the plonecli.

Visual Studio Code: snippets for Plone

There are snippets for schemas, fields, registry xml.

Use plone.api. It makes add-on code much easier to understand, without arcane incantations.

Ideas for the future.

  • plonecli:
    • add option to set Interface for view, viewlet.
    • REST API sub templates for de/serializer.
    • Graphical UI (ncurses-like) to make selecting options easier.
  • VS Code: plone.api autocompletion. VS Code is not so smart with namespaces. And it does not know buildout. You can use a buildout recipe that helps here though. Or use the generated zopepy script as your Python.

I would love more contributions. Don't be shy. There is no grand jury who makes decisions. Publish your own bobtemplates. Improve the VS Code snippets, or for other editors. Meet me at the sprints.

Question: is there a bobtemplate to create a new bobtemplate?

Answer: No. Should be possible.

Andreas Jung: Migrating a large university site to Plone 5.2

Posted by Maurits van Rees on October 24, 2019 12:50 PM

https://ugent.be is a Plone 4.3 site of a large Belgian university. Started in 2002 as Zope/CMF site. 90.000 pages, sub sites, 40.000 students, hundreds of editors, 90 add-ons. They wanted to move to Plone 5.2 and Python 3.

  • Traditional in-place migration: too manu add-ons, no one-on-one mapping possible.
  • Transmogrifier: not yet Python 3 at the time, too much magic hidden in too many places with blueprints.
  • So: custom migration solution.

Content types: standard types, plus four custom content types, including PloneFormGen. So that is quite reasonable. There is extensive usage of archetypes schema extenders.

Start: analyze and investigate your dependencies. - Based on Archetypes? Obsolete, replace. - No longer needed? Remove it. In Confluence we compiled a big table with for each package the basic information of how we would handle it: upgrade it, replace it, unknown yet, status of Python support.

Start with a minimal Plone 5.2 setup. Add one verified Python 3 compatible add-on at a time. Test extensively. Focus on content types first. Things like portlets can be handled later.

You need an export. We used a customized version of collective.jsonify. Core numbers: 90.000 json files, 55 GB data, 90 minutes, binary files base64 encoded.

We exported portlet assignments, default pages, layout information, workflow state, local roles, and pre-computed values for further efficient processing.

So we had 90.000 json files on the file system. We imported this in ArangoDB. Why use such a migration database? This allowed us to import only some portal types, or do parallel imports, and test complex migration steps like for PloneFormGen.

We briefly tried MongoDB, but that could not handle data over 16 MB. The json could be dumped unchanged in ArangoDB. This took 45 minutes.

Now we need to import this into Plone. Clean Python 3.7, Plone 5.2, plus the minimal set of packages needed for the content types. Import via plone.restapi. On top we have a dedicated migration package with special views. This handled things like translating between UIDs and paths.

The "magic" migration script is based on configuration in YAML.

  • Phase 1: pre-check the migration, remove target site if it already exists from previous test, create new Plone Site, install add-ons.
  • Phase 2: create all Folders, query ArangoDB for this.
  • Phase 3: create all non folders.
  • Phase 4: global actions. Check and migrate paths to UIDs in rich text fields. Assign portlets. Other specific fixup operations, like reindexing.

We migrated PloneFormGen (Archetypes) to easyform (dexterity). Export: one JSON for the FormFolder, one JSON file per field and action adapter. In easyform this needed to be turned into an EasyForm instance and a schema.

Topics (AT) to Collections (dexterity). Code was largely taken over from the plone.app.contenttypes migration.

From AT schema extenders to dexterity behaviors. First we made a list of which there were, are they in use, what do they do? Check which dexterity replacements there are. Create new behaviors.

Migrate packages to Python 3. This is mostly covered by talks of Philipp Bauer and David Glick. Common problems: utf-8 versus unicode, import fixes, implements to implementer. I rarely used the 2to3 and modernizr tools.

Some reimplementations: - portal skins to browser views - some packages with AT replaced with new packages with dexterity

Other problems:

  • improper file and image metadata
  • migration of vocabulary values, like old to new departments
  • repetitive cycles: always a bug occurs after a day of migration right before the end.

Quality control: - you need to check that migrated content and configuration is complete - "works for me" is nice, but others need to check too

Most of the packages have been removed, the setup is much smaller.

Status: - content migration is complete - must be tested in detail - integrate with the new theme, test this - need a replacement of a specific membrane usage - need work on castle/cas plugin

Takeaways: - Export Plone to JSOJN: 2 hours. Fast. - Import JSON to ArangoDB: 45 minutes. Fast. - Import ArangoDB to plone.restapi: 36-48 hours. Painfully slow. - 1.5 - 2.0 seconds per content object on average - cannot parellellize this import, because you would get conflict errors - So Plone and ZODB and painfully slow for creating mass content.

Question: in a similar setup we did a live migration from a live site to a separate new site. Did you consider that?

Answer: I tried this for other sites, but here I wanted to be able to partial imports, independent of the live site.

Question: default migration patches away all kinds of expensive indexing. You might want to consider looking at that. Migrating ten items per second is possible, although that is inline migration. And can you share the code, especially for the easyform migration?

Answer: could be done, but is not the primary focus of the budget currently.

A slightly older version of the code is on community.plone.org.

Anton Caceres: Performance Is Not Milliseconds

Posted by Maurits van Rees on October 24, 2019 10:11 AM

You can measure the performance of your website in the milliseconds it takes to handle a request, or how many requests per second. But this talk is about subjective speed: how fast does it feel? The contents of the page come to us at the speed of light. But it has to show up on the screen, and then your brain needs to process it.

And when you do an online payment, it may be irritating when it is slow, but even worse when you just end up on a homepage without indication of success or failure.

Perceived performance is dependent on the actual performance, the UX (user experience), and the expectec performance.

If your brain is active, time seems to move faster. If a page takes less than a second to load, do you really need a spinner or progress bar? The Polar app at some point changed to show a spinner for every action. Nothing changed in the speed, but reviews dropped: it reminded you that you had to wait all the time.

So instead, you may be able to distract the user. The user needs time to focus. So don't show a blank page, but show a skeleton page, which has roughly the structure of the page. Then the user already has a chance to see where the focus should be. I hope Plone will do this.

Show a spinner for ten seconds: slow. Show a simple animation for ten seconds: you don't notice that it is just as slow.

The perception of performance is often just as important as actual performance.

David Erni: Package spotlight: ftw.upgrade.

Posted by Maurits van Rees on October 24, 2019 08:40 AM

The upgrade-step:directory ZCML directive allows us to use a new upgrade step definition syntax. Less room for typos. Upgrades in the given directory are auto-discovered. Each directory is a generic setup upgrade profile. You no longer need a profile version in metadata.xml.

With the bin/upgrade command you can upgrade your Plone site, install an add-on, upgrade an add-on. This includes progress logging, so you see that something happens during long upgrade steps.

Why do we write upgrades? We want to consistently alter content and configuration. Prevent undocumented changes through the Plone UI: your policy package should have the needed configuration, without relying on manual changes done on the live site. We want to apply a minimal set of changes, instead of reinstalling an add-on.

We have several helpers methods, for example for:

  • class migration and in-place migration
  • rebuild, add, remove indexes
  • update object or workflow security

And there is much more, like deferrable upgrade steps, possible because we separately track which upgrade steps have been executed.

We are working on getting it running on Plone 5.2.

Code: https://github.com/4teamwork/ftw.upgrade

Michele Finelli: Software engineering during the Italian Renaissance

Posted by Maurits van Rees on October 24, 2019 08:08 AM

My first Plone conference was in Vienna. Glad to be here now.

Everything begins in ancient Babylonia. They had problems like measuring the size of fields after a flood. So they developed geometry (in Egypt too). They had to solve quadratic equations, like this:

a*x*x + b*x + c = 0

They used a different formula than what we are used to, because they had difficulty with the concept of negative numbers: you cannot have a field with a length of minus one meter.

This is basically software engineering, right?

Fast forward to the Renaissance. It is the year 1494, and Luca Pacioli publishes a book on mathematics. He said that the cubic equation (similar to the last, but raising to the third power) would probably never have a general solution. He was wrong. I show you a solution in Python, taken from Wikipedia.

Scipione dal Ferro found a solution a few years later. He seems to have come up with an algorithm to calculate it. Start with a formula, turn this into a series of instructions, and you have an algorithm.

Enter Gerolamo Cardano, Niccolo Tartaglia, and Lodovico Ferrari, all from the sixteenth century. Tartaglia also found a solution, shared it secretly with Cardano, who taught it to his student Ferrari. Cardano found the original solution from Dal Ferro, thought himself no longer sworn to secrecy, and eventually published the solution with Ferrari, together with a solution for the quadratic equation. Tartaglia (nickname for stutterer) became an enemy, and they fought mathematical battles, and we have some papers from this.

When you calculate with cubic roots, part of the solution uses the square root of minus one. At the time this was garbage. Currently we know this as an imaginary number. So it raised questions, which were finally answered two centuries later by Lagrange, Gauss, Ruffini, and Abel, between 1770 and 1826.

Why did this take so long? They lacked the abstractions needed to move forward. Babylonians did not know how to deal with negative numbers. Dal Ferro and company did not know how to deal with imaginary or complex numbers.

Moving forward. Is there a field where the lack of abstractions is preventing progress. I think yes, and it is called software engineering.

  • How do we specify what a software should and should not do?
  • How do we validate the behavior of code we write?
  • How do we describe our architectures?

The answer is currently text: sentences, diagrams, graphs.

The history of the discovery of the mathematical solutions was a tale to tell that in science, progress usually stalls until we are able to go from one language to another. In software, of course there are new programming languages, but I don't mean that.

The Agile Manifesto tells fruitful and interesting things, and gives important results. But it does not improve how we do the software thing.

Software has built a different world, just for example the internet. It should also be a better world. Privacy is going down, are we building a police world, without intending to?

We need a quest to find a better theory of software.

Save the Date: PLOG 2020

Posted by PLONE.ORG on October 23, 2019 02:13 PM

It's back again! The annual Plone Open Garden where the Plone community gathers, sprints, talks, learns, visits, eats, drinks and enjoys spring in Sorrento.

When: April 19-26, 2020

Where: Hotel Mediterraneo

Registration deadline: February 28, 2020

Like last year we hope to spend our mornings training and our afternoons sprinting. Training topics will be decided at the Plone Conference and funding for training still needs to be approved, so stay tuned.

Bring your own project to sprint on, or work with others on a community project. There will be daily standups so everyone can share, and discussions on hot topics scheduled throughout the week. Plus one day we'll have an excursion! Vesuvius, Pompei, Capri and Sorrento are all nearby.

The hotel is a beautiful, family friendly venue, with pool, garden, spectacular views and wonderful food. So think about bringing the whole family.

Details can be found on the event page. This year we have 1 quad room that accommodates 4 people - first come first served. Please plan to register early!

Nathan, Ramon and Eric: Status of Guillotina

Posted by Maurits van Rees on October 23, 2019 01:39 PM

Guillotina is a full stack REST API data framework to build applications. It is built on AsyncIO. We think it is simple, but with batteries included. We are inspired by the best of Python: ideas from Pyramid, Plone. We want to keep to the spirit of the Plone community. We want to build tooling so that creating a front end on top of Guillotina is easy.

Everything is defined in a JSON Schema / Open API 3 / Swagger. We have hierarchical content with security, so very Plone-like. Transactionally safe with PostgreSQL. Pub/Sub, indexing, lots more.

We released Guillotina 5 this Summer, requiring Python 3.7 or higher. AsyncIO has matured there, with a great context vars api, letting us simpligy our code.

Use cases. What are we using it for in production?

  • CMS.
  • Data Framework mix. Secure personal data and application data, gathered automatically. Reports. Partially data that needs to be searched: pgcatalog and ElasticSearch. Partially data that needs to be secure and massive: cockroach.
  • Machine learning processing. I often meet data scientists that always want a csv export. That takes too long for so much data. So have a front end that directly accesses the data in Guillotina, without needing an export. We use pickles to store data, and now this can contain numpy arrays.
  • Web / Mobile applications.
  • Knowledge Management. At Onna we work with some companies that have really big data, in emails, Jira, other tools. We can plugin this information from different sources and index it fast, because of AsyncIO.

How is Guillotina different from other frameworks?

  • First: AsyncIO has won. Django is also moving towards this.
  • Django: great OOTB feature set, lots of tooling, add-ons, support. Not a great REST API framework, and still comes from the days of server side templates.
  • Pyramid: no batteries included.
  • Plone: full feature CMS. Less good for large amounts of structured data. ZODB cannot really use AsyncIO.

Deployments:

  • Guillotina is good for large amounts of data, but also for small amounts.
  • The memory footprint is really low, again due to AsyncIO. If you use ElasticSearch for indexing you need much more memory.
  • PostgreSQL has lots of tooling around it.
  • In most cases we have three Guillotina processes, accessing 1 Postgres database.

Front end:

  • Volto has 80 percent compliance with guillotina_cms.
  • Grange (French for a barn, where you put all your farm tools).

Grange assembles your application fast. A lot of ready to use components.

Powerful principles: - traversing - state management - view overriding

Featureful:

  • Pastanaga-based UI library
  • Dynamic form generation, using the schemas that you get via json from Guillotina
  • Standard views and components, like breadcrumbs and navigation
  • i18n
  • npm install @guillotinaweg/grange. This covers the standard CRUD scenario with any schema, and you only need to declare your custom views.

Roadmap:

  • Guillotina 5 will be LTS.
  • Guillotina 6: ASGI, dbusers in core, tooling, more behavior APIs. May have longer alpha phase.
  • Some ideas for a Guillotina Object Server in Rust.
  • Protobuffer instead of pickles, as pickles tie us to Python.

Fred van Dijk: Generating a Word document from collaboratively edited structured content in your Plone Website

Posted by Maurits van Rees on October 23, 2019 12:44 PM

At Zest in The Netherlands we have several clients in Flanders, a part of Belgium. One is a government organisation for the environment. One of their sites is about water management. Information about water levels in separate areas, floodings, pollution, etcetera. The country has several partially overlapping governments, and water is flowing through all of them. So it is a bit complex.

For a new part of the site, they said: wouldn't it be nice to let Plone help in getting the next big report with plans for the next six years for the government together? So the report should be viewable as web pages. But wouldn't it be nice to also let this generate the official Word document?

A business analysis followed, with requirements and scope:

  • collectively edit it
  • seperate between core and background information,
  • export only the core going to a main PDF document

Ah, PDF export. Don't we love it? Always tricky, always corner cases, always too many pages, locking up your Zope server or crashing the export. For another site also dynamic graphics, table of contents. So: alarm bells went off. And do we really need PDF, can't it be something else?

Begin with the end in mind. Do I even want this project? Do we have the required skill sets in the team?

So what did we do? We made an MVP, a Minimum Viable Product. A prototype. Can we do all the parts of the process and have a basic document? Let the client test early to see we are on the same page.

Our internal customer was used to writing lots of plans, but a business plan was something else. Lots of things in there that would be nice. But which of those are really absolutely necessary?

Actually, the customer already made a mockup in a website. We had a meeting, things changed, new plans.

In the end, the Word document did not seem that much of a problem. But our initial main structure with two content types was too rigid. We simplified to one content type, and it was fine. Some different views for the content type to choose from.

We use:

  • plone.api to find content.
  • beautifulsoup4 to analyze the html.
  • python-docx for generating a Word document.

Interesting extra problems were internal links, and later footnote support. Not really supported in python-docx yet, but with some lower level code it worked.

Risk: some html things may be too difficult to put in Word reliably. But they can edit the document later.

Something extra: we use pas.plugins.ldap and it is working smoothly now. We have some code to migrate from plone.app.ldap to this, and want to publish that.

Remark from Andreas: with a similar project we discovered that it was difficult for editors to make a section from say level 1.2 to 3.2 in the default Plone UI. So we made a special view for that. And watch out: tricky to lock everything down in TinyMCE.

Question: did you try collective.documentgenerator?

Answer: no. The things we found, were too big and generic.

Plone Beethoven Sprint 2019

Posted by kitconcept GmbH on September 25, 2019 09:00 AM

beethoven sprint group image

21 developers from nine different countries gathered in Bonn, Germany between June 20th and 24th to work on implementing the upcoming Plone 6. The sprint at the office of the kitconcept GmbH has been declared a “strategic sprint” by the Plone framework team. Sprint topics included working on Volto, the Plone REST API, and Guillotina.

Volto

Volto is a new ReactJS-based frontend for Plone. It was started by Rob Gietema in 2017 and it is actively developed since then. Volto will become the default frontend for Plone 6. It implements a complete new user interface called Pastanaga UI, which was developed by Albert Casado.

Victor Fernandez de Alba and Rob Gietema led the efforts to further enhance Volto together with Paul Roeland, Nicola Zambello, Piero Nicolli, Janina Hard, Jakob Kahl, Thomas Kindermann, Maurizio Delmonte, Stefania Trabucchi, Rodrigo Ferreira de Souza, Fred van Dijk, and Steffen Lindner.

Pastanaga UI Toolbar

The new editing toolbar has been part of Volto right from the start. Though, so far we did not fully implement every detail that Albert has imagined for the Pastanaga UI toolbar. Victor has been working on this for quite a while already. During the sprint, he integrated his work and polished it. Victor and Rob also worked on a new sidebar that shows up on the right side of the Volto user interface that holds additional information like page metadata and controls the settings of the individual tiles.

volto toolbar The new Volto edit toolbar

New Tiles: Collection, Slider and Table

At the Plone conference in Tokyo, we came up with a list of tiles for the new Volto composite page editor that we would like to implement. On top of that list were the listing/collection tile, the slider tile and the table tile. Piero, Rodrigo, and Rob started to work on the listing/collection tile that is supposed to become a replacement for the Collection Type in Plone.

Jakob and Janina worked on copying over the code of a slider tile that was developed by kitconcept. Rob implemented the table tile. Before you could ask Rob about the progress, he already finished his work, like always…

volto table The new Volto table tile

Accessibility

Accessibility has always been a first class citizen in the Plone community. Plone is used by many government and public websites where accessibility is a basic requirement. Paul, Thomas, Timo, and Victor worked on improving the accessibility of Volto. Our goal is to fully support the WCAG 2.1 standard with the “AA” level of conformance.

Timo set up ESlint-based static code analysis checks and Cypress-based axe accessibility tests. Paul, Thomas and Victor then started to fix the reported accessibility issues. At the end of the sprint, we were able to fix all the reported violations. Our CI build now fails if new accessibility violations are committed.

Documentation

Sven Strack, the Plone documentation team leader, joined our sprint remotely from Amsterdam to discuss how to maintain and structure the Volto documentation in the future. We all agreed that we will use Markdown as format for our docs, since this is the de-facto standard in the JavaScript community as well as in most other Open Source communities. As much as we like Restructured Text, it seems that Markdown won that battle.

We currently use MKdocs for the Volto documentation and Styleguidist to document our React components. There are lots of different tools to choose from, Docz, MKDocs, MDX, Gatsby to just name a few. We agreed that we need to do more research to make an educated decision regarding our doc toolchain.

There will be a Google Season of Docs project this year where a technical writer will help us to enhance the Volto docs further. More Volto Features Nicola added a feature to display the currently used Volto version in the Plone control panel, he added animations for page transitions and added an Italian translation for Volto. Together with Victor, he worked on a toast component for user notifications.

More Volto Features

Piero worked on improving and fixing the event type view and added styling for the listing tile.

Victor and Steffen worked on the image sidebar.

Rodrigo added a feature that allows the user to use SHIFT+RETURN to create a new line in a Volto text. By default Volto creates a new text tile on RETURN. Rodrigo also looked into the Add-ons control panel work that Eric Steele started some time ago.

Nilesh joined us remotely from India and continued his work on the users and groups control panel for Volto.

Steffen fixed a bug in the subjects search. Rodrigo and Thomas looked into our Cypress-based acceptance tests.

Fred looked into what consequences it would have for us to get rid of the description tile and how we could customize the domain in Plone to allow to send portal emails via the Volto frontend.

Stefania and Maurizio worked on a product definition of Volto and an elevator pitch for clients and users. Together with Paul, Fred and Timo, they had a longer discussion about how to position Volto and Plone 6 in the market.

Plone REST API

The Plone REST API builds the foundation for the new Volto frontend and Plone 6. It has been under active development in the last four years and seen more than 80 releases in that time period. The REST API is stable, battle-tested and production-ready and part of the Plone core since the release of Plone 5.2.

Thomas Buchberger and Lukas Graf led the REST API efforts during the sprint and worked with Carsten Senger, Roel Bruggink, Janina Hard and Timo Stollenwerk to further enhance the REST API, fix bugs, improve the documentation, and the error handling.

API Team in the graden Plone REST API discussion in the garden

Querystring Endpoint for the Collection Type

One of the most important new features, that has been developed during the sprint, is the new querystring endpoint. This endpoint will allow us to implement the Collection type for the Volto frontend. Lukas implemented this essential feature with help from Rob, who took care of the frontend implementation in Volto.

New Features, Bugfixes, and first time contributions

Thomas added support for the retrieval of additional metadata fields in summaries in the same way as in search results. Lukas made the @types endpoint expandable and the @users endpoint easier to customize. Janina fixed setting the effective date on workflow transitions. This was her first contribution to the Plone core. On the same day she signed the contributor agreement, got her fix merged and released, which is quite an accomplishment for a single day. Way to go Janina!

In addition to that, the team fixed lots of smaller bugs (#780, #729, #764, #755, #777, #732, #738) during the sprint. REST API Error handling Carsten, Nathan, Thomas, Lukas and Timo had a longer discussion about error handling in the Plone REST API and about how we can improve things. We considered and discussed implementing RFC7807 and decided against adopting it.

Developer in office Sprint discussions…

The RFC did not see much adoption and it only slightly differs from what we are already doing. A breaking change does not seem to be worth the effort. We decided to create a separate error reporting component unifying the error reporting and ensuring consistency. We also started to fix a few inconsistencies right at the sprint.

API Documentation

Guillotina is using Swagger to document its API. Nathan explained the approach he took for that and Carsten started to integrate Swagger to generate dynamic api docs in plone.restapi.

REST API Releases

Most of the work we did on Plone REST API during the sprint has been released in Plone REST API 4.1.4, 4.2.0, 4.3.0, and 4.4.0.

Dexterity Site Root

Roel Bruggink joined us on Saturday and Sunday to continue his work on turning the Plone site root into a Dexterity content object. He fixed some complex recursion and acquisition problem when migrating the site root to Dexterity. His Plone Improvement Proposal (PLIP #2454) has been accepted by the Plone framework team and it will build the foundation for a new Plone 6 branch that we plan to cut this year.

Guillotina

Guillotina is a new AsyncIO REST data API. It has been written from the scratch by Ramon Navarro Bosch and Nathan van Gheem. It can easily scale billions of objects via its REST API. With the CMS addon, it is compatible with the Plone REST API. Therefore you can run Volto on a Guillotina backend and Guillotina might be able to replace our existing Plone backend in the future.

Guillotina Developer The Guillotina master thinking hard. :)

Nathan and Ramon worked on Guillotina 5. The Guillotina 5 release brought Python 3.7 support and the use of the context vars APIs. They refactored quite a bit, moved parts of addons into core(swagger, caching, pg catalog), implemented OOTB PostgreSQL catalog, added a Helm/Kubernetes/Docker configurations for the Guillotina CMS addon that can be configured to act as a Volto backend, worked on improving caching and filestorage.

Websockets for Plone

At the Plone Open Garden 2019, Asko Soukka started to work on bringing websocket support to Plone. Websockets allow bi-directional communication between the server and the browser. With websockets, the server can send messages and notifications to the browser. This can be used to implement notification systems or collaborative editing to just name a few possible applications.

Asko continued his work during the Beethoven sprint and created an example where comments that are added to Plone automatically pop up as toast notifications for other users. He published a Docker image with a Twisted-based ZServer and implemented the foundation to use that feature for gatsby-source-plone. This will allow us to create an extension that will allow editors to live-preview changes they do in Plone in a GatsbyJS site.

Developer in Office

Karaoke, Barbecue and Garden Parties

A Plone sprint is much more than just programming. Collaborating, meeting old friends, talk, discuss and having a good time together are the ingredients that make a Plone sprint. We started our sprint on Thursday with a free React and Volto training for newbies. At the evening when everybody arrived, we went to a local Karaoke bar for a revival of the epic Karaoke sessions we had in Tokyo last year.

Karaoke Karaoke at the “Nyx”

On Friday evening we ordered Pizza and hacked the night away at the office and our garden. On Saturday we went to the “Arithmeum”, a mathematics museum owned by the Research Institute for Discrete Mathematics at the University of Bonn.

Arithmeum Our tour at the “Arithmeum”

Afterwards we had a garden party with a barbecue and we spend the afternoon sitting in the garden and the office and continued to drink and hack on Plone.

Developer in garden Barbecue in the garden of the kitconcept office

Summary

The Beethoven Sprint 2019 was a major success. We were able to start working on the main missing pieces for Volto. The new edit toolbar is in place and the foundation for the new tiles/metadata toolbar on the right side of the UI has been finished. We also implemented a new table tile.

We implemented a proof-of-concept for the new listing / collection tile and implemented the required querystring endpoint in Plone REST API. Both Volto and the Plone REST API got lots of enhancements and bugfixes.

Both the Plone REST API and Volto are stable and used in production. We will use the next month to further enhance Volto and implement the missing bits and pieces. Our plan is to present Volto 4 at the Plone conference in Ferrara at the end of this year and cut a Plone 6 branch afterwards that contains the Dexterity Site root, folderish content types and other enhancements that are needed make Volto the default frontend for Plone 6.

Guillotina and the Websockets support for ZServer together with Gatsby as another frontend for Plone are amazing projects by the smartest folks in our community. Our diversity is one of our main strength and we are looking forward what the future will bring!

We had a great time at the sprint. Thank you to everyone who attended! We thank the Plone and Python community for their ongoing support. The Plone Foundation, the Python Software Foundation, and the German Python Software Verband made this event possible with their sponsorship as well as our company sponsors Abstract, Starzel, Werkbank and Zest.

We are looking forward to see you all in Bonn next year!

Developer in garden The “Hofgarten” near the kitconcept office

Nominations Open for Plone Foundation Board of Directors

Posted by PLONE.ORG on August 28, 2019 09:48 AM

If you have an interest in helping the governance of Plone, and particularly the energy and time to pitch in, please consider nominating yourself to serve on the Plone Foundation board of directors for 2019-2020.

Nomination Process

  1. Log in on plone.org and go here: 
    https://plone.org/foundation/meetings/membership/2019-membership-meeting/nominations
  2. Add a page there with your name in the title.
  3. For the body, discuss:
    • Who you are
    • Why you're interested
    • What you think you can add to the Plone Foundation
    • Most importantly, the name(s) of one or more Plone Foundation members who "second" your nomination
  4. Once ready, click "submit" in the workflow drop-down menu to get a reviewer to look at your nomination.
  5. Nominations will be accepted until October 20 2019, 23.59, UTC. The election will be conducted in conjunction with the annual meeting, which will take place in Ferrara, Italy at the Plone Conference 2019. All active members of the Plone Foundation will be eligible to vote.

About Board Membership

The Plone Foundation is a not-for-profit, public-benefit corporation with the mission to "promote and protect Plone". That has meant that the board is involved in:

  • protecting the trademark, copyrights and other intellectual property, including considering licensing and usage issues;
  • hiring the release manager;
  • working with other communities like Zope, Guillotina, Pyramid, ...
  • working with various committees, including marketing and membership;
  • handling "other stuff in the community" as needed, e.g. helping craft policy on plone.org and plone.com about commercial listings
  • but not: directing Plone development. The board facilitates, but does not direct, the development of Plone itself.

While there's lots of work that happens online, much of the critical business of the board is conducted during video meetings every two weeks — typically, board meetings last about an hour to 90 minutes though occasionally they can run over to handle time-critical issues.  Please consider whether this fits your schedule, since missing more than an occasional meeting severely limits the ability of the board to reach quorum and conduct business.

Historically, board meetings have been organized to occur during daytime hours in America and evening hours in Europe, currently at Thursday nights, 19.00 UTC in northern hemisphere summer and 20.00 UTC in northern hemisphere winter. That can always change with new board members.

In addition, there is a board mailing list (private), where we discuss things in addition to the meetings.

This is a working board. Be ready to regularly take on and complete responsibilities for board business.

The board writes no code and makes no development decisions. It is much more concerned with marketing, budgets, fundraising, community process and intellectual property considerations.

You do not need to be a Foundation member to serve on the board (in fact, board leadership is an excellent way to become a Foundation member). All you need is to get an active Foundation member to second your nomination.

The Plone Foundation is interested in broadening the diversity of our leadership, with regards to gender, ethnicity, and geography.

If you have questions about the nomination process, contact the board: board@plone.org

Announcing the Plone Conference 2020 selection process

Posted by PLONE.ORG on August 22, 2019 07:23 AM

It is time to begin planning the 2020 edition of the main Plone event of the year.

The annual Plone Conference brings together users, integrators, developers, designers, and other interested folk from throughout the world for a week of training, talks, and sprinting. Plone conferences are also an expression of community spirit: they are organized by a company, user group, or other entity with ties to and a history with the Plone community and are in essence not-for-profit events.

They tend to cover a wide range of subjects: not just Plone, but also related technologies such as Volto, Guillotina, React, Angular. And of course topics such as accessibility, user experience, documentation, continuous integration, community organizing and others. Past conferences have also been places where special interest groups like universities, NGO's and others have held meetups.

The Plone Foundation is soliciting proposals to host the 2020 Plone Conference. The timeline of the selection process allows groups and organizations interested in hosting the 2020 Plone Conference (or beyond) to work with the Ferrara team for hands on experience during this year's conference.

Previous editions of Plone Conference were held in many places all around the world:

  • 2003: New Orleans, Lousiana, USA
  • 2004: Vienna, Austria
  • 2005: Vienna, Austria
  • 2006: Seattle, Washington, USA
  • 2007: Naples, Italy
  • 2008: Washington, D.C., USA
  • 2009: Budapest, Hungary
  • 2010: Bristol, UK
  • 2011: San Francisco, CA, USA
  • 2012: Arnhem, Netherlands
  • 2013: Brasilia, DF, Brazil
  • 2014: Bristol, UK
  • 2015: Bucharest, Romania
  • 2016: Boston, MA, USA
  • 2017: Barcelona, Catalunya, Spain
  • 2018: Tokyo, Japan

and, for 2019, we will gather at Ferrara, in Italy.

If you have a place in mind, and are willing to host the next edition, don't be shy: submit a proposal! If you want, the Board is of course willing to help you in refining your proposal. So don't hesitate to contact the Board while still developing your proposal; most of the proposals of the last year were developed in cooperation.

The Plone Foundation will accept proposals beginning September 1st through September 30th, 2019. Please submit your proposals to board@plone.org.

The Foundation Board of Directors will review proposals and open those that are viable for voting by the Foundation membership between October 14–18, 2019.

Announcement of the winning proposal will happen at the end of Plone Conference 2019 in Ferrara.

Everything you need to know to submit a proposal, including the full schedule for the process and in-depth requirements for hosting, is outlined in the official Plone Conference 2020: Call for Proposals.

On behalf of the entire Plone community, we look forward to your conference proposals!

Looking forward to the Season of Docs!

Posted by PLONE.ORG on August 18, 2019 04:04 PM
Cris Perez is an experienced tech writer who will collaborate with a team of Plonistas to work on the Volto documentation for the next three months
 
This is made possible by the Google Season of Docs, who have announced the projects that participate.
 
Volto is a new, modern, React frontend for web content management systems.Through use of a mature REST API, Volto has access to all the features of Plone, one of the most secure, reliable and flexible content management systems, with an almost 20 year track record.
 
During the Google Season of Docs, we aim to streamline the Volto documentation ffor developers from different backgrounds. The documentation will be extended and reorganized to cater to a more diverse set of developer experiences and backgrounds.
 
Developers and designers familiar with React need information on how to interact with Plone features so they can create feature-rich websites with custom workflows and permissions, and enjoy Plone's comprehensive backend services such as the built in search engine and its rock-solid content management features.
 
Developers already familiar with Plone will want to know how to leverage modern JavaScript tools to create better user experiences.
 
And, given the worldwide and diverse Plone and React/Volto communities, we will pay particular attention to avoid unwanted cultural, gender or other biases, thus ensuring that the documentation is inclusive and accessible for the widest range of readers.
 
We will use clear, concise language to help the many developers for whom English is not their first, or even second, language.
 
Cris Perez will be working with the following team: Timo Stollenwerk, Víctor Fernández de Alba, Kim Paulissen, Sven Strack, Paul Roeland.
 
As always, we thank all volunteers for their contributions!

Three Members Join the Plone Foundation

Posted by PLONE.ORG on August 11, 2019 12:46 PM

The Plone Foundation welcomes 3 new members after unanimous confirmation by the Foundation's Board of Directors on August 1, 2019. Membership in the Foundation is conferred for significant and enduring contributions to the Plone project and community. The Plone Foundation Membership Committee overwhelmingly recommended each applicant for their ongoing contributions to Plone.

Christine Baumgartner

Christine Baumgartner

Christine is the "Partner" in Klein & Partner KG, a founding member of the BlueDynamics Alliance. She has been part of the Plone community since the (second) Vienna Plone Conference in 2005. In 2006 she began organizing the first of many sprints: the Golden Roof Sprint 2006, Castle Sprint 2017, and five (!) Alpine City Sprints in Innsbruck, where for the past two years they have been joined by Zope core contributors. To reward sprinters for their hard work, she has organized noteworthy excursions to promote team building and forge long lasting personal connections in the Plone community and between the Plone and Zope communities. 

Kim Paulissen

Kim Paulissen

Kim is a senior software developer and avid Plone Conference attendee, starting with Plone Conference 2010 in Bristol, then San Francisco (where she gave two talks), Arnhem, Bucharest, Barcelona, and, most recently, she participated in the Plone Open Garden Sprint 2019. She will join the ranks of Plone Conference trainers when she helps give the Mastering Plone Training at the Plone Conference in Ferrara this October. Kim has contributed code and many bug fixes for Plone and its add-ons, and she is co-maintainer of the collective.ckeditor package.

Rikupekka Oksanen

Rikupekka OksanenRikupekka has been using and promoting Plone at the University of Jyväskylä since 2004 (and Zope since 2002) where there is a huge Plone presence. He started at first as a web developer then later took on project management, customer support, and trainer roles, training hundreds of people to use Plone. Rikupekka has been blogging since 2009 about Plone, has presented at Plone conferences since 2014, and organized the renowned 2017 Sauna Sprint.

 


The Foundation is the trustee for Plone's intellectual property, works to protect and promote Plone, and has over 80 active members.

Released is released

Posted by Andreas Jung/ZOPYX on August 02, 2019 04:52 PM
Should you be allowed to remove your own packages from PyPI or any other public package repository? Short answer: NO!!!

The Guardians of the Zope and Plone Galaxy

Posted by PLONE.ORG on July 24, 2019 11:03 AM

Over the past three years, the Zope and Plone communities have been working hard to prepare both applications for Python 3. The end of the journey is near, and as a result the applications are faster and modern again! This was achieved without sacrificing the long-standing high security standards of Zope and Plone.

Work began in 2016 after the Alpine City Sprint in Innsbruck. PLIP #1351: Update Zope Dependencies  by Johannes Raggam, who with the support of Philip Bauer and Jens Klein, initiated the work on Plone. At the time, the mood among Plone and Zope was fairly negative due to the daunting task at hand.

Momentum started with joint sprints like gocept's "Resurrection" sprints (later named "Saltlabs Sprints") in Halle/Saale on the Zope side and the "Alpine City Sprints" in Innsbruck on the Plone side. Many additional sprints followed: the Buschenschank Sprints, Berlin Strategic Sprints and Plone Conference sprints around the world. The need to modernize both systems brought the developers of the Zope and Plone communities together. The close cooperation of the development teams made the effort a success. Now the communities stand side by side together.

"From the Zope community's point of view, it is very nice to see Plone core developers' vested interest in Zope. It would be wonderful to have regular exchanges between the Zope and Plone communities", said Michael Howitz, Zope release manager and core developer at the Alpine City Strategic Sprint 2017.

In March, just before the release of Plone 5.2, representatives of the Zope and Plone community were so kind to dedicate their time to an interview by Christine Baumgartner. Many thanks to Michael Howitz from Germany (gocept), Davi Lima from Brazil and Alessandro Pisa from Italy (syslab).

How have the worlds of Plone, Zope, and Python changed with Python 3, and what are the old and new goals?

Davi pointed out:

"By moving to Python 3, Plone gets more inline with the current Python community to share its differentials to other available languages and products. With current tooling explosion on the front-end, companies feel the vacuum of a solid back-end on which they can trust. Their projects may grow for many years without hitting basic, but deep rooted system limitations, like workflow not being extensible enough."

Alessandro told me:

"The most positive thing is that the two worlds now seem to be more connected than in the past."

Michael said:

"With Python 3, Zope is back on the cutting edge of technology. Efforts have shown that there is a living Zope community, and that Zope and Plone can benefit from each other."

What changes are relevant for users, or what should users expect?

Alessandro said:

"A better user experience and introduction of up-to-date technologies."

Davi explained to me:

"Developers can expect easier syntax - for example: handy string interpolation and formatting, more predictable behavior for data structures, higher level dynamic structures like context managers, and more performance with the all-around preference for generators and lazy data structures, as opposed to in-memory ones."

Michael pointed out:

"In principle, nothing should change for the end-users. For developers, adapting code and data to be compatible with Python 3 means a lot of work. But this is also an opportunity to modernize existing code."

Are any new and/or other use cases for Zope and Plone? Or, do certain use cases disappear?

Davi explained:

"Most companies adopt React or Angular for their front-end and develop their back-end mostly from scratch, tackling problems as they come instead of preparing a solid set of features (e.g. workflow, permissions, content types, content rules and so on). What happens is these features are invariably requested and refactorings of such size commonly bring companies to the floor, or lead to development paralysis situations from which it's hard to get unbound."

With Plone 5.2 there are changes in the development process. What have developers, administrators and editors paid attention to?

Michael said:

"I can only speak for developers: First, they should upgrade their projects to Plone 5.2 on Python 2, then migrate to Python 3. Zope 4, which is used for Plone 5.2, is intended as an interim release only. Relatively soon after the final Zope 4 release, work will begin on Zope 5, which will no longer support Python 2."

Alessandro explained:

"I will give you a concern about system administrators: the new infrastructural changes in how Zope is served with WSGI will introduce new patterns."

Davi told me:

"It's important to understand how front-end is architected, its basic building blocks are key for a project that only executes goals correctly, but also that feels good while using. Also it's important to learn how to use Plone API and Plone RESTful API to make content operations trivial. Check out the Volto project!"

I know there were a lot of stumbling blocks on the way to the new release. How were they solved? Which stumbling blocks still have to be solved?

Alessandro claimed:

"Porting the code to Python3 was an epic success."

Davi explained:

"Plone has an extensive codebase and also a very fragmented one. This means changes were necessary in dozens of packages, in a coordinated effort touching heterogeneous parts: from database migrations to displaying of widgets. This was solved with a series of community immersion meet-up events, called sprints, in which people from different countries gather to discuss, code and improve. It seems to me that the next steps revolve around making add-ons compatible with Plone 5.2, so a lot of focus is on documentation, talks and getting the word out."

Michael pointed out:

"Stumbling blocks were some subtle differences between Zope 2.13 and Zope 4 that were not noticed within Zope or where Plone expected a very specific behavior, which was not available in Zope 4. The migration of ZODB to existing projects in the real world that go beyond toy applications is still open."

After all this work, can we expect advantages compared to the old Plone and Zope?

Davi said:

"Faster user interface, better integration with current front-end tooling."

Michael expected

"A higher execution speed."

And Alessandro opined

"The adoption of mainstream technologies."

In closing, what advantage do we have compared to other systems?

Alessandro claimed:

"Well, the main one for sure is the way we deal with security."

Michael confirmed:

"There is a current, well-maintained system, which has an active community that supports each other and actively works on the care and development."

Davi Lima was convinced:

"Permissions and workflow systems, contextual editing (as opposed to a separate "Admin Interface"). In Python, everything is an object, in Zope, everything is a component. Decades of accumulated experience and solutions that you have not even realized you already or soon will have. Relying on the same CMS as CIA.gov and FBI.gov ;-) "

These are the Guardians of the Zope and Plone Galaxy:

(At least a whole bunch of them taken from our records. This list is in alphabetical order, but is not complete)

Alessandro Pisa, Alexander Loechel, Andreas Jung, Christian Kreutzer, Christian Theune, Christine Baumgartner, Cillian de Róiste, Davi Lima, Daniel Havlik, David Glick, Fred van Dijk, Georg Bernhard, Gil Forcada, Godfroid Chapelle, Hanno Schlichting, Harald Frissnegger, Jan Mevissen, Jens Klein, Jens Vagepohl, Johannes Raggam, Joni Orponen, Jörg Zell, Jürgen Gmach, Katja Süss, Maik Derstappen, Manuel Reinhardt, Markus Hilbert, Matthew Wilkes, Martin Häcker, Maurits van Rees, Michael Howitz, Michael Töpfl, Nathan Van Gheem, Robert Niederreiter, Patrick Gerken; Paul Roeland, Peter Holzer, Peter Mathis, Phillip Bauer, Roel Bruggink, Stephan Klinger, Steffen Allner, Sune Brøndum Wøller, Thomas Lotze, Thomas Massmann, Thomas Schorr, Tres Seaver, Jens Vagelpohl, …

text by Christine Baumgartner and Jens Klein - 2019-07-24

Plone 5.2, the Future-Proofing Release

Posted by PLONE.ORG on July 21, 2019 04:17 PM

An update to secure the future

Plone 5.2 is the latest version of the most fully-featured, open source web content management system. Plone 5.2 is the culmination of over 3.5 years of development that was driven by the looming deadline of January 1, 2020: the impending end of support for Python 2.x.

Python 3 Compatibility

Plone is written in Python, the programming language that has taken the world by storm and is popular for learners, scientists, web-developers and machine-learning alike. Support for Python 3.6, 3.7, and 3.8 is now included with Plone 5.2, opening up the future for Plone’s continuing 20 year reputation for unrivaled security and stability.

Plone 5.2 ships with Zope 4, which itself comes with Python 3 compatibility, Web Server Gateway Interface (WSGI) support, and a new look.

Work on Plone 5.2 began to take shape in January 2016, with a Plone Improvement Proposal (PLIP) to update its dependencies to use early Zope4 versions and culminated in the completion of the proposal to port to Python 3 in February 2019.

REST API

Once available as a separate add-on, a REST API is now also included in Plone 5.2, making it even easier to use Plone with modern JavaScript front ends such as Volto (based on React) and to integrate Plone with other applications.

User-Facing and Developer-Focused Changes

But Plone 5.2 isn’t just a backend upgrade: it includes user-facing features such as new dropdown menus, configurable redirects (customizable URL shortening and content linking).

Developers will also find a more pluggable and customizable login mechanism and a modernized virtualenv-based buildout.

Community-driven development

For almost 20 years now, the Plone software has continued to innovate and evolve with the times, and its world spanning community maintains a dynamic schedule of collaborative sprints and an annual conference bringing together its integrators and users, which include corporations, universities, all levels of governments and well-known government agencies.

Feature-rich and ready-to use

Plone is unsurpassed in the out of the box experience it provides for collaboratively creating, maintaining, and managing large amounts of content. Some of its feature highlights include:

  • customizable workflow for controlling how content gets reviewed and published
  • built in content history and version control
  • content rules that automatically take actions, such as notify users or move content based on customizable conditions
  • sophisticated, fine-grained security through permissions, roles, users, groups, and folder- and object-level sharing authorization
  • accessibility compliance, meeting WCAG 2.1 AAA
  • integration with enterprise authentication and authorization via LDAP/ActiveDirectory
  • deployments that scale, whether Docker, on-premise, or in the cloud

Future Proofing

Plone 5.2 is a must-have as of January 1, 2020, for individuals and organizations with complex content serving needs, a rigorous publishing workflow, a multi-person content creation team, or all of the above.

Links

About Plone: https://plone.com

Demo site: http://demo.plone.org

Download Plone 5.2: https://plone.org/download

Release notes: https://plone.org/download/releases/5.2

Documentation: https://docs.plone.org

Upgrade guide: https://docs.plone.org/manage/upgrading/

Roadmap: https://plone.org/roadmap/2019-plone-roadmap

Screenshots

New dropdown menus

screenshot showing mouse hovering over the navigation bar

Configurable Redirects

Redirects Screenshot_2019-07-21 demo plone org.png

Refreshed Zope4 Look

Zope4 Screenshot_2019-07-21 Website.png

About Plone

Released under the GNU Public License (GPL), Plone was created in 2001 as an extension of the groundbreaking Zope web application framework to support the easy creation and management of rich websites and intranets. Plone is among the top 2% of all open source projects worldwide.

About the Plone Foundation

As Plone’s adoption and contributor base grew tremendously the Plone Foundation was established in 2004 and tasked with protecting and promoting Plone. It is a US 501(c)3 non-profit modeled after similar ventures such as the Apache Software Foundation. The Plone Foundation provides support for development and marketing. It is the legal owner of the Plone codebase, trademarks, and domain names. Its goal is to ensure Plone remains the premier open source content management system to broaden its acceptance and visibility.

 

 

Q&A with Philip Bauer

Posted by PLONE.ORG on June 30, 2019 07:11 PM
Philip Bauer (Starzel.deis the Munich-based organizer of the effort to create and release Plone 5.2, the version that brings Plone into the modern Python 3 world. 
 
What follows is a question and answer session with Philip, for which he promised not to make up any new German superlongcompoundwords.
 

Philip, you’re a pop star!

*plays wicked air guitar solo*
 

What does Plone 5.2 mean to you?

It's the best thing since sliced bread since I can finally develop in Python 3 while still doing Plone.
 

How much work did you put into Plone 5.2?

About 8 Quadrillion Mythical Man-Months. Frankly I've grown a lot of grey hair not because of working on it but during.
 

When did you start working on Python 3 compatibility for Plone?

I started work in October 2017 shortly before the Plone Conference in Barcelona, right after we merged Zope 4 support into Plone – which in turn we began to work on in January 2016. So altogether it took three years. It's been on the roadmap for a good while, I just never thought it would be me doing it but somehow nobody stepped up – so I did. [Interviewer’s note: it helps when you have a bottle of Ardbeg at the ready *cough*] 
 

What will you do with your free time now that “it’s all done”?

I'll add support for Python 3.8. I actually did that this week, and it only took about 8 hours. See https://github.com/plone/Products.CMFPlone/issues/2890[Interviewer’s note:“Oops, I did it again!”] 
 
After porting to Python 3 was finally done I really needed to do some paid work since my bank account took a real hit from all the unpaid work. I'm glad to be able to do some exciting new projects now with my company, most of them already on Python 3 or soon to be migrated.
 

Does your family still recognize you?

The kids loved the beard that I grew during the last months of porting and don't know me now that I’ve shaved it off. [Interviewer’s note: it seemed a rather scary looking beard.]
 

Was it worth it?

Totally! I had the privilege of working with some of the smartest and nicest people I know. I also learned so much about Python, Plone and Zope which probably made me a better developer. But there are also some things I'd love to unlearn; there are some dragons in our codebase and fighting these can be frustrating.
 

Has your liver recovered?

Still working on it.
 

What do you look forward to the most in Ferrara?

It will be weird to see how everyone will take for granted that we now run on Python 3 and have a great new frontend (Voltoin the works. 
 
Many take it for granted how much time and love is put into Plone. 
 
I love the new energy that is noticeable in the community. I think it comes from the certainty of having a stable backend without an expiration date and the excitement of developing a new frontend.
 

When will you finally !@#$#@$ host the Plone Konferenz (note the “K”) in Munich?

In 2030. We'll do a keynote (with “K”) on Python 6. [Interviewer’s note: it does NOT take that many years to make a new major version of Plone!] [Another interviewer’s note: Philip is not going to get off that easily, either.] 
 

Are you seriously now a Volto believer?

It is amazing to finally have a modern frontend without having to develop the framework for it ourselves. Volto is a huge step into the future, and choosing React for it was a very wise decision. 
 
I still hate JavaScript as a programming language but since the toolchain has matured so much it is finally fun to develop with it. Also: Most of the heavy lifting is still done in Python.
 

What has Plone done for you lately?

Plone paid for the beer in my fridge; it also paid for the fridge and for the house the fridge stands in. 
 
Speaking of beer: I'll have one right now.

Medical guideline portal Onkopedia.com relaunched - on top of Plone 5.2 and Python 3.7

Posted by Andreas Jung/ZOPYX on June 24, 2019 10:07 AM
In the third major iteration of the Onkopedia project, ZOPYX and partners relaunched the complete onkopedia.com portal with a new fully responsive layout and an updated Plone backend based on the latest Plone 5.2 version and finally Python 3.7 under the hood.

Sign up for Plone Conference 2019 in Ferrara!

Posted by PLONE.ORG on June 18, 2019 09:15 PM

Plone Conference 2019 in Ferrara is revving up! 

This is the first ever Plone conference at a UNESCO World Heritage site... not to be missed.

  • Early bird tickets are on sale now
    • (full pass, daily pass, party-only)
  • Sign up for the included training classes
    • see tentative training class list and schedule below
  • Now accepting your talk submissions
    • Deadline for submissions is June 30, 2019

Now is also a good time to begin making travel arrangements.

Our friendly conference organizers have provided a lovely guide to Ferrara that covers getting there, accommodations, places to eat, and things to visit.

We look forward to seeing you there!

Training Classes (tentative)

Become a Plone Power User How to effectively manage your Plone website content and use the full power of Plone Paul Roeland, Kim Nguyen 1/2 day
Mastering Plone 5 Development Learn how to develop custom projects with Plone 5.2 and Python 3 using the latest best-practices Philip Bauer, Thomas Schorr, Kim Paulissen 2 days
Guillotina Getting started with Guillotina: the asyncio REST resource API Nathan 1 day
React Get started with React & Friends Rob Gietema, Roel Bruggink 1 day
Volto Create your own site using Volto Rob Gietema, Roel Bruggink 1 day
Testing Plone Learn what and how to test yout Plone products Andrea Cecchi 1/2 day
Pyramid APIs in the Real World With Pyramid and pyramid_openapi3, will will fire up a demo RealWorld.io application. We will then introduce bugs, and demonstrate techniques for troubleshooting, debugging, and testing. From there we will extend the application to include generating reports, working with task queues for long-running processes, importing and exporting data, swapping front-ends, and a few other common use cases. Nejc Zupan and Steve Piercy 1/2 day
Volto theming hands-on: Building plone.org in Volto Expand your Volto skills by attending this hands-on training. We will be
building together the next plone.org site. We will cover Volto theming,
customization, and the required components and backend plone.restapi
endpoints to match plone.org features and look and feel.
Victor Fernandez de Alba 1/2 day
Waitress, there's a Plone in my WSGI. Can I talk to the Supervisor? How to deploy and operate Plone 5.2 on Python 3 using WSGI. Thomas Schorr 1/2 day

Tentative Schedule

20190618_ploneconf2019_training.png

Using Glitch for Pyramid Learning

Posted by Jazkarta Blog on May 09, 2019 08:02 PM

Screenshot: Pyramid Quick Tutorials showing pyramid-view-decorators, pyramid-view-predicates, pyramid-view-renderers and pyramid-asset-specifications

At this year’s Jazkarta sprint we decided to invest some time in creating Pyramid learning resources. We are fans of the Pyramid web framework, and we felt that it would be nice to work on something that would make it easier to get started with, especially for novice developers.

Enter Glitch. This is a platform that is designed for letting users explore web apps created by other Glitch users. When you see an application that does something that you want to do, you can “remix” it, which means that an exact copy, with all its code and resources, is created in your own workspace (which is some sort of Docker container). You can then change it in any way you like. This is very powerful, because you get a working application that is hosted for you, and every code change that you make is immediately reflected in the web app page.

This turns out to be really useful for teaching web application development concepts, because students can see the code for the example, read an explanation, and then right away remix the code to try out their changes, which they can see working instantly. No need for setting up a workspace, checking out code, or running a web server. Just try out an idea and see the result right there, or share it with other people. Like “view HTML source” but with code, which is one of Glitch’s goals.

We decided to create a Glitch collection that would showcase a few of Pyramid’s most distinctive features, along with a simple Hello World style tutorial. The first thing was to verify that Pyramid could be installed. Glitch’s main development target is Node.js, but its containers install Python and they allow the user to run pip, so it’s possible to install most PyPI packages including Pyramid. (For those who might be wondering: sorry, Plone is not installable, because it needs system dependencies that are not installed by Glitch.)

Glitch supports a setup file named glitch.json, which is what allowed us to install Pyramid:

{
 "install": "pip3 install --user pyramid",
 "start": "python3 pyramid_app.py"
}

In the json above, you can see that we install Pyramid using pip, and we can add any other dependencies for our project there. We can then use the “start” key to tell Glitch how to start our application.

Once we did that, the only other thing needed was to create our Pyramid app, which in this case uses code directly lifted from the trypyramid.com site:

from wsgiref.simple_server import make_server
from pyramid.config import Configurator
from pyramid.response import Response

def hello_world(request):
    return Response('Hello Glitch')

if __name__ == '__main__':
    with Configurator() as config:
       config.add_route('hello', '/')
       config.add_view(hello_world, route_name='hello')
       app = config.make_wsgi_app()
    server = make_server('0.0.0.0', 3000, app)
    server.serve_forever()

That’s it. Glitch automatically runs the application (the only caveat is that it needs to use port 3000). You can configure it so that any code change automatically restarts the app. After this, if a potential new user sees your project, she can just use the remix button and have a copy made in her workspace. Any change that she makes, like changing the hello text, will be immediately visible on the page.

Overall, we found Glitch to be a very intuitive and easy to use service. You can see the collection of simple tutorials that we created here:

Pyramid Quick Tutorials

 

Jazkarta Goes To LA

Posted by Jazkarta Blog on May 09, 2019 07:47 PM

The Jazkarta team at LAX-C, a city block sized Asian food distributor and restaurant supply

This year Jazkarta’s annual sprint was held in the city of Los Angeles, graciously hosted by Alec Mitchell who lives a block away from LA’s City Hall.

View of the Los Angeles City Hall from Los Angeles Street

We had fun exploring the city – Union Station, Chinatown, LAX-C (pictured above, a gigantic Asian food distributor and restaurant supply), Grand Central Market, Griffith Observatory, and the Hollywood Farmers’ Market where we bought lots of delicious fruit.

Citrus and strawberries from the Hollywood Farmers' Market

We attended an improv comedy performance at the Upright Citizens Brigade Theater, ate lots of tacos, had original Phillipe’s French Dip sandwiches, visited brew pubs and drank lots of local beer.

The Jazkarta team in front of the Highland Park Brewery.

We even had a private tasting of cheese, beer, and wine pairings.

Plate with five different cheeses and a variety of fruits and nuts

Oh, and we also sprinted on a variety of topics!

Alec and Jesse Snyder ported several of our Plone add-ons to Python 3 and Plone 5.2:

Carlos de la Guardia created a set of Pyramid Quick Tutorials on Glitch – more about that coming in another post.

Matthew Wilkes and Jesse researched the best way to set up Pyramid functional tests using pytest and webtest in an efficient manner. They were unhappy with their previous model, which relied on in-memory databases. They settled on a pattern that switched out private variables of the sessionmaker to join a longer-running transaction for each test. This is more self-contained than adapting the session setup machinery in the app itself, and works transparently on different database backends. Matthew made a gist of the relevant code.

Alec made a prototype of an app to provide faceted search of bibliography references kept in Zotero. The app has a Elasticsearch backend and a React front end. It uses the Zotero API to pull the references into Elasticsearch for querying.

Carlos investigated what it would take to move the CastleCMS quality checking features into a Plone add-on. It’s a work in progress.

Left to right: Carlos, Sally, Matthew, Alec, Jesse and Nate in front of a painting of the Griffith Observatory

It’s wonderful to be able to work together in person, instead of remotely from our home offices spread across 3 countries and 2 continents. We try to do it once a year and always have a fantastic time. This year was no exception.

Sprinting in Sorrento

Posted by Jazkarta Blog on May 08, 2019 10:44 PM

Sunset over the island of Capri, as viewed from the sprint venue

This year I helped organize the annual sprint in Sorrento called the Plone Open Garden. Plone is a highly secure and feature rich open source content management system written in Python. Plone has no single company behind it, its IP is held by a non-profit foundation. Progress on the software happens through the efforts of a self-directed community, whose members sponsor sprints to work on particular problems.

PLOG took place April 12-19 and it was a great success, attended by 18 people from 7 countries. There were two training classes:

  • Porting Plone sites and add-ons to Python 3
  • Creating a web application using the restapi and Volto, the new React front end for Plone

We also accomplished a lot of work:

  • Python 3 ports of over a dozen Plone add-ons
  • Accessibility improvements for classic Plone and Volto front ends
  • New Volto features and documentation
  • Marketing discussions
  • Documentation discussions
  • …and various other Plone improvements

I posted a news item on plone.org that describes the details of what we accomplished. But Plone sprints have a social side as well as a technical side. Friendships are renewed, meals and news are shared, and a good time is had by all. This year’s PLOG was true to form. Here are some of the things we enjoyed.

1) Breakfasts at the Hotel Mediterraneo, the sprint venue since 2007. Pictured below is the pastry table; there are also tables for cheeses, meats, fruits, breads, cereals, eggs, and more. You can squeeze your own orange juice and optionally add some Prosecco.

Breakfast buffet at the Hotel Mediterraneo - the pastry table

Dinners, also included, are a wonderful way to end a day of sprinting. First and second course, plus dessert.

A typical dessert at the Hotel Mediterraneo

2) Happy hour at the Mediterraneo. We book the rooftop for happy hour at least once during the sprint, where we enjoy the sunset over Capri (pictured at the top of this post) and the extensive cocktail menu. That’s a Black Mohito at the top and a Negroni at the bottom.

 Drinks from the Hotel Mediterraneo cocktail menu

3) Excursions. One day we walked to Sorrento’s old harbor and had lunch with a view of Mount Vesuvius.

A view looking down at Sorrento's old harbor

4) Being with old friends and meeting new ones. Always the best part.

Sprinters on the rooftop of the Hotel Mediterraneo

 

Plone Welcomes Students for Google Summer of Code 2019

Posted by PLONE.ORG on May 08, 2019 11:00 AM

Google Summer of Code is a global program focused on bringing more student developers into open source software development. Students work with an open source organization on a 3 month programming project during their break from school.

Competition to participate is intense, both for open source projects and for students. While Plone has participated successfully many times, for this year other projects got in. Yet, thanks to the friendly folks at the Python Software Foundation, who function as an 'umbrella' organization, we can still work together with three students. Thanks, PSF!

The students are:

To them: welcome to the Plone community – we wish you a great learning experience over the coming summer. Congratulations!

Mentoring them will be these fine folks:

  • Ramon Navarro Bosch
  • Nathan Van Gheem
  • Kumar Akshay
  • Timo Stollenwerk
  • Asko Soukka
  • Nilesh Gulia
  • Shriyansh Agrawal
  • Oshane Bailey
  • Dylan Jay

with support from Cris Ewing.

Fun fact: an astounding 5 out of those 9 have been GSoC students themselves!

Plone Selected to Participate in First Google Season of Docs

Posted by PLONE.ORG on May 02, 2019 07:54 PM

Google is trying out a new program this year called Season of Docs whose goal is to provide a framework for technical writers and open source projects to work together on improving the project's documentation. Paul Roeland and Sven Strack recognized this as an excellent opportunity to improve Plone documentation, and aided by discussions at PLOG (as described in the sprint report), Paul put together the winning proposal.

We are now listed as a Season of Docs participating organization, which means that tech writers interested in the program will soon be browsing our Season of Docs ideas page where we have two projects listed:

  • Improve the Developer Onboarding Experience for Plone 6
  • Improve Documentation for the React-based Frontend "Volto"

Thank you to everyone who made this happen!