Planet Plone - Where Developers And Integrators Write

Tokyo Plone Conference 2018 Keynote Speakers Announced

Posted by PLONE.ORG on October 12, 2018 03:56 PM

Our three daily Plone Conference 2018 keynote speakers will be:

  • Eric Steele, Plone's release manager overseeing the Plone 4 and 5 releases, a longtime contributor of Plone add-ons and member of the legendary Penn State WebLion team
  • Jim Fulton, who led Zope development for many years and created many key Zope components and a co-creator of  buildout
  • Torajiro Aida, a Japanese high school student who started programming at the age of 6, whose interest is not only technology itself but also how technology affects our society and ethics. For his art work, he received the New Face Award at the Japan Media Art Festival.

Join us in Tokyo next month! Registration and details at https://2018.ploneconf.org

A conference sponsorship would be a great way to show your support for Plone and to enhance your visibility in the community! Find out more about conference sponsorships

2018 ploneconf keynote speakers

Three Members Join the Plone Foundation

Posted by PLONE.ORG on October 02, 2018 10:20 AM

The Plone Foundation welcomes 3 new members after unanimous confirmation by the Foundation's Board of Directors on September 13, 2018. Membership in the Foundation is conferred for significant and enduring contributions to the Plone project and community. The Plone Foundation Membership Committee overwhelmingly recommended each applicant for their ongoing contributions to Plone.

Katja Süss

Katja Süss

Katja got started in Plone by posting issues, committing features to existing add-ons, adding translations and submitting bug fixes. She has been co-organizer of monthly meetups in Zurich, welcoming integrators and developers and getting insights into customer needs and technical features. In addition to participating in several Plone and Python conferences, she has joined several sprints over the years, including the Innsbrück Alpine City sprint (Plone on Python 3) this past January. Most recently, she has contributed a new add-on to the community: the audio playlist player, collective.playlist.

Davi Lima

Davi Lima

Davi has been working with Plone and participating in the community since 2007, initially building and maintaining Plone 2.5 sites for a group of Brazilian universities and then moving on to bigger projects with several organizations: Simples Consultancy, Unified, Interlegis, Brazil's Presidency of Republic, Enfold Systems and Kitconcept. He has actively participated in community discussions, answering questions on IRC, StackOverflow, GitHub issues and the community forum, in English and Portuguese. He has participated in many sprints, and attended four Plone conferences (volunteering at two of these), where he gave Python and Plone training classes, presented talks, workshops, and lightning talks on XDV/Diazo, Plone React (PyConWeb Munich, EuroPython Edinburgh).

Eric Bréhault

Éric Bréhault

Eric's main contributions are Plomino and Rapido, two add-ons for non-experts to create their own applications in Plone. He has made contributions to Plone core, mainly for Plone 5. More recently, he focused on frontend technology, developing and maintaining plone.restapi-angular, an Angular 2+ toolkit allowing the building of sites and applications based on the Plone REST API. He is an active member of the Framework Team and has given talks, including inspirational keynotes, and provided training at several Plone conferences.

 

 

The Foundation is the trustee for Plone's intellectual property, works to protect and promote Plone, and has over 80 active members.

First Redactor Release

Posted by TestTheDocs on October 01, 2018 04:32 AM
What Is Redactor A work in progress editor-in-chief to highlight potential linguistic and structural issues with your text. Vision The aim of Redactor is to be a tool that helps check text with a variety of best practice based tests. While the test configuration is opinionated, inquisitive users may change and override the defaults. Initially, users run the tool locally, in the future we plan a remote (maybe commercialized) service triggered by user actions such as a CLI command or Git push.

Plone Goes To Pycon

Posted by Jazkarta Blog on September 12, 2018 09:50 PM

Plone booth at Pycon

I’ve just returned from Pycon where I had a great time staffing the Plone Foundation’s booth. It was fun to see old friends and introduce new people to Plone, Python’s open source, enterprise-grade content management system. Pycon attendees are wonderfully friendly and curious, always interested in learning new things. The venue – Cleveland’s downtown convention center – was lovely and the rain mostly held off. We gave away dozens and dozens of Plone-branded neck pillows and water bottles and multi-tools and pins and magnets, not to mention blue M&Ms arranged in the shape of the Plone logo. I arrived with 2 suitcases full of swag and left with 2 empties. Thanks to Witek and Nate from Jazkarta and to Chrissy, Anthony and Carol from Six Feet Up for helping set up and run the booth, and to the Plone Foundation for sponsoring Pycon. Open source rocks!

Announcing collective.siteimprove

Posted by Jazkarta Blog on September 12, 2018 09:47 PM

Screenshot of collective.siteimprove UI, expanded

As I reported back in May, at our last Jazkarta sprint Witek and Alec began work on a Plone add-on, collective.siteimprove, that provides integration with the Siteimprove content quality checking service. I’m pleased to announce that the add-on has now been thoroughly vetted by the folks at Siteimprove and the resulting version is available from Pypi!

What Is Siteimprove

Siteimprove is a respected service for maintaining and improving web content quality. Customers who sign up for the service get automated scans of their websites which check for content quality, accessibility compliance, SEO, data privacy, and performance. Site rollups and per page reports are available via email and from a customizable dashboard at siteimprove.com.

Siteimprove also provides an API which allows for the development of CMS plugins – integrations of the Siteimprove service within content management systems. This allows content editors to get immediate feedback on pages that they publish. This is great because it lets editors see problems while they are in the process of editing a page, instead of getting a report after the fact and needing to click through links to fix things.

Graphic explaining how Siteimprove works

 

Why Siteimprove for Plone

Plone, the premier Python-based open source content management system, is an enterprise scale CMS that is widely used by large organizations with large websites. These are just the types of organizations that can benefit from a tool like Siteimprove, which has a reputation for being an excellent service for maintaining and improving website content.

The Jazkarta team was delighted to be able to contribute to the Plone community by creating an add-on that integrates Siteimprove’s CMS plugin features into the Plone editing process. Now anyone with a Plone website can easily integrate with Siteimprove simply by installing an add-on – all the integration work has been done.

collective.siteimprove

After collective.siteimprove is installed on a Plone site, there will be a new control panel where Siteimprove customers can request and save a token that registers the domain with siteimprove.com. After that, authorized users will see an overlaid Siteimprove button on each page that shows the number of issues found.

Screenshot of the collective.siteimprove UI, collapsed

 

When clicked, the overlay expands to show a summary report of page errors and an overall score, as shown in the image at the top of this post. After an edit, users can click a button on the overlay to request that Siteimprove recheck the page. They can also follow a link to the full page report at siteimprove.com.

Plone+Siteimprove FTW

Now anyone who has a Plone website can easily integrate with the Siteimprove service and take advantage of all of Siteimprove’s enterprise-scale features while they are working on their content!

Plone 4.3.18 has been released

Posted by PLONE.ORG on September 08, 2018 02:26 PM

Plone 4.3.18 has been released, containing bug fixes and updated packages.

To download it, please see the download page.


Changes

Zope2: 2.13.27 → 2.13.28

DocumentTemplate: 2.13.3 → 2.13.4

plone.recipe.zope2instance: 4.3 → 4.4.0

New features:
  • Added support for setting instance-home option. [zupo]

  • Added support for setting CGI environment variables. [zupo]

Bug fixes:
  • Regard 'parsed_version' of setuptools > 38.7.0 does not return iterable anymore, fixes #37. [ida]

zest.releaser: 6.13.4 → 6.13.5

robotframework: 3.0 → 3.0.4

robotframework-selenium2screenshots: 0.7.0 → 0.7.2

selenium: 2.53.5 → 2.53.6

sphinx-rtd-theme: 0.1.5 → 0.1.9

Pillow: 3.3.0 → 3.3.3

importlib: 1.0.3 → 1.0.4

lxml: 3.6.0 → 4.2.1

WebOb: 1.4.1 → 1.4.2

Plone: 4.3.17 → 4.3.18

Products.Archetypes: 1.9.17 → 1.9.18

Products.CMFPlone: 4.3.17 → 4.3.18

Products.CMFUid: 2.2.1 → 2.2.2

Products.GenericSetup: 1.8.8 → 1.8.9

Bug fixes:
  • When metadata.xml parsing fails, show the filename in the ExpatError. Fixes Plone issue 2303 <https://github.com/plone/Products.CMFPlone/issues/2303>_.

  • Require five.localsitemanager less than version 3. Version 3 requires a too new Zope2 version.

Products.PlonePAS: 5.0.15 → 5.1.0

New features:
  • Notify PropertiesUpdated event when member properties are changed [ezvirtual]

collective.monkeypatcher: 1.1.3 → 1.1.5

Bug fixes:
  • Fix import for Python 3 in the tests module [ale-rt]

  • Fix import for Python 3 [pbauer]

five.localsitemanager: 2.0.5 → 2.0.6

  • Don't complain if the site root has no Acquisition parent. [davisagli]

  • Removed zope.site dependency. Using Zope 2.12 it is an indirect dependency and using Zope 2.13 or later it is no longer required. [yuppie]

  • Ensure that the PersistentComponents has no aquisition wrapper before passing to the superclass, to allow the caching of component roots in zope.interface to make a weakref to this root. [MatthewWilkes]

plone.app.layout: 2.3.17 → 2.3.18

plone.app.textfield: 1.2.10 → 1.2.11

Bug fixes:
  • Python 3 fixes [pbauer]

plone.app.upgrade: 1.4.4 → 1.4.5

plone.app.z3cform: 0.7.7 → 0.7.8

plone.cachepurging: 1.0.14 → 1.0.15

Bug fixes:
  • consider purging to be enabled when it's enabled (even if no servers are listed) [skurfer]

plone.folder: 1.0.10 → 1.0.11

New features:
  • Improve logging in case ordered index is not consistent [tomgross]
Bug fixes:
  • Remove ancient buildout config [tomgross]

  • Replace deprecated testing assertion calls [tomgross]

grokcore.security: 1.6.2 → 1.6.3

plone.app.lockingbehavior: 1.0.4 → 1.0.5

Bug fixes:
  • Add coding header on python files. [gforcada]

  • Unskip test for Zope 4, as isolation problems are already fixed. [thet]

plone.app.versioningbehavior: 1.2.0 → 1.2.10

New features:
  • Used plone i18n domain and removed locales folder. [klinger]
Bug fixes:
  • Do not break in the case of dexterity objects with relations migrated from something else (usually Archetypes). [ale-rt]

  • Use zope.interface decorator. [gforcada]

  • Fixes #25: URLs like ${absolute_url}/@@images/${uuid}.png are not converted on @@version-view. [rafaelbco]

  • Updated Traditional Chinese translations. [l34marr]

  • Update Italian translations [ale-rt, cekk]

  • Fixes #10: Views for Image and File versions don't work. [rafaelbco]

  • Update French translations [enclope]

  • Updated basque translation [erral]

  • Correct functional test, it was not checking correct on version1. [bloodbare]

  • Synchronize translations [vincentfretin]

  • provide better description of how new versions are created when in manual mode [vangheem]

  • Ported tests to plone.app.testing. Removed PloneTestCase / p.a.testing compatibility hack. [jone]

  • Remove dependencies on zope.app.container and rwproperty. [davisagli]

  • Added Italian translations. [cekk]

plone.api: 1.8.3 → 1.8.4

Bug fixes:
  • Call processForm with {None: None} dict as values. This prevents processForm using REQUEST.form and overwriting values already set by invokeFactory. Fixes issue 99 <https://github.com/plone/plone.api/issues/99>_. [david-batranu]

  • Simplification/minor speedup: Permissions checks now directly use AccessControl. Technical its now exact the same as before. Before a tool lookup was needed, calling a utility function, calling AccessControl. [jensens]

Project resources

Learn about Plone

Nominations Open for Plone Foundation Board of Directors

Posted by PLONE.ORG on August 29, 2018 03:55 PM

If you have an interest in helping the governance of Plone, and particularly the energy and time to pitch in, please consider nominating yourself to serve on the Plone Foundation board of directors for 2018-2019.

Nomination Process

  1. Log in on plone.org and go here: https://plone.org/foundation/meetings/membership/2018-membership-meeting/nominations
  2. Add a page there with your name in the title.
  3. For the body, discuss:
    • Who you are
    • Why you're interested
    • What you think you can add to the Plone Foundation
    • Most importantly, the name(s) of one or more Plone Foundation members who "second" your nomination
  4. Once ready, click "submit" in the workflow drop-down menu to get a reviewer to look at your nomination.
  5. Nominations will be accepted until October 31 2018, 23.59, UTC. The election will be conducted in conjunction with the annual meeting, which will take place in Tokyo, Japan at the Plone Conference 2018. All active members of the Plone Foundation will be eligible to vote.

About Board Membership

The Plone Foundation is a not-for-profit, public-benefit corporation with the mission to "promote and protect Plone". That has meant that the board is involved in:

  • protecting the trademark, copyrights and other intellectual property, including considering licensing and usage issues;
  • hiring the release manager;
  • working with various committees, including marketing and membership;
  • handling "other stuff in the community" as needed, e.g. helping craft policy on plone.org and plone.com about commercial listings
  • but not: directing Plone development. The board facilitates, but does not direct, the development of Plone itself.

While there's lots of work that happens online, much of the critical business of the board is conducted during video meetings every two weeks — typically, board meetings last about an hour to 90 minutes though occasionally they can run over to handle time-critical issues.  Please consider whether this fits your schedule, since missing more than an occasional meeting severely limits the ability of the board to reach quorum and conduct business.

Historically, board meetings have been organized to occur during daytime hours in America and evening hours in Europe, currently at Thursday nights, 19.00 UTC in northern hemisphere summer and 20.00 UTC in northern hemisphere winter. That can always change with new board members.

In addition, there is a board mailing list (private), where we discuss things in addition to the meetings.

This is a working board. Be ready to regularly take on and complete responsibilities for board business.

The board writes no code and makes no development decisions. It is much more concerned with marketing, budgets, fundraising, community process and intellectual property considerations.

You do not need to be a Foundation member to serve on the board (in fact, board leadership is an excellent way to become a Foundation member). All you need is to get an active Foundation member to second your nomination.

The Plone Foundation is interested in broadening the diversity of our leadership, with regards to gender, ethnicity, and geography.

If you have questions about the nomination process, contact the board: board@plone.org

Plone Conference 2018: Registration and Call for Talks are now open

Posted by PLONE.ORG on August 27, 2018 01:27 PM

See the Plone Conference 2018 website to register and to submit your talk proposals!

Visit 2018.ploneconf.org

The conference will be held in Tokyo, Japan, from November 5-11, 2018.

It will include outstanding training classes, keynotes and presentations, and will be followed by two days of collaborative sprints.

Join the Plone and Python web communities at this annual event, not to be missed!

For more news about the conference, sign up for the announcements list at 2017.ploneconf.org

 Plone Conference 2018 logo

 Photo credit: Eric Montfort, "Tokyo"

Auch bei Vodafone sind nur Sicherheitsdilletanten am Werk

Posted by Andreas Jung/ZOPYX on August 24, 2018 03:35 PM
Offenbar hat auch bei großen Konzernen niemand Ahnung von Security oder man ist ignorant. Wieviele Datenlecks, wieviele Hacks müssen noch passieren bis man wach und sensitiv wird?

Successful Beethoven Sprint in Bonn

Posted by PLONE.ORG on August 23, 2018 05:12 PM

During this three-day sprint at the office of kitconcept in Bonn we finished the second iteration of the new Pastanaga editor for Plone-React. The new editor is based on a tiles implementation for plone.restapi that we built during the sprint. The user can add, edit, and delete text, image, and video tiles and arrange them via drag-and-drop.

Group photo of the Beethoven Sprint 2018

In addition to the new tiles endpoint, the Plone.restapi team worked hard on multiple enhancements that resulted in four Plone.restapi releases and two Plone.rest releases during and right after the sprint. Plone.restapi and Plone.rest now work with Plone 5.2, which uses Python 3. We also ported the JSON field type from Guillotina to plone.schema and released it with plone.schema 1.2.0.

In addition to working on the new Pastanaga editor for Plone-React, we significantly enhanced and polished Plone-React, resulting in the Plone-React 0.7.0 release.

We made Plone-React work with a Guillotina backend and discussed the roadmap to Plone 6.

The sprint was kindly sponsored by the Plone Foundation, the German Python Software Foundation (PySV), The Python Software FoundationCodeSyntaxGanzgraphkitconcept GmbH, and Iskra.

Full daily sprint reports can be found here: day oneday two, and day three.

Announcing the Plone Conference 2019 selection process

Posted by PLONE.ORG on August 18, 2018 12:45 PM

With Plone Conference 2018 drawing near, it is time to begin planning for our next conference in 2019.The annual Plone Conference brings together users, integrators, developers, designers, and other interested folk from throughout the world for a week of training, talks, and sprinting. Plone conferences are also an expression of community spirit: they are organized by a company, user group, or other entity with ties to and a history with the Plone community and are in essence not-for-profit events.

The Plone Foundation is soliciting proposals to host the 2019 Plone Conference. The selection process this year begins in time to allow for final selection of the conference venue during this year's Conference. The extended timeline allows groups and organizations interested in hosting the 2019 Plone Conference (or beyond) to work with the Tokyo team for hands on experience during this year's conference.

Let's revisit where we've been so we can determine where we might want to go: we've traveled the world from New Orleans, Louisiana, USA for the first Plone Conference to:

  • Vienna, Austria
  • Seattle, Washington, USA
  • Naples, Italy
  • Washington, D.C., USA
  • Budapest, Hungary
  • Bristol, UK
  • San Francisco, CA, USA
  • Arnhem, Netherlands
  • Brasilia, Brazil
  • Bucharest, Romania
  • Boston, MA, USA
  • Barcelona, Catalunya, Spain

and this year to Tokyo, the capital of Japan. But, there are many places yet to explore! If you have a place in mind, don't be shy: submit a proposal!

The Plone Foundation will accept proposals beginning September 1 through October 15, 2018.

The Foundation Board of Directors will review proposals and open those that are viable for voting by the Foundation membership between November 1–5, 2018. The winning proposal will be announced at the end of Plone Conference 2018 in Tokyo.

Everything you need to know to submit a proposal, including the full schedule for the process and in-depth requirements for hosting, is outlined in the official Plone Conference 2019: Call for Proposals.

On behalf of the entire Plone community, we look forward to your conference proposals!

TestTheDocs Sprint

Posted by TestTheDocs on August 01, 2018 04:32 AM
Sprinting Time Join us in Szeged the sun city of Southern-Hungary, or online from your favorite spot!

Pastanaga Editor Status Report - Plone Beethoven Sprint 2018

Posted by kitconcept GmbH on July 20, 2018 11:24 AM

During the Plone Beethoven Sprint in Bonn, we worked hard on creating a first version of a new content editor for Plone-React.

Here is a short demo of what the editor looks like right now:

Demo of the Pastanaga Editor with tiles

We already had a first implementation based on DraftJS that allows inline styles (e.g. bold, italic), block styles (headlines, (un)ordered lists), and links to remote URLs.

The new version of the editor is based on a “tiles” backend, that is build by Victor Fernandez de Alba during the sprint and released with plone.restapi 3.2.0.

This allows us adding more complex content elements such as images, videos, and in the future more complex layout elements.

With the new backend in place, Rob Gietema went ahead and implemented the basic editor that Albert Casado designed as part of the new Pastanaga UI for Plone.

pastanaga editor Mobile Pastanaga Editor design by Albert Casado

The user can type in the title, description and the text content of the document without worrying about form fields or be distracted by tabs and fieldsets.

In addition to the standard text editing it is now possible to add an image tile that can be placed on the left or right side, on the center of the page or in full page width.

Rob also added a YouTube tile that allows the editor to add a YouTube video URL and then displays the video within the editor and the page view.

Text, image, and video tiles can be added to a page. They can be deleted and moved up and down to change the order of the elements.

Next Steps

The new editor is a great accomplishment. The tiles endpoint in plone.restapi allows us to further enhance the current version of the editor with more advanced layout variants and tiles.

We plan to continue with our iterative and agile approach of building a useful, fully functional version of the editor with each step, that allows Plone companies to use the editor and Plone-React today in their client projects.

The next steps are polishing the editor and the existing tiles. Work out some UX issues that we found when working with the editor and building more advanced tiles.

Stay tuned for more news and features after the Costa Brava sprint…

IMIO wins European Commission prize, donates to the Plone Foundation

Posted by PLONE.ORG on July 06, 2018 06:53 PM

imio-award2.jpeg

Award and Donation

On March 29, 2018, the European Commission awarded IMIO the €15,000 first prize in the local government category of the Sharing and Reuse Award.

IMIO has generously donated €10,000 of the prize money to the Plone Foundation.

The Sharing and Reuse Award, for interoperable solutions for public administration, businesses and citizens in Europe, recognizes government agencies that have set up and shared IT solutions with broad potential for reuse.

Of the 16 nominated projects, 10 (including IMIO's) are open source.

Governments are able to realize substantial savings and efficiencies when they reuse "cross-functional" components such as authentication systems and electronic invoicing. 

in Sorrento, Italy

IMIO's Statement on Plone

"If IMIO has won the Share and Reuse Award of the European Commission, it is due not only to the support and unconditional collaboration of its members, but also to the innovative aspects of its approach, be it organizational or technical.

This latter element, namely "Plone", is often unknown because the close relationship between IMIO and local governments masks the activity of a host of other private and public actors who created this technology that is the basis for most of IMIO's tools.

However, without them, IMIO would probably not have reached this technological maturity and nor have been able to respond as effectively to the needs of local authorities. Given its limited capabilities, IMIO has not been able to contribute directly to Plone core software. The software is indeed "free": no funds in the form of license fees were spent on this technology. Nevertheless, IMIO contributed to the community in the form of our team's services during international sprints, creation of generic modules, community support (contributions to forums), presentations at conferences, and evangelizing, either directly or through our subcontractors.

The prize money donated by IMIO is a token of appreciation for the hard work of the Plone community and its Foundation, with which we have worked for 11 years. The €10,000 donation will allow the Foundation to continue supporting its activities."

About the Plone Foundation

Plone is 100% funded by sponsorships from forward-thinking organizations and individuals. With the help of these financial contributions, the Plone project has continued to thrive and maintain an astonishing level of activity and innovation for 17 years.

The mission of the Plone Foundation is to protect and promote Plone. Foundation expenses are primarily related to:

  • an ambitious support programme for organizing and attending strategic sprints, conferences and marketing events
  • paying the Plone release manager a travel stipend to evangelize, plus a small stipend for major and minor software releases
  • registration of Plone’s trademark in countries around the world
  • legal work to secure Plone’s intellectual property in the Foundation’s “software conservancy”
  • the development and production of marketing materials
  • provisioning servers for our web sites, community forum, testing and continuous integration processes
  • a stipend to support the Plone marketing/communications team lead's attendance at Plone and related events

In order to continue this work, the Plone Foundation relies on sponsorships to create and sustain a reliable stream of income.

In recent years, we have aggressively supported strategic sprints, which are designated by the Framework Team as being significantly important to the continuing development of Plone. These include projects such as the headless CMS initiative, the porting of Plone to Python 3, the new Pastanaga UI, the integration of Angular and React JavaScript frameworks, and the REST API. Each of these represents important avenues for Plone's future growth, and these have been the primary expenditures of the Foundation, funded by vital sponsorships.

For further information:

Plone Beethoven Sprint 2018 - Sprint Report - Day 3

Posted by kitconcept GmbH on June 30, 2018 06:24 PM
Report of the third day of the Plone Beethoven Sprint 2018 in Bonn, Germany.

The Beethoven Sprint was a “strategic” sprint that took place June 21-25 at the kitconcept office in Bonn, Germany. The focus of the sprint was to work on the Pastanaga editor, Plone-React, plone.restapi, and Guillotina. This is the report of the third day of the sprint.

day3 wrap up all Day three wrap-up meeting at the kitconcept office

UTC DateTime Discussion

After having breakfast and doing a quick stand-up, we started a discussion about how to store datetimes in Plone. This was a discussion that came up when we implemented plone.restapi. Though, this question is more related to plone.app.event. As seen many times in the Plone community, when it comes to complex technical matters, offline discussions work way better than discussing issues back and forth via github comments.

Since we had all the experts at the sprint, we decided to schedule a discussion to solve the issue. In the end we reached an agreement that dates should be stored as UTC and that the local time zone should be stored separately in an additional field, for instance to calculate recurring dates.

plone.restapi

Thomas Buchberger worked on the workflow endpoint and added a feature to change the workflow state recursively, to set the effective and expiration date and to allow workflow comment.

Lukas Graf finished his work on implementing redirects in plone.rest based on plone.app.redirector. The redirects do not only work for simple GET requests but also redirects POST, PATCH, und DELETE requests.

Sune Brøndum Wøller finished the portlets endpoint for plone.restapi.

Mikel Larreategi made plone.restapi return cachable image resources to improve the caching of images in plone.restapi.

Roel finally wrote a PLIP for the IPloneSiteRoot and fixed edge cases and a few bugs in the current implementation.

Plone-React

Victor completed his work on plone.schema, allowing to use a JSON field in Plone that validates the JSON structure within the field and also allows through-the-web editing. He also finished the backend behavior for tiles and blocks to store the tiles and the tiles layout on a Plone content object.

Eric Steele added a backend implementation for the add-ons control panel that he wrote for Plone-React. Eric presented a fully working add-ons control panel that allows to install add-ons via Plone-React by the end of the day.

day3 wrap up eric add ons control panel Plone-React add-ons control panel presented by Eric Steele

David completed his work on the vocabularies endpoint by implementing the frontend widgets that rely on those vocabularies to make sure the endpoint serves its purpose.

Davi Lima and Victor continued to work on the override mechanism for Plone-React (JBOT) to customize widgets and views in a config.js file using a babel plugin.

Pastanaga Editor

day3 wrap up rob editor video Rob presenting the new video tile for the Pastanaga editor

As always, Rob did not give us any time to breath and added a YouTube video tile to the Pastanaga Editor, that shows a YouTube video in the editor itself and on the content view.

Of course that wasn’t enough for a day, so he fixed a few other smaller issues and worked on exposing the sitemap.xml via Plone-React.

Dinner and some late night hacking

Dinner Hans im Glueck Dinner at “Hans im Glück”

Rob might not admit it, but after claiming he could fix all those issues mentioned above in one day, he continued to work on the sitemap during our dinner at a local burger restaurant in the inner part of the city. We’ve been there ourselves, so we didn’t mind, while we were enjoying our tasteful cocktails. After a good meal, some drinks, and having to say good by to Eric, we went back to the office.

We said good bye again to our fantastic Dutch Plone-React team Rob and Roel and then went on to hack a little bit more on the stuff we were (and still are) so enthousiastic about.

Plone Beethoven Sprint 2018 - Sprint Report - Day 2

Posted by kitconcept GmbH on June 29, 2018 06:24 PM
Report of the second day of the Plone Beethoven Sprint 2018 in Bonn, Germany.

The Beethoven Sprint was a “strategic” sprint that took place June 21-25 at the kitconcept office in Bonn, Germany. The focus of the sprint was to work on the Pastanaga editor, Plone-React, plone.restapi, and Guillotina. This is the report of the second day of the sprint.

Guillotina

After having breakfast at the office, we started the day with a stand-up/wrap-up meeting with Ramon Navarro Bosch presenting Plone-React running on Guillotina. The user can add new content, edit existing content, and browse the content with Plone-React and Guillotina, which is a huge acomplishment and a promise for the future of Plone. After the stand-up Ramon had to say good bye to the other sprinters and head home to Barcelona.

Pastanaga Editor

Rob Gietema continued his work on the Pastanaga editor. At the end of the day, he was able to present a working version where the user can add a title and a description tile as well as a text tile with inline styles such as bold, italic, headlines, links, and lists.

IMG 6754 6b052ccf7a9ed291599dc6a7047a71cc Pastanaga Editor with basic text editing

Rob also added an image tile that allows you to upload images. The uploaded images then can be aligned left, right, in the middle and in full width or just be deleted.

IMG 6756 abcbc897d406fc882d25dca889402298 Image tile that can be aligned left, right, center and displayed with full width.

Victor Fernandez de Alba finished his work on the tiles backend for the Pastanaga editor (https://github.com/plone/plone.restapi/pull/547), so Rob could present a fully functional editor that actually stores the content (text and images) in Plone tiles.

Plone-React

Eric Steele continued his work on the add-on control panel and was able to present a first version at the wrap-up at the end of the day.

IMG 6758 144f319424d81d358e6410f2aea3d41f Add-ons control panel written in React by Eric Steele

Carsten Senger worked on implementing the users and groups control panel in Plone-React which now allows to add and delete users.

Victor worked on being able to use Plone-React as a library that allows developers to override React components. The system is supposed to work the same way as JBOT (just-a-bunch-of-templates) works in Plone.

Johannes Raggam and Andrea Cecchi continued their work on the reference widget and were able to present a first prototype. We agreed that we have to put some effort into the UX/UI of that widget, but that was beyond the scope of a sprint and requires the help of a UX specialist.

plone.restapi

Thomas Buchberger finished the pull request for the object create order and started to to work on the workflow endpoint enhancements.

David Glick worked on enhancing the vocabularies endpoint with batching and filtering.

Sune Brøndum Wøller fixed some nasty bugs with time freezing and transactions errors in the Plone 5.2 of plone.restapi (e.g. https://github.com/plone/plone.restapi/pull/562, https://github.com/plone/plone.restapi/pull/561).

Lukas Graf continued his work on the translations of the REST response data, fixed a few bugs and wrote a script that tests the Sphinx-based documentation for warnings and errors and fails the Travis build if there is a problem.

Mikel Larreategi finished the pull request for the history endpoint and documented the Accept-Language headers in plone.restapi. He also started working on making the image scales that plone.restapi returns cachable.

Roel Bruggink continued his quest on the folderish site root and fixed errors in CMFPlone, plone.app.content, and in plone.app.dexterity.

We were able to release plone.rest 1.1.1 and plone.restapi 2.1.0 as well as plone.schema 1.2.0 with a JSON field that is required for the tiles endpoint in plone.restapi.

Hacking Night

Germany was playing at the soccer world cup in the evening, so we decided to order Pizza for dinner and then split up between people that wanted to code and people that wanted to watch the game. As you might have guessed the latter group was rather small. We ended up in a bar because the public viewing was already full. Though I guess the sprinters got a taste of the German soccer culture. After the game we went back to the office to join forces with the others again to wrap up the day and hack the night away.

PloneGov growing in the Basque Country

Posted by CodeSyntax on June 29, 2018 08:11 AM
PloneGov is an international initiative with the goal of getting a powerful on-line eGovernment tool. Most eGovernement needs and requirements are similar and PloneGov wants to satisfy them in a effective and efficient way thanks to its open source project. CodeSyntax is part of PloneGov thanks to its UdalPlone initiative.

Plone Beethoven Sprint 2018 - Sprint Report - Day 1

Posted by kitconcept GmbH on June 28, 2018 06:24 PM
Report of the first day of the Plone Beethoven Sprint 2018 in Bonn, Germany

The Beethoven Sprint was a “strategic” sprint that took place June 21-25 at the kitconcept office in Bonn, Germany. The focus of the sprint was to work on the Pastanaga editor, Plone-React, plone.restapi, and Guillotina. This is the report of the first day of the sprint.

Stand-up

day1 standup Stand-up of the first day

We started the day with a stand-up meeting giving people a heads up on the current state of affairs of plone.restapi, plone-react, and Guillotina.

I started with plone.restapi, which is considered stable and battle tested in production.

Victor Fernandez de Alba then gave a brief introduction to our first Plone-React based project VHS-Ehrenamtsportal, that we successfully shipped to our client a few weeks ago and run without any issues since then.

IMG 4031 565f311943de0d1be7694a4b1c0e79a8 Victor introducing VHS Ehrenamtsportal

After this, Rob Gietema gave a short introduction to the current state of Plone-React.

Last but not least, Ramon Navarro Bosch presented “Guillotina”, an async server written in Python with a Cockroach DB / ElasticSearch backend that adopts some of the core concepts of Zope and Plone.

With a group of 15 sprinters, we decided to split up in four different groups for the main sprint topic. Thomas Buchberger led the plone.restapi group, Rob the Plone-React group, Victor the Pastanaga Editor group, and Ramon the Guillotina group.

Pastanaga Editor

22 06 18 4 Rob Gietema going though the different approaches we discussed during the Tiles planning meeting

Right after the stand-up, we had a longer discussion about the “tiles” endpoint in plone.restapi and the editor implementation in Plone-React. We already reached an agreement of the API design at the Plone-React sprint a few months ago. Though, it turned out that implementing that on top of the existing plone.tiles implementation was harder than we thought and we did not anticipated all the problems that came along with that.

We decided to keep the API design and to write a simple Dexterity behavior that adds a “tiles_layout” field for the layout information and a “tiles” field that holds the actual data of the tiles. Ramon already wrote a JSON-field in the Guillotina code that we decided to re-use for our implementation.

Rob Gietema already wrote a first prototype of the new editor at the Plone-React sprint and he was waiting for the backend code to be implemented. While we were working on the backend implementation, he focused on the prototype.

plone.restapi

22 06 18 24 Planning board with plone.restapi issues

Lukas Graf added missing translations in plone.restapi responses and simplified the test setup and did some clean up on the code (https://github.com/plone/plone.restapi/pull/545).

Thomas Buchberger worked on fixing the plone.restapi object creation logic to behave more like through-the-web object creation (https://github.com/plone/plone.restapi/pull/549) and separated the object creation from firing the events.

Sune Brøndum Wøller cleaned up and upgraded multiple Plone versions in plone.restapi (https://github.com/plone/plone.restapi/pull/537), worked on portlets and portletmanager serialization and fixed a ReadConflictError in the plone.restapi tests for the documentation that was bugging us for quite some time (https://github.com/plone/plone.restapi/pull/543).

David Glick and me worked on Zope 4 compatibility for plone.rest and plone.restapi. It turned out that one of my fixes on plone.rest was already sufficient and that the test failures in plone.restapi were caused by a plone.testing issue that David found and fixed (https://github.com/plone/plone.testing/pull/49).

Roel Bruggink continued his efforts on turning the Plone site root into a Dexterity object. He worked on making the IPloneSiteRoot interface / content object behave more like content and he attached behaviours to the IPloneSiteRoot to edit them without relying on a default page.

Mikel Larreategi finished his work on the translation of the content-type names on the @types endpoint (https://github.com/plone/plone.restapi/pull/540) and the translation of the actions and workflow state and transitions on the @history endpoint (https://github.com/plone/plone.restapi/pull/541).

Plone-React

Rob gave an introduction to the Plone-React codebase and explained the basic concepts and libraries that we use in Plone-React.

Eric Steele started to work on creating the add-ons control panel in React. Carsten Senger took over the work that Rob started before the sprint to bring the users and groups control panel to Plone-React.

Andrea Cecchi. and Johannes Raggam worked on the React-based widget for references in Plone.

Guillotina

After giving an introduction to Guillotina to the sprinters, Ramon went ahead and made Plone-React work on top of Guillotina. Since it origins, plone.restapi and Guillotina were supposed to share the same API to allow us to switch the backend for our new frontend at some point in the future. Ramon was also heavily involved in the API design of plone.restapi and wrote the first version of plone.rest before he decided to invent Guillotina. Over the time both APIs differed because of differences in the underlying implementation.

Ramon and Rob worked on this and by the end of the day they could present a working version that allows basic content editing and browsing.

Roadmap / Plone 6

We had a hangout with Philip Bauer from Munich who is leading the efforts to migrate Plone to Python 3 and Zope 4.

Hangout with Philip Bauer

Philip and I already had a longer discussion about a possible roadmap for Plone 6 and how to bring our efforts on the frontend together with the efforts of the group that works on Python 3 and Zope 4. We discussed the outlined roadmap and the upcoming sprints where we plan the implementation of the roadmap.

Lunch / Dinner / Evening

We went to have lunch in a Vegan cafe (Black Veg) and to a traditional brewery in the old part of the town for dinner (Bierhaus Machold). After dinner and a few drinks, we decided to head back to the office for some late night hacking. Not without stopping by at a local “Kiosk” for some customary buying of beverages for the evening.

World Plone Day 2014: manufacturing internationalization strategy

Posted by CodeSyntax on June 22, 2018 08:47 AM
World Plone Day was last wednesday, april the 30th, and as in previous occasions, we did celebrate it at CodeSyntax's offices, with some customers and Plone users of the Basque Country.

Presenting Buildout at PySS 14

Posted by CodeSyntax on May 29, 2018 01:10 PM
Buildout is a tool we use in all of the development and deployments of our applications, and we have given a talk about it at PySS 14.

Porting Plone to Python 3

Posted by Starzel.de on May 23, 2018 07:26 PM

Since I wrote the proposal to Port Plone to Python 3, so much has happened that a status update is needed.

Let's step back a little: The first steps towards Python 3 were taken during the sprint at the Plone conference in Barcelona. The epic PLIP to update to Zope 4 was merged and we started porting individual packages to Python 3 without being able to run any tests. With the help of sixer and python-modernize we tried to get the most obvious import and syntax issues out of the way. That work was continued at the Alpine City Sprint in Innsbruck and in the end, we treated more than 150 packages like this. Some of this work is documented in the Plone tracker.

Along the way I wrote a best-practice guide on how to port Plone packages this way.

As the PLIP states, there are several approaches to porting Plone to Python 3:

  1. Migrate and test packages that have no dependency to CMFPlone and test them in Python 2 and 3.
  2. Prepare packages for Python 3 without being able to test them in Python 3.
  3. Start up Plone on Python 3 and fix whatever breaks. When start-up works, create a Plone Site and again, fix whatever breaks.
  4. Port plone.testing and plone.app.testing to Python 3 and start running tests. Fix what breaks during the setup of the layers.
  5. Run the tests with Python 3 and fix all broken tests.

At the sprint in Innbruck I started with #3 and kept going after the sprint until I was able to create an instance. At the Plone Tagung in Berlin I was able to demo the creation of a site but nothing was rendered yet.

After that, I kept going and I was finally able to create a site and manage content, which is what a CMS is about. It looked a bit raw but I was able to add and edit some content - yay!

early screenshot of plone with python 3

Work continued at an unsteady pace, and with important contributions from Michael Howitz, Alessandro Pisa and David Glick, things started to get better. Even the theme and js started working. Slowly broken features became the exception, not the rule.

Last week at the Zope 4 Welcome Sprint in Halle we removed all feature blockers that we had found so far. Plone with Python 3 looks and feels like it used to in Python 2. During the sprint there was also a lot of progress on:

  • the wsgi setup
  • logging and tracebacks when using wsgi
  • porting plone.testing
  • a new theme for the ZMI
  • beta releases for many packages.

There was also some progress on the difficult issue of database migrations. It seems like zodbupdate is the best tool to do that but there is probably a lot of work ahead.

Read more about the sprints outcome in the blogpost by Michael Howitz.

There is a Jenkins job and as of today, it is running (not passing) tests for all packages (except Archetypes) with Python 3.

At the moment we run 6594 tests from 115 packages with 257 failures and 315 errors - not bad when you keep in mind that we still focus on features, not on tests. Tests for a couple of packages are green, including plone.api which is great since plone.api uses most of our stack one way or another. Jenkins runs every three hours and here you can see the progress over time.

Here is a screenshot from today that shows Plone doing some pretty non-trivial things: Editing a recurring Event and also Richtext with Images and Links:

screenshot today

Next steps:

  • Create a demo site running on Python 3 that people can use to find broken features. This will happen at the Plonator Sprint in Munich.
  • Fix all tests that look like they fail because of a broken feature.
  • Fix all remaining tests to pass in Python 2 and Python 3. Since there are a gazillion doctests that will take some time.
  • Port plone.app.robotframework to python 3 and fix robottests.
  • Experiment with porting a ZODB with Plone to Python 3.

If you want to participate you can simply look at the failing tests and start fixing them. You can also try to fix one of the open issues.
The setup of a coredev environment with Python 3 is really simple and documented.

Rural Sprinting: Two New Plone Add-ons and Progress on Python 3

Posted by Jazkarta Blog on May 17, 2018 06:56 PM

Jazkarta Team

We returned to my house in rural Massachusetts for our annual sprint this year and had a great time sampling local beers and ciders,

Buying supplies for the sprint

eating big meals together,

Making pizzas at the sprint

admiring the Milky Way,

Nightime view at Sally's house

and working on some fun projects. Normally we work remotely – everyone in their own home office, spread across 3 countries and 2 continents. But it’s really nice to get together in person, and we try to do it once a year.

Here’s what we worked on.

Witek and Alec created 2 new Plone add-ons and released them to Pypi.

  • jazkarta.abtesttile Provides a new Mosaic tile type that can be used for A/B testing parts of a page layout. Managers can define 2 rich text fields on one tile, and a ratio for how often each should be displayed (for example, 70%/30% or 50%/50%). Plone will randomly show users one field or the other in that ratio. Managers can optionally specify Javascript snippets for use in analytics tracking. Managers can also optionally enable a query string variable, which is added in the rendered HTML to links in the rich text fields. This will indicate whether option A’s or B’s rich text was the source of a page visit. A custom permission allows usage of the add-on to be restricted to privileged users.
  • collective.siteimprove Provides integration with siteimprove.com. There is a control panel for requesting and saving a siteimprove.com token that registers the domain with Siteimprove. (You must first sign up for a Siteimprove account.) A Siteimprove button is shown to authorized users on all default views. Publicly visible content shows authorized users a Siteimprove recheck action in the Plone toolbar that checks the individual page. This add-on is essentially done but untested since we have not yet met with the Siteimprove sales person who will provide an account for us to test with. We hope to be able to do that next week.

David, Matthew and Jesse decided to contribute to the ongoing effort to port Plone – Python’s open source enterprise CMS – to Python 3. They were in the porting groove because we recently ported our Dallinger project to Python 3. They made some good progress:

  • David made it possible to run Plone tests without including Archetypes, so that developers can run the tests in Python 3 without worrying about porting Plone’s old content type system. After that he investigated why cookies on Python 3 are preventing logins from staying logged in. He traced it to PAS and create a branch with all tests passing on both Python 2 and 3. Hopefully it will get merged during the Halle sprint this week.
  • Matthew found and fixed some Chameleon problems. This included an old error that had nothing to do with Python 3 where Chameleon puts spurious context into error messages. He also converted plone.namedfile to Python 3 and fixed some tests in plone.protect (CSRF protection). His work resulted in a fairly large pull request, which was approved and merged.
  • Jesse got all tests running for plone.app.workflow and made a bit of progress on the test failures for plone.app.dexterity.

Thanks to all these fixes, we got to the point of being able to save a Plone page on Python 3 and it “sort of” works. (At least it didn’t give an error!)

David also did some evaluation of plone-react with Nate. This is a React-based front-end for Plone that is built on plone.restapi. It’s in early stages of development but looks promising. In the process, Nate ran into a bug in Plone’s unified installer, tested it in Plone 5.1.2 and and filed a ticket for it. Back to his Plone roots!

Plone Welcomes Students for Google Summer of Code 2018

Posted by PLONE.ORG on May 03, 2018 08:07 PM

For 2018, the Plone Foundation has been granted four Google Summer of Code student project slots.

Google Summer of Code is a global program focused on bringing more student developers into open source software development. Students work with an open source organization on a 3 month programming project during their break from school.

After careful selection of the many project proposals presented, we are pleased to announce that the following students will be working on various aspects of Plone:

  • Shriyansh Agrawal (IFTTT plugin for Plone)
  • Akshay (Command Line Plone Tools)
  • Ajay NS (GatsbyJS Integration with Plone)
  • Nilesh Gulia (Create-React-App for Plone-React)

Read more about their projects

To our new students: welcome to the Plone community – we wish you a great learning experience over the coming summer. Congratulations!

Mentoring these students will be:

  • Timo Stollenwerk (former Plone GSoC student!)
  • Rob Gietema
  • Victor Fernandez de Alba
  • Asko Soukka
  • Andrea Cecchi
  • Paul Roeland
  • Alexander Loechel
  • Maik Derstappen
  • Encolpe Degoute
  • Nejc Zupan (former Plone GSoC student!)
  • Sally Kleinfeldt
  • T. Kim Nguyen

To our mentors: thank you for introducing new developers to open source and our community!

Plone-React Sprint Bonn 2018

Posted by kitconcept GmbH on April 23, 2018 04:46 PM

From March 12th to 15th, we hosted a Plone-React sprint at our office in Bonn. The main goal of this three day sprint was to contribute back the work we did for a recent client project. We also planned to upgrade Plone-React to React 16 and improving the error handling.

A small and very focused sprint with the core contributors of Plone-React was the perfect opportunity to do so. Participants were Rob Gietema, Roel Bruggink, Asko Soukka, Victor Fernandez de Alba, Carsten Senger and me.

Sprint Day 1

We started the first day of the sprint by doing a quick stand up and planning meeting. We used the days before the sprint to outline our main goals and objectives, so we could get started right away.

IMG 2572 725ac0c3d0acdf94e31efebb6c23f318 Planning board after the stand up

React 16

On the first day Rob and Victor worked on upgrading plone-react to use React 16 and Webpack 4.

We postponed the upgrade to React router version 4, because some of the dependencies that we need, are still in alpha phase and staying with the old version does not hurt right now.

IMG 2576 4ffa46024dfcc49456d639f9059e5581 Card to upgrade to React 16

Error Boundaries

The main reason why we were eager to upgrade to React 16, was a new feature called “error boundaries”, which allows catching errors in React components and handle them gracefully, without failing the entire app.

Victor implemented error boundaries for client and server side components and for the Redux middleware. It is also possible to pass those errors to Sentry for aggregation and further error handling.

Since we are about to launch our first Plone-React-based project in the next weeks, this was something that was especially important for us.

IMG 2580 d0283e182b2c866ad960a1d3c3a44de3 Error message in plone-react

Token Expiration Middleware

Rob added a token expiration middleware to plone-react that improves the handling of JWT auth token in Plone-React. Working with plone-react on a daily basis revealed a few edge cases where the current authentication failed. The new middleware solves those issues.

plone.restapi

Roel and Carsten focused on plone.restapi. Even though this was not planned to be a major topic for the sprint, we had to fix some issues that were causing troubles in the Plone-React frontend.

Carsten worked on fixing an issue that was preventing to allow the API consumer to reset field values to “None”.

Roel started to work on exposing widgets with tagged values via plone.restapi. This is necessary to support autocomplete or reference widgets in Plone-React.

Test Setup

Asko, who joined the sprint remotely, worked on a brand new end-to-end testing setup with Robot Framework. Because this would be way to easy for a super-smart guy like Asko, he decided to wrap the components into a Docker containers, to make it easier for non-Python devs to set it up. Making Plone PIP-installable is also something that we want to have for quite a while. Asko decided to also mix that in. To make this fun, he decided to add support for Jupyter notebook, which makes it super easy to write Robot Framework tests.

IMG 2579 7764cc8777f896ebb057573f308488c1 Hangout with Asko

I started the day discussing the acceptance testing setup with Asko, quietly listening to Rob and Victor and discussing the plone.restapi issues with Roel and Carsten.

Pastanaga Toolbar

During the first day, we started a discussion how plone.restapi could support a toolbar that automatically adapts to the permissions of the logged in users and shows only the actions that this particular user has permission for.

Before the sprint Victor worked on implementing the new super fancy adaptable Pastanaga toolbar, and we were eager to build a proper backend implementation for this.

Victor's tweet with a short demo of the new Pastanaga toolbar

Sprint Day 2

Create-React-App

On the second day of the sprint, Rob and Victor started to look into building a “create-react-app”-like functionality for Plone-React. create-react-app is a widely popular code skeleton generator by the React team at Facebook. It hides lots of complexity from the user (e.g. Webpack, libraries, configuration) and makes it easy to get started with React. This is super important for the adoption of Plone-React because it also allows to use Plone-React as a library and basis for custom client projects.

Rob and Victor created a proof-of-concept that kind of works but it became clear that this requires a lot more effort before this becomes ready-to-use.

Pastanaga Editor

We scheduled a time slot for a discussion and planning session about the new “Pastanaga editor” user experience. We already implemented basic text editing based on the DraftJS editor from Facebook. The current editor allows basic text editing as well as inline (italic, bold, etc.) and block styles (e.g. headlines, bullet points) and external links.

The next step on our agenda to make this editor based on Tiles to make it extendable and allow the user to add images, videos and other media objects.

We agreed on moving forward with an agile approach of building something useful step-by-step, making sure to build a fully functional and useful editor at any point of the development stage, rather than building the full Mosaic-like functionality at once.

Right after this was settled, we started to draft a tiles endpoint that would build the basis for the next iteration of the Pastanaga editor.

Toolbar

Rob started to work on implementing the context-aware toolbar. This was a challenge because the toolbar needs to adapt to the content that is shown in the main column. This means a component deep down the component hierarchy (content) needs to be able to update a component that lives outside of the DOM hierarchy of the parent component. Luckily for us, “Portals” in React support exactly that use case.

Victor worked on the flexbox styling of the toolbar as well as on optimizations of the Webpack configuration.

plone.restapi, Tiles, and Testing

Roel and Carsten continued to work on plone.restapi issues. Roel continued to work on the tagged values representations and Carsten finished the actions endpoint for the context-aware toolbar.

I followed our established documentation-first approach on plone.restapi and wrote the docs for the new tiles endpoint.

With the basic test setup already in place, Asko struggled with the Travis CI setup running different versions of Node and Python at the same time. Problem is that the ZEO version requires the latest Python 2.7.13 which is not shipped by default with Travis CI.

He also worked on a Docker compose option to support an option to run the API server. This would allow faster development of Robot Framework tests since the API server does not need to restart for each test (iteration).

Sprint Day Three

We started the last day of our sprint discussing the create-react-app use case we worked on the day before. Afterwards, we did a hangout with a student who is interested in working on this during the Google Summer of Code 2018.

IMG 2583 f900eaa919019cf508c8a20e6313de48 Hangout with a possible Google Summer of Code student

Right after this call, Asko gave me a tour of the new, Docker compose based, PIP-installable, Jupyter notebook enhanced test setup.

My head was still spinning from Asko’s amazing work, when Rob asked me for a minute to show me a prototype of the new Medium-like Pastanaga editor.

Summary

The sprint was extremely productive and fun. Having a small and dedicated group of developers with a clear goal and focus really worked well for us.

IMG 6425 1 Group photo in front of the kitconcept office

We contributed back all our re-usable code from our client project, upgraded Plone-React to React 16. We fixed some important issues in plone.restapi and build a super fancy test setup that will allow us to further improve the software quality of Plone-React.

We also laid the groundwork for the next important steps forward: Building the new Pastanaga editor with a tiles-based backend and allowing to use Plone-React as an extensible library with a “create-react-app”-like functionality.

We are already looking forward to the upcoming Beethoven Sprint and the Costa Brava sprint where we will continue to push Plone-React, Pastanaga and plone.restapi.

Plone 5.1 Has Been Released

Posted by PLONE.ORG on March 08, 2018 02:48 AM

As with all Plone minor version releases, 5.1 includes many bug fixes and under-the-hood improvements, with a modest selection of enhancements visible to users, editors, and site administrators.

Plone 5.1 contains these user-facing improvements:

  • Plone pages load faster now with better bundling of JavaScript and CSS resources
  • Plone’s built in search indexing is much faster and CPU-efficient
  • Plone supports the display of higher resolution HiDPI (“Retina”) images

Content editors and site administrators will find these enhancements:

  • Site administrators can create, edit, and manage portal actions (menu items and links) through a new control panel
  • Direct linking to groups’ details from the Sharing tab
  • New configuration registry control panel lets site administrators export, import, and add new records
  • Improved HTML filtering for preventing malicious HTML
  • Fine grained control over thumbnail images in portlets and display views
  • Support for image rotation metadata: the image will show up with the rotation that the photographer intended
  • The default order of search results is now configurable

Plone 5.1 also includes these developer-oriented changes:

  • Ability to conditionally import configuration registry records (by checking on the presence or not of specific features and Plone versions)
  • Removal of OpenID login support from Plone core
  • Removal of the old portal_quickinstaller (may be a breaking change for add-on developers)


To get Plone 5.1, please visit the downloads page

For help using or upgrading to Plone 5.1, see the support page

 

Photo by Airman 1st class Alex Echols

The Mountaineers Keep Climbing

Posted by Jazkarta Blog on February 22, 2018 04:40 PM

The Mountaineers' Website's New Look

Some non-profits fund raise for major technology upgrades, then breathe a sigh of relief and ignore technology until things start breaking. Rinse and repeat. Other non-profits avoid this feast-or-famine approach to technology spending by budgeting for continuous improvements. The Mountaineers is in the latter camp. In 2014 they completed a major technology upgrade and launched a new Plone+Salesforce website. Every year since, they have budgeted for significant improvements to their website and its Salesforce.com back end. They did this through modest support contracts with their technology partners (Jazkarta and Percolator Consulting), plus focused spikes of work that were done through a series of agile iterations. Over the last 3 years we have released enhanced versions of the website 10 times, with an overwhelming number of improvements – you can read all about that on their technology blog. These changes have allowed members, staff, volunteers, instructors, and the organization to get better and better at what they do.

The latest update is the biggest one yet. It has brought mountaineers.org and mountaineersbooks.org together into one, integrated website. The new and improved header and main navigation helps users know where they are and get to where they’re going. It looks great! We’re really proud to have helped bring it into the world.

 

Plone Selected for 2018 Google Summer of Code

Posted by PLONE.ORG on February 17, 2018 09:43 PM

Plone, the secure, enterprise-scale Python web content management system, is one of the organizations that have been selected for the 2018 Google Summer of Code (GSoC).

https://summerofcode.withgoogle.com/organizations/5685175707500544/

GSoC is a global program focused on bringing more student developers into open source software development. Students work with an open source organization on a 3 month programming project during their break from school.

Organizations are selected on the basis of their application, which takes the form of a proposal based either on a student’s own idea or one selected from a list compiled by Plone. Proposals are judged on their utility to Plone, depth of planning and likelihood to succeed.

Cris Ewing, the main organizer behind our GSoC 2017 programme, reprises his role as Plone’s GSoC administrator for 2018. Cris is a former Plone Foundation Board member and co-organizer of the 2016 Boston Plone Conference.

Student developers interested in submitting a GSoC proposal to the Plone Foundation should read the introductory information about Summer of Code at https://developers.google.com/open-source/gsoc/ and take a look at Plone’s ideas list at https://plone.org/gsoc. Applications open on March 12, 2018, but students are welcome to get in touch before then in our forum at community.plone.org for feedback on their ideas.

For more information

 

Three New Members Join the Plone Foundation

Posted by PLONE.ORG on February 17, 2018 02:23 PM

The Plone Foundation welcomes 3 new members after unanimous confirmation by the Foundation's Board of Directors on February 8, 2018. Membership in the Foundation is conferred for significant and enduring contributions to the Plone project and community. The Plone Foundation Membership Committee overwhelmingly recommended each applicant for their ongoing contributions to Plone.

Sven Strack

Sven Strack

Sven has given talks and training at several Plone conferences, was the organizer of the Stroopwafel sprint, the creator of Plone Docker images and of the UI installer. He is best known as Plone's documentation czar, having led the Documentation Team for years with verve. His most recent new responsibilities in the Admin and Infrastructure Team have already resulted in the move of critical Foundation servers to new, more reliable facilities.

David Bain

David Bain

David first presented at the 2008 Plone conference and hasn't stopped giving conference talks and training since then. He is a steadfast advocate for Plone, Python, and Jamaica, having organized Plone meetups and Python events in Jamaica, including PyCon Jamaica. David regularly engages with newcomers and old timers alike in our forum, and has been a key member of Plone's Google Summer of Code participation. David's continuously innovative spirit has resulted in new code and new ways of thinking, particularly in the areas of approachability and onboarding.

Franco Pellegrini

franco pellegrini

Franco first burst onto the public Plone scene with his mind-blowing creation, the PloneIDE. He has authored and contributed at least a dozen add-ons, including the popular collective.cover, is a member of the Framework Team, and a contributor to Plone core and the Mockup project. He has given talks and trainings at Plone symposia, conferences, and local events since 2010.

 

 

The Foundation is the trustee for Plone's intellectual property, works to protect and promote Plone, and has 78 active members.

Random News And Updates

Posted by TestTheDocs on February 11, 2018 12:32 PM
Docs We are happy to announce that docs.testthedocs.org is back ! For the moment there is not much to see, we are starting slowly to add more and more content ! Contribute Also, we are looking for contributions !!! We would like to see more content about testing, tools, tip and tricks, etc, etc !!! Future Planning We are playing with a new way of testing, building and deploying our docs.

Plone News Roundup, February 2018

Posted by PLONE.ORG on February 03, 2018 09:24 PM

Welcome to 2018! Plone has continued its flurry of activity since our wonderful conference in Barcelona last October...

Phew! If we missed something, please let us know!

RoboCon 2018 and Robot Framework Jupyter support

Posted by Asko Soukka on January 27, 2018 06:48 PM

It's already over a week since I got back home from the first Robot Framework conference ever – RoboCon 2018. It was a pleasure to be there, and I really feel privileged that I was accepted there as a speaker.

My RoboCon 2018

RoboCon 2018 was a single day conference about Robot Framework test automation ecosystem, held in the heart of Helsinki, Finland, on 18th of January 2018. The conference venue was quite if not completely full, so there must have been around 250 participants. The event was in English and had pretty good international participation. Yet, most of the participants came from Finland, where Robot Framework has become de-facto standard for test automation.

https://1.bp.blogspot.com/-tuaZhsT3Ves/WmwL97tsinI/AAAAAAAABPE/i-cRg8VQBNIO3E89SFY2Z9A4qmDvYAtSwCKgBGAs/s1600/LND_8D2D2079-6B5F-4DC1-A14C-6D30182A63FB.JPG

Pekka Klärck presenting the history, present and future of Robot Framework

RoboCon 2018 had only a single track, so that had to be packed to include something for everyone in its diverse audience. In addition, there was plenty of time and a separate space for networking with the other participants and conference sponsors. There was also organized social program before and after the conference, but unfortunately, I was unable to attend those at this time.

In my opinion the program was well balanced: The conference started with introductory talks, continued with variety of differnet case studies (Kone, Plone and Texas Instruments), and ended with more technical talks about specific Robot Framework addons (SeleniumLibrary, the most awesome new REST library and pabot). And in the middle of everything, there was my personal favorite: Ed Manlove's talk about building successful open source communities. My presentation was called Robot Framework in Plone CMS Project: a case study, story and some technical details, how Robot Framework got successfully adopted in distributed open source community behind Plone.

The most important part of this conference, of course, was getting a lot of Robot Framework users and developers to meet in the same place at the same time. After all, RoboCon 2018 was the first Robot Framework conference ever. My personal absolute highlight during the conference was meeting a former Plonista, Ed Manlove. He was the one who first introduced me to Selenium testing in San Francisco Plone Conference in 2011, and whom I had not seen after that. Until now. I really hope takes less than seven yeras to see him again...

https://3.bp.blogspot.com/-pHUpSJ725vQ/WmwL91OCKzI/AAAAAAAABPE/DA1nMNfVEPcG8NbGnn0V6FdsahlwK5vhwCKgBGAs/s1600/IMG_6460.JPG

Ed Manlove presenting his talk: The Importance of Open Source Communities

Jupyter kernel for Robot Framework

After the conference on Thursday came the single day conference sprint on Friday. And if the conference was a success, the sprint was even more so: the sprint venue, three large office rooms, was packed full of sprinters, many of them participating their first open source sprint (and got a good introduction to open source development from Ed).

https://4.bp.blogspot.com/-j8En_gW7h08/WmwL931Sz7I/AAAAAAAABPE/AEoDgF84z8YMIm7npuP831Fp4bhnY1aJwCKgBGAs/s1600/IMG_6482.JPG

The sprint facilities were provided by Eficode

Because I had to leave early in Friday, I had planned a very specific sprint goal for myself: a MVP Robot Framework kernel for Jupyter notebook.

Jupyter notebook (previously known as IPython notebook) is an open-source web application for creating and sharing documents that contain live code, equations, visualizations and narrative text. The architecture behind Jupyter notebook separates the notebook application from its language specific ”kernels” that are responsible for executing the code in notebooks. Syntax highlighting in notebooks, on the other hand, is provided by CodeMirrorproject for the interactive frontend, and Pygments for server side generated highlighting.

I'm happy to say that I made it. And the more I use it, the more confident I get on that Jupyter makes a great platform for learning also Robot Framework. And not only for learning by yourself, but also for sharing your notes with others.

Check my example notebooks to judge the kernel by yourself.

https://3.bp.blogspot.com/-M10Tz__nv9g/WmygIgebHLI/AAAAAAAABPc/3-_87CO3LBwvX3ID8STfCo6BdO1jnkiTgCKgBGAs/s1600/Screenshot-2018-1-27%2BNotebook%2Bon%2Bnbviewer.png

Please, note that while these examples are static renderings at http://nbviewer.jupyter.org/, they can be opened and interacted live in any running Jupyter notebook with the new kernel and required Python packages. See the repository for more details.

The main Jupyter Robot Framework support features shown in those examplesare:

  • Support for defining and executing Robot Framework test suite cumulatively step by step in successive notebook cells. The main limitation is that each cell should start with a test suite section header (settings, variables, keywords or test suites) even when the same header was already defined in some cell before.
  • HTML log and report files are linked below the executed cells containing the tests. Both files are actually bundled with the notebook in a way that sharing the notebook also shares the log and report files.
  • Images generated during test execution are shown below the executed cells generating the images. Similarly to HTML logs and reports also images are bundled with the notebook for sharing.
  • Support for %%python module LibaryName ”cell magic” to allow defining custom Robot Framework keyword libraries in Python in fly. Once thell cell with a Python library class definition is executed, it can be imported in a successive Robot Framework code cell.
  • Syntax highlighting. But, unfortunately, until the CodeMirror plugin derived from brackets-robotframework-project is accepted into upstream, it must be manually patched into CodeMirror version shipped with Jupyter notebook-distribution. (I have not yet submitted a pull for it.)
  • If the last keyword of the last test case in the executed cell returns JSON string, it is rendered as cell execution output. I added this quite specific feature to make it more fun to learn RESTinstance library with Jupyter (Output keyword of RESTinstance library returns JSON).

Obviously, while the current versions is already fully functional on Python 3, there's still a lot of work (QA, packaging and Python 2 support) left to polish the code for release. I'm looking forward to finish it during the spring.

Happy hacking! And hopefully see you in RoboCon next year – or whenever it is organized and I'll manage to participate it for the next time! :)

Pastanaga icon system

Posted by kitconcept GmbH on January 25, 2018 07:24 PM

The way we deal with icons in the web has evolved over the years. Images, images sprites, fonts, SVG, SVG sprites… I’ve been looking lately for the current best practice in order to include it in Pastanaga and I wanted to share with you my results. Please note that it’s not closed and I’m open to suggestions. PRs are welcome too!

Abandon font-based systems

It was clear to me that font-based icon systems are a no longer an option today. For several reasons:

  • The font is loaded every single time on the page, regardless if we use all the icons or none of them. This bloats the application size and forces an additional request (in the best case scenario).
  • An existing font is difficult to create, maintain and update. You can use some online (free) services to do that and even you can forge your custom icon font with them but it’s cumbersome and not practical.
  • Forces you to maintain a parallel CSS that maps the icon name with its actual character in the font (which is obscure). The font creation tool helps you with that, but…
  • Extending them with new icons is also complex, especially for newbies, and you need access to the source and reload the source in the same tool that was created.

only to name a few.

Time to move on: inlining SVG

The rise of SVG in modern web developing is for a good reason:

  • SVG is a vector format, so it looks great in HiDPI displays
  • It’s lightweight and portable, since it’s not a binary file
  • It can be styled (and animated) easily using CSS (provided they are inlined, not used with the <img /> tag
  • You can control it via JS

My initial feeling was that using a SVG sprite based system would be the best approach, but I soon was discouraged, after reading to Chris Coyier, CSSTricks: A Pretty Good SVG Icon System

All in to inlining SVGs, then.

So, we need an SVG icon system. Luckily for us, Pastanaga already has a complete set of icons based on SVG organized in one file per icon.

Goals

Our main goal is to provide inline SVGs in our applications, having in mind:

  • It should be performant and small in size
  • Only the used icons should be loaded in the given view, compatible with lazy loading
  • Has to be a no-brainer and clutter-less from the developer point of view
  • You should be able to extend (or override) the default icon set with your own icons easily
  • Valid for all modern frameworks, with focus on Angular and React

Harnessing the power of Webpack and modern JS

As developers we want to use the tooling that we have at our hands in the best possible way. So our icon system should use simple ES6/7/* and TypeScript conventions.

import myIcon from './icons/my-nice-icon.svg';
import Icon from './components/Icon';

and the from JSX:

<Icon name={myIcon} />

or angular template:

<Icon [name]="myIcon"></icon>

or

<div icon [name]="myIcon"></div>

Deconstructing the SVG and put it back together again

According to all the use cases shown in this interesting article by Amelia Bellamy-Royds in CSSTricks: How to Scale SVG the most sensible approach when inlining SVGs is to simply just set the viewBox on your <svg> tag, and set one of height or width to auto. The browser will adjust it so that the overall aspect ratio matches the viewBox. As Amelia points out, that would work for all modern browsers back until 2014. If we have to support older ones, we will need to apply for those the famous padding-bottom hack. Let’s keep things simple for now.

Let’s assume that our SVG is not perfect, and we want to have the all the flexibility that a modern browser can achieve handling SVGs. We will take the existing SVG, deconstruct it and get all the SVG attributes, then the content. We will then put it all together in our components, exactly the way we want it.

The Webpack part

We can accomplish all our goals by using a Webpack loaders combo for loading SVG:

{
    test: /\.svg$/,
    include: path.join(paths.appSrc, 'icons'),
    use: [
        {
        loader: 'svg-loader',
        },
        {
        loader: 'svgo-loader',
        options: {
            plugins: [
            { removeTitle: true },
            { convertPathData: false },
            { removeUselessStrokeAndFill: true },
            { removeViewBox: false },
            ],
        },
        },
    ],
},

We will use svg-loader a super simple inline svg loader that provides you extra flexibility when handling your SVG. Initially I tried the popular Webpack Team’s svg-inline-loader but it was not that flexible at the end. svg-loader returns an object with the contents and the attributes of the svg separatedly that we can later manipulate in our components. We are also filtering the SVG using the well known SVGO utility svgo-loader, we can extend or add more filtering options to optimize our SVGs thanks to it.

We are also restricting this loader to the icons folder, just in case we are handling the other SVGs in our app differently, but of course, you can use it for all SVGs removing the include key.

React

Make it work in React is very straight forward. We need to add the loader to our Webpack config, then add an icons folder and the Icon component.

import React from 'react';
import PropTypes from 'prop-types';

const defaultSize = '100%';

const Icon = ({ name, size, color }) => (
  <svg
    xmlns={name.attributes.xmlns}
    viewBox={name.attributes.viewBox}
    style={{ height: size, width: 'auto', fill: color }}
    dangerouslySetInnerHTML={{ __html: name.content }}
  />
);

Icon.propTypes = {
  name: PropTypes.shape({
    xmlns: PropTypes.string,
    viewBox: PropTypes.string,
    content: PropTypes.string,
  }).isRequired,
  size: PropTypes.string,
  color: PropTypes.string,
};

Icon.defaultProps = {
  size: defaultSize,
  color: null,
};

export default Icon;

That’s it. Our React component takes as props: the name of the imported module of the SVG, the size, and the color. If not given, the SVG will inherit the fill color set in the parent element (or itself). Also, if not specified, the SVG will scale to the parent container height.

Take a look into the JSX of the example

<div style={{ height: '100px' }}>
    <Icon name={Add} />
</div>
<Icon name={Add} size="45px" />
<Icon name={Add} size="45px" color="red" />
<Icon name={Plone} size="60px" color="#1782BE" />
<Icon name={Guillotina} size="60px" color="#EC5528" />

Angular

For the angular icon component we needed the same recipe for the Webpack config and this icon component.

import {
    Component,
    Input,
    HostBinding,
    ViewEncapsulation,
    ChangeDetectionStrategy } from '@angular/core';

import { DomSanitizer, SafeHtml } from '@angular/platform-browser';
import { OnInit } from '@angular/core';

const defaultSize = '100%';

@Component({
  // tslint:disable-next-line:component-selector
  selector: '[icon], icon',
  template: `
  <svg
    [attr.viewBox]="name.attributes.viewBox"
    [attr.xmlns]="name.attributes.xmlns"
    [innerHTML]="svgContent"
    [style.height]="height"
    [style.width]="'auto'"
    [style.fill]="color"
    >
  </svg>
  `,
  encapsulation: ViewEncapsulation.None,
  changeDetection: ChangeDetectionStrategy.OnPush
})
export class IconComponent implements OnInit {

  constructor(private sanitizer: DomSanitizer) {}

  svgContent: SafeHtml;
  defaultSize = defaultSize;
  height: string;

  @Input() color: string;
  @Input() size: string;
  @Input() name;

  ngOnInit() {
    this.svgContent = this.sanitizer.bypassSecurityTrustHtml(this.name.content);
    this.height = this.size ? this.size : defaultSize;
  }

}

We also use the same approach using the component, the Angular template way:

<div icon [name]="Add"></div>
<div icon [name]="Add" color="green"></div>
<icon [name]="Add" color="red" size="45px"></icon>
<icon [name]="Plone" color="#1782BE" size="60px"></icon>
<icon [name]="Guillotina" color="#EC5528" size="60px"></icon>

Our Angular component takes the same three properties as the React one.

In addition, Typescript forces us to overcome some tiny things.

Typings

In order to be able to import the SVG as a module, we need to add this typing to our app:

declare module "*.svg" {
  const content: any;
  export default content;
}

Add the imported SVG object as a Class member

The Angular template won’t be able to use it if the imported SVG object is not a Class member, like:

import { Component } from '@angular/core';
import Add from '../icons/add.svg';
import Plone from '../icons/plone.svg';
import Guillotina from '../icons/guillotina.svg';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  Add = Add;
  Plone = Plone;
  Guillotina = Guillotina;
}

Conclusion

While there are other approaches out there like the Icon component that @angular/material has, they all feel to me like too much and all of them are bloated with lots of options that we don’t really need. I’d like to use a more lightweight and approachable solution like the exposed here that only does what we really need. At the end, it’s not rocket science.

If you have any suggestion, please contact me or open an issue on Github. PRs are welcome!

collective.recipe.backup version 4

Posted by Maurits van Rees on January 25, 2018 12:48 PM

Since the end of 2017, there is a new version 4.0 of collective.recipe.backup. There are lots of changes since version 3.1. Let's see some of the highlights.

Safety and exactness of restore

  • When restoring, first run checks for all filestorages and blobstorages. When one of the backups is missing, we quit with an error. This avoids restoring a filestorage and then getting into trouble due to a missing blobstorage backup.
  • When restoring to a specific date, find the first blob backup at or before the specified date. Otherwise fail. The repozo script does the same. We used to pick the first blob backup after the specified date, because we assumed that the user would specify the exact date that is in the filestorage backup. Note that the timestamp of the filestorage and blobstorage backups may be a few seconds or minutes apart. So now the user should pick the date of the blob backup or slightly later. This date will give the same result with 3.1 and 4.0. But: when you use the new blob_timestamps == true option, these dates are the same.

Blob timestamps

  • Added blob_timestamps option. Default is false. By default we create blobstorage.0. The next time, we rotate this to blobstorage.1 and create a new blobstorage.0. With blob_timestamps = true, we create stable directory names that we do not rotate. They get a timestamp, just like the repozo backup. For example: blobstorage.1972-12-25-01-02-03.
  • When backing up a blobstorage, use the timestamp of the latest filestorage backup. If a blob backup with that name is already there, then there were no database changes, so we do not make a backup.
  • Automatically remove old blobs backups that have no corresponding filestorage backup. We compare the timestamp of the oldest filestorage backup with the timestamps of the blob backups. This can be the name, if you use blob_timestamps = true, or the modification date of the blob backup. This means that the keep_blob_days option is ignored, unless you use only_blobs = true.
  • Note: it is fine to switch to blob_timestamps even when you already have 'old' backups. Restoring those will still work.
  • blob_timestamps = true may become the new default later (maybe 4.1). This may even become the only valid value later (maybe 5.0), removing the creation of blobstorage.0. This would simplify the code. If you don't like this, please speak up and create an issue.

Archiving and compressing blobs

  • Renamed gzip_blob option to archive_blob. Kept the old name as alias for backwards compatibility. This makes room for letting this create an archive without zipping it.
  • Added compress_blob option. Default is false. This is only used when the archive_blob option is true. When switched on, it will compress the archive, resulting in a .tar.gz instead of a tar file. When restoring, we always look for both compressed and normal archives. We used to always compress them, but in most cases it hardly decreases the size and it takes a long time anyway. I have seen archiving take 15 seconds, and compressing take an additional 45 seconds. The result was an archive of 5.0 GB instead of 5.1 GB.
  • Note that with both archive_blob and blob_timestamps set to true, you get filenames like blobstorage.1972-12-25-01-02-03.tar.
  • Added incremental_blobs option. This creates tarballs with only the changes compared to the previous blob backups. This option is ignored when the archive_blob option is false.

Various

  • No longer create the fullbackup script by default. You can still enable it by setting enable_fullbackup to true.
  • Added Python 3 support. The integration with plone.recipe.zope2instance is not tested there, because there is no Python 3 compatible release of it yet.

Upgrading

  • In most cases you can simply use the new version without changes.
  • Adding blob_timestamps = true is highly recommended. If you do this, you can remove the keep_blob_days option, unless you use only_blobs = true.
  • If you want the fullbackup script, enable it by setting enable_fullbackup to true.
  • When you used the gzip_blob option, you should rename this to archive_blob. Maybe enable the compress_blob option, but you are probably better off without this.

plone.restapi 1.0.0 released - A Story of Successful Open Source Collaboration

Posted by kitconcept GmbH on January 19, 2018 07:24 PM

After more than three years of development and 25 alpha and one beta release, we are very happy and proud to announce the release of plone.restapi 1.0.0.

plone.restapi is a RESTful hypermedia API for the Plone Open Source Content Management System. It exposes the unique and powerful features of Plone, including the core content management features as well as dynamic content type creation, workflows, permissions, versioning and more.

plone.restapi builds a bridge between a stable and mature Open Source CMS that has been around for more than 15 years and modern state-of-the-art JavaScript-based solutions like React, Angular, Vue and others.

A Little Bit of History

PLOG 2014

The development of plone.restapi started in beautiful Sorrento, Italy at the Plone Open Garden in 2014 after I gave a talk about building an AngularJS application on top of Plone.

plog2014

A long discussion with Simone Deponti under the Italian sun, about REST API design principles and hypermedia (of course), led to the first commit and the development of a first proof-of-concept implementation.

plone.rest and PLOG 2015

One year later we gathered in Sorrento again. Laurence Rowe, Ramon Navarro Bosch and I spent our days and nights discussing the details of the REST API design and drafted multiple endpoints.

plog2015

One of the main obstacles to building a RESTful API on top of Plone was the missing ZPublisher support for HTTP verbs such as PATCH, PUT or DELETE. In 2015, I sat together with Ramon Navarro Bosch in Sorrento (again) and we (he really did all the heavy lifting) started to build plone.rest, a small package that adds support for HTTP verbs to Plone.

Archetypes and Serializers

We never planned to support Archetypes in plone.restapi. Though, when Thomas Buchberger and Lukas Graf came along and offered to build it, we did not object (of course not, this is Open Source). Their company 4teamwork planned to build a REST api on top of Plone for their OneGov GEVER platform.

Instead of building something on their own, they decided to join forces and share their work and code with the community. Along the way, they heavily refactored the code, added tons of adapters for loose coupling and the ability to customize the JSON serialization.

After this, we were confident to do a first alpha release of plone.restapi on June 14th 2016.

Beethoven Sprint

In March 2017, fourteen Plone developers from eight different countries gathered in Bonn, at the kitconcept office, for the Beethoven Sprint to work on plone.restapi and related topics. In addition to sorting out the last remaining design decision, many exciting new projects were started and announced.

BeethovenSprint065

Angular

At the Beethoven sprint, Eric Brehault started to work on an Angular SDK for plone.restapi. A release followed soon and Eric gave a very successful and crowded training at the Plone Conference 2017 in Barcelona.

Today, Angular SDK is a mature package for Angular 2 that makes it really easy for front-end developers to interact with Plone and a fantastic starting point for newbies.

Eric and I mentored Noel Varghese during last year’s Google Summer of Code to build a Progressive Web App for Plone in Angular 2. Noel gave a nice presentation of his successful project at the Plone Conference in Barcelona.

React

Rob Gietema and Roel Bruggink started to build a React-based front-end on top of plone.restapi at the Beethoven sprint in Bonn. Later that year, they went to Toulouse in September 2017 to implement the Pastanaga CSS together with the Plone Angular team.

In November they visited Bonn again for the Pastanaga Sprint where we started to implement the new Pastanaga UI for plone-react.

BeethovenSprint183

At kitconcept, we started to use plone-react with Pastanaga for an ongoing project. We can’t wait to release our work and contribute it back to the community.

Vue.JS

Inspired by the Angular SDK and plone-react, Kevin Bieri started to build a VueJS plone-vuejs implementation on top of Plone at the Plone Conference 2017 in Bareclona.

Guillotina

Ramon Navarro Bosch and Nathan van Gheem revelead the name of “Guillotina”, a blazing fast async Python framework that shares the public API with plone.restapi at the Beethoven sprint in Bonn.

Successful Open Source Collaboration

plone.restapi started with an idea and discussions. People and companies jumped in and contributed in many ways that haven’t been dreaming about at first.

Simone, Laurence, Ramon and other helped to shape the initial idea. Lukas, Thomas, Roel, Carsten, Victor, Mikel and many others contributed new endpoints, bugfixes, etc.

Eric, Rob, Noel, Kevin, and others started to build frameworks and solutions on top of plone.restapi.

Many companies such as 4teamwork, Code Syntax, Markina Corpus, VNC invested and contributed to plone.restapi.

The Plone Foundation always supported our efforts by funding sprints.

plone.restapi is a true community effort and the joy that we feel when collaborating with wonderful people pays us back for the countless hours we spend on hacking on code.

Future Plans

A Plone Improvement Proportal (PLIP) to ship Plone 5.2 with plone.restapi has been accepted by the Plone Framework Team:

https://github.com/plone/Products.CMFPlone/issues/2177

With plone.restapi considered stable and close to being feature complete, we will continue working on what could become the next Plone…stay tuned.

Configuring the ufw firewall to allow Cloudflare IP addresses

Posted by T. Kim Nguyen on January 03, 2018 01:56 AM

I have a Linode running Ubuntu 16.04, and I use the ufw firewall.

I have a web site running on that server, originally accessible via HTTPS on port 443 from anywhere on the internet.

The domain for that web site is managed via Cloudflare. I want the site to be available only through the domain, and not via the Linode's IP address.

Cloudflare publishes the IP addresses it uses to access your web site: https://www.cloudflare.com/ips/

Here is a page describing the overall idea of using ufw to allow access to your web site only from those Cloudflare IP addresses: https://www.ajsalkeld.com/blog/tutorial/2016/08/01/how-to-use-ufw-to-whitelist-cloudflare-ips-ubuntu-debian-digitalocean/

In this repo https://github.com/Paul-Reed/cloudflare-ufw there is a script that does this: https://github.com/Paul-Reed/cloudflare-ufw/blob/master/cloudflare-ufw.sh

I modified it a bit so that:

  • it uses the /tmp directory
  • it uses a unique filename (containing the current process ID) when retrieving the Cloudflare IP addresses
  • it specifically allows connections only on port 443 (you may want to allow connections on port 80 as well or instead)
  • it just outputs to the screen the commands that it would issue using ufw; If the commands look sane/good to you, copy and paste them into your terminal to run them

Here is my script:

#!/bin/sh
cd /tmp
wget https://www.cloudflare.com/ips-v4 -O ips-v4-$$.tmp
wget https://www.cloudflare.com/ips-v6 -O ips-v6-$$.tmp

for cfip in `cat ips-v4-$$.tmp`; do echo "ufw allow from $cfip to any port 443 proto tcp"; done
for cfip in `cat ips-v6-$$.tmp`; do echo "ufw allow from $cfip to any port 443 proto tcp"; done

Once I ran the script and copied and pasted its output into a terminal, ufw was configured as follows:

# ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 22                         ALLOW IN    Anywhere
[ 2] 443/tcp                    ALLOW IN    103.21.244.0/22
[ 3] 443/tcp                    ALLOW IN    103.22.200.0/22
[ 4] 443/tcp                    ALLOW IN    103.31.4.0/22
[ 5] 443/tcp                    ALLOW IN    104.16.0.0/12
[ 6] 443/tcp                    ALLOW IN    108.162.192.0/18
[ 7] 443/tcp                    ALLOW IN    131.0.72.0/22
[ 8] 443/tcp                    ALLOW IN    141.101.64.0/18
[ 9] 443/tcp                    ALLOW IN    162.158.0.0/15
[10] 443/tcp                    ALLOW IN    172.64.0.0/13
[11] 443/tcp                    ALLOW IN    173.245.48.0/20
[12] 443/tcp                    ALLOW IN    188.114.96.0/20
[13] 443/tcp                    ALLOW IN    190.93.240.0/20
[14] 443/tcp                    ALLOW IN    197.234.240.0/22
[15] 443/tcp                    ALLOW IN    198.41.128.0/17
[16] 22 (v6)                    ALLOW IN    Anywhere (v6)
[17] 443/tcp                    ALLOW IN    2400:cb00::/32
[18] 443/tcp                    ALLOW IN    2405:8100::/32
[19] 443/tcp                    ALLOW IN    2405:b500::/32
[20] 443/tcp                    ALLOW IN    2606:4700::/32
[21] 443/tcp                    ALLOW IN    2803:f800::/32
[22] 443/tcp                    ALLOW IN    2c0f:f248::/32
[23] 443/tcp                    ALLOW IN    2a06:98c0::/29

I tested by browsing to my web site's domain (e.g. https://mysite.com) and it worked. Then I tried to browse to my server's IP address (e.g. https://123.45.67.89) and it did not work, as expected and as intended.


Update: January 3, 2018: Thank you to Florian Schulze who suggested the use of Cloudflare's authenticated origin pulls, described at https://support.cloudflare.com/hc/en-us/articles/204494148-Setting-up-NGINX-to-use-TLS-Authenticated-Origin-Pulls. With this method, you don't have to worry that Cloudflare may have changed its IP addresses (the reason why you would need to update your ufw rules periodically).

There is also TLS client side authentication, a feature described at https://support.cloudflare.com/hc/en-us/articles/115000088491-Cloudflare-TLS-Client-Auth. It is, however, available only to Enterprise Cloudflare customers.

Continuous Performance Analysis with Lighthouse and Jenkins

Posted by kitconcept GmbH on December 22, 2017 06:11 AM

Lighthouse is an open-source, automated tool for improving the quality of web pages by Google. It measures the performance of a website and provides metrics for accessibility, best practices for modern web apps, search engine optimization, and assess web applications for adherence to Progressive Web App standards.

Lighthouse Logo Lighthouse Logo

Together with WebPageTest and Google Page Speed Insights it is an indispensable tool to optimize your website performance.

Installation

Lighthouse can be installed in any JavaScript-based project by just running ‘npm install’:

$ npm install lighthouse -g

If you don’t have a package.json in your project, just install npm and run ‘npm init’ before installing.

Running Lighthouse

You can check the performance of any website by calling the ‘lighthouse’ command with the URL of the website you want to test. Append the --view parameter to show the HTML report, right after the command has finished:

$ lighthouse https://kitconcept.com --view

The report will give you five different ratings about PWA, performance, accessibility, performance best practices, and SEO.

Lighthouse Results Lighthouse Results

Continuous Performance Measurements

If you run your performance test every now and then, you always risk to hurt your website performance without noticing. If a performance regression happens unnoticed, it is usually very hard and time consuming to figure out which change caused the performance regression.

You can easily fix this and save lots of time when you run your performance tests and analysis continuously.

Unfortunately Lighthouse does not allow you to set performance test specifications that your CI system can test against, like WebPageTest or Google Page Speed Insights do (we will cover those tools in later blog posts). Though, it is still very convenient to run the performance test on a regular basis for each commit and include them into your CI report.

Install Lighthouse locally for CI

When it comes to a Continuous Integration, a local installation is prefered over a global one, which is usually harder to manage and to maintain. Especially if you have multiple projects with different sets of package versions on your CI.

Therefore we install Lighthouse locally in our project directory:

$ npm install lighthouse --save-dev

This command will install Lighthouse to your local package.json file. We recommend to use yarn or npm package-lock.json to lock down the package version you are using for a repeatable and stable project build.

For convenience, we add a “lighthouse” script to our package.json:

"scripts": {
  "lighthouse:ci": "node_modules/lighthouse/lighthouse-cli/index.js \
  --output-path=./lighthouse-report.html --quiet \
  --chrome-flags='--headless' https://kitconcept.com"
}

We call the locally installed lighthouse binary and set a static output path (by default, Lighthouse creates a file with the current date/time in the filename which makes it harder to publish on your CI).

We also include the --quiet option and run it on headless chrome, so we don’t need to install and run an X server on our CI system.

At the end, we hard-code our project URL into the command so we do not have to type it manually each time we run this command.

Now we can just run:

$ npm run lighthouse:ci

and it will create a nice HTML report that we can publish in our CI.

Configure Lighthouse for your local development environment

For convenience, we also add a command that you can run locally:

"scripts": {
  "lighthouse": "node_modules/lighthouse/lighthouse-cli/index.js \
  --output-path=./lighthouse-report.html --quiet \
  --chrome-flags='--headless' https://kitconcept.com/blog"
}

The --view parameter will fire up a browser with the report at the end of the performance analysis. This is something we clearly don’t want on our CI system.

Publish Lighthouse Reports in Jenkins CI

Travis and other lightweight CI system usually lack the option to publish any reports except the command line output. Though, if you are using Jenkins CI, you can use the HTML publisher plugin to publish your Lighthouse report.

sh 'npm install'
sh 'npm run lighthouse'
publishHTML (target: [
  allowMissing: false,
  alwaysLinkToLastBuild: false,
  keepAll: true,
  reportDir: '.',
  reportFiles: 'lighthouse-report.html',
  reportName: "Lighthouse"
])

After adding publishHTML to your Jenkins pipeline, you will see a “Lighthouse” link under the ‘Artifacts’ tab:

Link to Lighthouse report in Jenkins Link to Lighthouse report in Jenkins

There is a caveat though. Jenkins 1.641 / 1652.3 introduce the Content-Security-Policy header to static files served by Jenkins. The default header is set to a very restrictive set of permissions to protect Jenkins users from malicious HTML/JS files in workspaces.

To allow Jenkins to display the Lighthouse reports, we have to add the following JAVA_ARGS to the Jenkins startup (for instance by adding the following line to your /etc/default/jenkins file):

JAVA_ARGS="-Dhudson.model.DirectoryBrowserSupport.CSP=\"sandbox
allow-scripts; default-src 'unsafe-inline'; img-src * data:\""

For more details see the Content Security Policy Reference and the Jenkins docs on configuring Content Security Policy.

After you fixed the Content Security Policy of your Jenkins you will see the full report when clicking on the ‘Lighthouse’ link on the ‘Artifacts’ tab on your Jenkins build:

Lighthouse full report in Jenkins Lighthouse Report in Jenkins

Jenkins Declarative Pipeline Stage for Performance Tests

A full declarative pipeline stage for lighthouse looks like this:

stage('Performance Tests') {
  agent {
    label 'master'
  }
  when {
    branch 'master'
  }
  steps {
    deleteDir()
    checkout scm
    sh 'npm install'
    sh 'npm run lighthouse'
  }
  post {
    always {
      publishHTML (target: [
        allowMissing: false,
        alwaysLinkToLastBuild: false,
        keepAll: true,
        reportDir: '.',
        reportFiles: 'lighthouse-report.html',
        reportName: "Lighthouse"
      ])
    }
  }
}

We run the performance test stage on ‘master’ agents and only on the master branch. The steps performed are a simple “npm install” to set up the project build and then we run ‘npm run lighthouse’ to produce the HTML report. If you already have an npm build from a previous step you can of course just unstash the build artifact.

Jenkins pipeline with lighthouse performance tests Jenkins pipeline with Lighthouse performance tests stage

Summary

Lighthouse is a valuable and indispensable tool if you want to deliver a fast and user friendly website. Running the analysis on a continuous basis on your CI is a good idea if you take performance seriously. Setting it up is fast and easy. Maybe in the future Lighthouse will also provide a testspec feature that will allow us to fail a CI build (or mark it as unstable) on performance regressions. Though, if you run WebPageTest or Google Page Speed Insights additionally, this is not really needed.

Jazkarta Sponsors Northwest Youth Leadership Summit

Posted by Jazkarta Blog on December 07, 2017 08:21 PM

NWYLS Group Shot

Jazkarta is pleased to have recently sponsored the North Cascades Institute‘s Northwest Youth Leadership Summit. This event is intended to empower Cascadia’s future leaders in conservation by:

  • Enhancing their skills in preparation for job and college applications
  • ​Connecting with regional environmental organizations and businesses to learn about jobs and internships
  • Learning from like-minded peers about career options available in the conservation, outdoor and environmental fields

More than 220 students participated and are now better equipped to take action towards conservation. The Summit was free to all participants to ensure that underrepresented youth are given opportunities to get involved in the outdoor and environmental fields.

The sponsorship added another dimension to our existing partnership with North Cascades Institute. Just before the summit, we had given the non-profit’s Plone+Salesforce website a mobile refresh to make it work smoothly on phones and tablets. If we say so ourselves, the results are quite beautiful. Kudos to Neal Maher for the designs and to the Jazkarta team (Christine Winckler and David Glick) for a smooth implementation.

North Cascades Institute is not the only environmental non-profit organization that Jazkarta is working with – we created The Mountaineers‘s website and the Washington Trails Association ‘s volunteer management system. Both organizations were involved in the Summit. It was hosted at The Mountaineers’ Seattle Program Center, here is one of the students using the climbing wall.

NWYLS Student on The Mountaineers Climing Wall

Andrew Pringle of the Washington Trails Association led a breakout session titled “Trip Planning 101: An Introduction to Leading Backcountry Adventures”, and both organizations ran booths, talking with participants about activities, internships and employment options for young outdoor leaders.  Here’s Andrew at the WTA booth.

WTA's Andrew Pringle at the NWYLS

We feel very lucky to be helping all of these organizations further their missions.

 

— Photos by North Cascades Institute staff

20171128

Posted by PLONE.ORG on November 28, 2017 12:00 AM
Several XSS and redirect fixes, and a sandbox escape fix.

Security patch released 20171128

Posted by PLONE.ORG on November 28, 2017 12:00 AM
This is a routine patch with our standard 14 day notice period. There is no evidence that the issues fixed here are being used against any sites.

CVE numbers not yet issued.

Versions Affected: All supported Plone versions (4.x, 5.x). Previous versions could be affected but have not been tested.

Versions Not Affected: None.

Nature of vulnerability: Low severity, no data exposure or privilege escalation for anonymous users.

The patch was released at 2017-11-28 15:00 UTC.

Installation

Full installation instructions are available on the HotFix release page.

Standard security advice

  • Make sure that the Zope/Plone service is running with minimum privileges. Ideally, the Zope and ZEO services should be able to write only to log and data directories. Plone sites installed through our installers already do this.
  • Use an intrusion detection system that monitors key system resources for unauthorized changes.
  • Monitor your Zope, reverse-proxy request and system logs for unusual activity.
  • Make sure your administrator stays up to date, by following the special low-volume Plone Security Announcements list via email, RSS and/or Twitter

These are standard precautions that should be employed on any production system, and are not tied to this fix.

Extra Help

If you do not have in-house server administrators or a service agreement for supporting your website, you can find consulting companies at plone.com/providers

There is also free support available online via the Plone forum and the Plone chat channels.

Q: When will the patch be made available?A: The Plone Security Team will release the patch at 2017-11-28 15:00 UTC.

Q. What will be involved in applying the patch?A. Patches are made available as tarball-style archives that may be unpacked into the products folder of a buildout installation and as Python packages that may be installed by editing a buildout configuration file and running buildout. Patching is generally easy and quick to accomplish.

Q: How were these vulnerabilities found?A: The vulnerabilities were found by users submitting them to the security mailing list.

Q: My site is highly visible and mission-critical. I hear the patch has already been developed. Can I get the fix before the release date? A: No. The patch will be made available to all administrators at the same time. There are no exceptions.

Q: If the patch has been developed already, why isn't it made available to the public now? A: The Security Team is still testing the patch against a wide variety of configurations and running various scenarios thoroughly. The team is also making sure everybody has appropriate time to plan to patch their Plone installation(s). Some consultancy organizations have hundreds of sites to patch and need the extra time to coordinate their efforts with their clients.

Q: How does one exploit the vulnerability?A: This information will not be made public until after the patch is made available.

Q: Is my Plone site at risk for this vulnerability? How do I know if my site has been exploited? How can I confirm that the hotfix is installed correctly and my site is protected?

A: Details about the vulnerability will be revealed at the same time as the patch.

Q: How can I report other potential security vulnerabilities?

A: Please email the Plone Security Team at security@plone.org rather than publicly discussing potential security issues.

Q: How can I apply the patch without affecting my users?

A: Even though this patch does NOT require you to run buildout, you can run buildout without affecting your users. You can restart a multi-client Plone install without affecting your users; see http://docs.plone.org/manage/deploying/processes.html  

Q: How do I get help patching my site?

A: Plone service providers are listed at plone.com/providers  There is also free support available online via the Plone forum and the Plone chat channels

Q: Who is on the Plone Security Team and how is it funded?

A: The Plone Security Team is made up of volunteers who are experienced developers familiar with the Plone code base and with security exploits. The Plone Security Team is not funded; members and/or their employers have volunteered their time in the interests of the greater Plone community.

Q: How can I help the Plone Security Team?

A: The Plone Security Team is looking for help from security-minded developers and testers. Volunteers must be known to the Security Team and have been part of the Plone community for some time. To help the Security Team financially, your donations are most welcome at http://plone.org/sponsors

General questions about this announcement, Plone patching procedures, and availability of support may be addressed to the Plone support forums If you have specific questions about this vulnerability or its handling, contact the Plone Security Team at security@plone.org

To report potentially security-related issues, email the Plone Security Team at security@plone.org We are always happy to credit individuals and companies who make responsible disclosures.

Information for Vulnerability Database Maintainers

We will apply for CVE numbers for these issues. Further information on individual vulnerabilities (including CVSS scores, CWE identifiers and summaries) will be available at the full vulnerability list.

Security vulnerability pre-announcement: 20171128

Posted by PLONE.ORG on November 26, 2017 10:30 PM
This is a routine patch with our standard 14 day notice period. There is no evidence that the issues fixed here are being used against any sites.

CVE numbers not yet issued.

Versions Affected: All supported Plone versions (4.x, 5.x). Previous versions could be affected but have not been tested.

Versions Not Affected: None.

Nature of vulnerability: Low severity, no data exposure or privilege escalation for anonymous users.

The patch will be released at 2017-11-28 15:00 UTC.

Preparation

This is a pre-announcement of availability of this security fix. 

The security fix egg will be named Products.PloneHotfix20171128 and its version will be 1.0. Further installation instructions will be made available when the fix is released.

Standard security advice

  • Make sure that the Zope/Plone service is running with minimum privileges. Ideally, the Zope and ZEO services should be able to write only to log and data directories. Plone sites installed through our installers already do this.
  • Use an intrusion detection system that monitors key system resources for unauthorized changes.
  • Monitor your Zope, reverse-proxy request and system logs for unusual activity.
  • Make sure your administrator stays up to date, by following the special low-volume Plone Security Announcements list via email, RSS and/or Twitter

These are standard precautions that should be employed on any production system, and are not tied to this fix.

Extra Help

Should you not have in-house server administrators or a service agreement for supporting your website, you can find consulting companies at plone.com/providers

There is also free support available online via the Plone forum and the Plone chat channels.

Q: When will the patch be made available?A: The Plone Security Team will release the patch at 2017-11-28 15:00 UTC.

Q. What will be involved in applying the patch?A. Patches are made available as tarball-style archives that may be unpacked into the products folder of a buildout installation and as Python packages that may be installed by editing a buildout configuration file and running buildout. Patching is generally easy and quick to accomplish.

Q: How were these vulnerabilities found?A: The vulnerabilities were found by users submitting them to the security mailing list.

Q: My site is highly visible and mission-critical. I hear the patch has already been developed. Can I get the fix before the release date? A: No. The patch will be made available to all administrators at the same time. There are no exceptions.

Q: If the patch has been developed already, why isn't it made available to the public now? A: The Security Team is still testing the patch against a wide variety of configurations and running various scenarios thoroughly. The team is also making sure everybody has appropriate time to plan to patch their Plone installation(s). Some consultancy organizations have hundreds of sites to patch and need the extra time to coordinate their efforts with their clients.

Q: How does one exploit the vulnerability?A: This information will not be made public until after the patch is made available.

Q: Is my Plone site at risk for this vulnerability? How do I know if my site has been exploited? How can I confirm that the hotfix is installed correctly and my site is protected?

A: Details about the vulnerability will be revealed at the same time as the patch.

Q: How can I report other potential security vulnerabilities?

A: Please email the Plone Security Team at security@plone.org rather than publicly discussing potential security issues.

Q: How can I apply the patch without affecting my users?

A: Even though this patch does NOT require you to run buildout, you can run buildout without affecting your users. You can restart a multi-client Plone install without affecting your users; see http://docs.plone.org/manage/deploying/processes.html  

Q: How do I get help patching my site?

A: Plone service providers are listed at plone.com/providers There is also free support available online via the Plone forum and the Plone chat channels

Q: Who is on the Plone Security Team and how is it funded?

A: The Plone Security Team is made up of volunteers who are experienced developers familiar with the Plone code base and with security exploits. The Plone Security Team is not funded; members and/or their employers have volunteered their time in the interests of the greater Plone community.

Q: How can I help the Plone Security Team?

A: The Plone Security Team is looking for help from security-minded developers and testers. Volunteers must be known to the Security Team and have been part of the Plone community for some time. To help the Security Team financially, your donations are most welcome at http://plone.org/sponsors

General questions about this announcement, Plone patching procedures, and availability of support may be addressed to the Plone support forums If you have specific questions about this vulnerability or its handling, contact the Plone Security Team at security@plone.org

To report potentially security-related issues, email the Plone Security Team at security@plone.org We are always happy to credit individuals and companies who make responsible disclosures.

Information for Vulnerability Database Maintainers

We will apply for CVE numbers for these issues. Further information on individual vulnerabilities (including CVSS scores, CWE identifiers and summaries) will be available at the full vulnerability list.

fail2ban configuration error fix

Posted by T. Kim Nguyen on November 26, 2017 04:09 PM

If you have this in your /etc/fail2ban/jail.local configuration file:

# "bantime" is the number of seconds that a host is banned.
bantime = 31536000 # 1 year

# A host is banned if it has generated "maxretry" during the last "findtime"
# seconds.
findtime = 604800 # 7 days

and you get these errors when you restart fail2ban (service fail2ban restart):

WARNING Wrong value for 'findtime' in 'ssh'. Using default one: '600'
WARNING Wrong value for 'bantime' in 'ssh'. Using default one: '600'

change it to this (put the comment on a separate line):

# "bantime" is the number of seconds that a host is banned.
# 1 year
bantime = 31536000

# A host is banned if it has generated "maxretry" during the last "findtime"
# seconds.
# 7 days
findtime = 604800

This is explained in the following bug report:

fail2ban: Incorrect parsing of commented text after reading a value from config file

If you want to set a permanent ban time, use a negative number.

# "bantime" is the number of seconds that a host is banned.
# permanent ban
bantime = -1

Pastanaga Sprint Bonn 2017

Posted by kitconcept GmbH on November 23, 2017 05:02 PM
Pastanaga is a new user experience framework for the web, designed by Albert Casado.

pastanaga

Pastanaga was first presented in March 2017, at the Plone Open Garden in Sorrento. In July, we started with an initial implementation during the Midsummer Sprint in Jyväskylä, Finnland.

Pastanaga was also present at the recently held Plone Conference in Barcelona, where Albert gave a presentation on it. In addition, Eric Steele, the Plone release manager, gave us the opportunity to present Pastanaga to the audience during his keynote on the first day of the conference.

With all the positive feedback and energy we took from the Plone Conference, we wanted to push things further and we just couldn’t wait until our “Beethoven Sprint”, which is planned for early 2018. Therefore we decided to organize a small and focused sprint at our office in Bonn to work on the implementation of Pastanaga.

The Pastanaga Minimal Viable Product

As an Open Source community (and software engineers) with many years of experience in designing and building complex Content Management System applications, we sometimes have the tendency to try to solve all problems at once.

Over the years we encountered and solved many complex problems and when we build something new, this can be both a source of wisdom as well as a baggage that you carry around.

This sometimes led to a situation where we were over-engineering solutions, to solve all the problems that we encountered over the years at once. Enhancements sometimes stayed around for years without really becoming production ready and usable in real-world projects.

To avoid this from happening when working on implementing Pastanaga, we decided in Jyväskylä to focus on a Minimal Viable Product.

A Minimum Viable Product (MVP) is a product with just enough features to satisfy early customers, and to provide feedback for future product development. The Pastanaga MVP needs to provide what we consider the essentials of a Content Management System:

  • A site administrator can and add, edit, and delete a page

  • A user can view the created pages and navigate the site structure

In order to be usable for public facing website projects, we added two additional technical requirements:

  • The page should be fully rendered within 500 milliseconds

  • Google should be able to crawl the contents of the website

Those requirements might sound very simple, but they are actually not.

Pastanaga aims to leverage the editing experience and reduce the complexity that we took for granted over the years. We aim to simplify the user experience for the editors by getting rid of things that we got used to. For instance, adding an image to a page should be as simple as just dragging and dropping an image to the page and Plone will take care about the heavy lifting of automatically uploading and resizing the image.

You can find a list of all the user stories that we plan to implement as part of the MVP here:

Having the goals and scope for this set the only thing that was needed was a bunch of Plone devs and three days and nights of coding.

Sprint Day One

After the sprinters arrived, we started with our sprint planning session. We decided to focus on the implementation of the Pastanaga MVP and work on the other issues (e.g. plone.restapi) only if we need them for the MVP.

After the planning meeting, Rob gave us an introduction to plone-react, a ReactJS-based implementation of the Plone UI that he and Roel worked on over the past months and that we decided to use as a basis for our MVP.

We went through all components, reducers, bells and whistles of the application and discussed best practices, developer environments and developer approachability.

After that session, Rob and Victor started with the implementation of Pastanaga. Davi created a pull request that adds an uninstall profile for plone.restapi and started to learn about React. Roel started to look into a way to turn the Plone site root into a Dexterity object, something that we would need to simplify the Plone editing experience. I worked on the basic Robot Framework acceptance test setup and updated the contents of the Pastanaga github repository, which is supposed to be just an entry point for all our initiatives around Pastanaga:

Day Two

On the second day, Victor finished the login form and made the error messages work.

Rob implemented the document edit accordion menu, fixed the button styling, made plone-react use the Pastanaga icons and started to work on the toolbar.

document edit

Davi added a search widget to the header, implemented the breadcrumbs navigation and added styles for the document heading and description.

document view

Right before the wrap-up meeting of day two, Roel showed us a Plone site with a “containerish” Dexterity-based site root. We did not really expect that much progress and went to bed (some of us a lot later) still very impressed by his accomplishment.

Day Three

On day three, Rob started to work on the new Pastanaga document edit view. He made the new edit view to show multiple content items (e.g text, image, video) and allowed to change the order of those content items via drag and drop.

Davi continued to work on the header and breadcrumbs styling. Victor looked into the mobile views of our responsive design, fixed some issues with the status messages and briefly started to look into GatsbyJS (which we plan to use to implement pastanaga.io).

Summary

After three days (and nights) of hacking, we had:

A fully functional login form with error messages and password forgotten functionality:

Login

A fully functional Pastanaga Toolbar that can be collapsed or expanded. With all the menu items present and the personal toolbar functionality available:

toolbar

A view to add and edit pages with all the existing functionality:

document edit

In three sprint days, we accomplished our main goals and were able to create the first iteration of a Minimal Viable Product that we can use to build things upon. We plan to continue to work on this, use it in our current and upcoming projects, and of course: contribute back as much as we can.

Stay tuned for more updates on this soon!

Successful Pastanaga Sprint 2017 in Bonn

Posted by PLONE.ORG on November 23, 2017 01:00 PM

Pastanaga is a new user experience framework for Plone designed by Albert Casado. Albert presented his vision during his talk at the Plone Conference 2016 in Boston. Eric Steele, the Plone release manager gave us the opportunity to present Pastanaga during his keynote at the same conference. Albert gave another presentation on the progress made and the more developed vision for Pastanaga at the Plone Conference 2017 in Barcelona.

To bring Pastanaga to life, five Plone developers gathered in Bonn, at the office of the kitconcept GmbH, for a small and focused sprint (see the announcement and the event) from November 15-17, 2017, to work on the implementation on top of plone-react, a ReactJS-based Plone frontend written by Rob Gietema.

The three-day sprint was a major success. The sprinters were able to implement a "Minimal Viable Product" of Pastanaga with React on top of the new Plone RESTful API. See the full sprint report for details.

The sprint was kindly sponsored by the Plone Foundation and kitconcept GmbH.

Successful Google Summer of Code 2017

Posted by PLONE.ORG on November 16, 2017 05:47 PM

Google Summer of Code ("GSoC") is an annual international program open to university students in which Google awards stipends to all students who successfully complete a free and open-source software  project.

The Plone community is proud to announce four successful projects were completed for GSoC 2017. 

All five GSoC students were offered sponsorship by the Plone Foundation to travel to Barcelona for the Plone Digital Experience 2017 conference. Oshane, Mikko, Noel, and Shriyansh Agrawal (content import and export) were able to attend and present their work to enthusiastic audiences.

Cris Ewing was our new-for-2017 coordinator of the Plone community's GSoC involvement. The Plone Foundation Board expresses its gratitude to him on behalf of the entire Plone community for having managed this very important project.

We also truly appreciate the time and effort of our GSoC students and their mentors in continuing to move Plone forward.

On to 2018, for which we have already begun soliciting project ideas

Plone Foundation Officers 2017-2018

Posted by PLONE.ORG on November 15, 2017 08:25 PM

All seven Plone Foundation Board members' nominations were accepted at the Annual General Meeting held in Barcelona on October 20, 2017: 

  • Paul Roeland
  • Alexander Loechel
  • Carol Ganz
  • Chrissy Wainwright
  • Víctor Fernández de Alba
  • Philip Bauer
  • T. Kim Nguyen

At the first Board meeting of the new term on November 2, 2017, the officers of the Foundation were voted in. The officers are elected annually:

  • President: Paul Roeland
  • Vice President: Alexander Loechel
  • Secretary: Chrissy Wainwright
  • Treasurer (non-voting): Jen Myers

Apart from these official Foundation roles, there are further roles and tasks that the Board attends to:

  • Marketing lead: T. Kim Nguyen
  • Framework team liaison: Philip Bauer
  • Security team liaison & Higher Education liaison: Alexander Loechel
  • Communications/Marketing team lead: T. Kim Nguyen
  • Front End team lead: Víctor Fernández de Alba
  • Foundation Membership committee co-chairs: Érico Andrei, T. Kim Nguyen

For more information on the Plone Foundation or its board, visit plone.org/foundation, or drop an e-mail to .

Plone is an open source web content management system excelling in usability, accessibility, and versatility. The Plone Foundation is a US 501(c)3 tax-exempt organization that protects and promotes Plone.

Thank you, Barcelona!

Posted by PLONE.ORG on November 15, 2017 06:02 PM

The Plone Digital Experience Conference 2017 in Barcelona was an exhilarating success, bringing together the Plone, Python web, and modern JavaScript front end communities in the beautiful city of Barcelona. 

IMG_0056.jpg IMG_0411.jpg IMG_0507.jpg IMG_7945.jpg

Some statistics: 

  • 10 training classes
  • 6 keynotes
  • 52 presentations 
  • 180 attendees from 21 countries
  • 2 organizing companies, 18 sponsors, 4 partners
  • 70 sprinters
  • 4 Google of Summer of Code 2017 students
  • 1 truly memorable conference dinner
  • 1 official Plone band 
  • dozens of volunteers

Some artifacts:

  • Speakers' slides can be found for almost all the presentations (video recordings still to come).
  • Photos of the conference
  • Tweets during the conference 

On behalf of the Plone community, thank you 2017 organizing team!

  • Victor Fernandez de Alba
  • Ramon Navarro Bosch
  • Agata Avalo
  • Albert Casado
  • Timo Stollenwerk
  • Philip Bauer
  • Paul Roeland
  • Kim Nguyen
  • Sally Kleinfeldt
  • Mikel Larreategi
  • Eric Bréhault

IMG_0627.JPG

Obstacles on the road towards Plone 2020

Posted by Starzel.de on November 10, 2017 09:45 AM

During the sprint at the Plone Conference 2017 in Barcelona, Plone achieved a major milestone towards what is often called "Plone 2020". This is basically the effort to modernize Plone's backend and achieve Python 3 compatibility. In 2020, support for Python 2.7 will officially end, hence Plone 2020.

A necessary part of that effort was to migrate Zope to Python 3, a daunting task that was only possible by a flurry of activity that combined the efforts of many stakeholders (not only the Plone Community). Learn more about that in Hanno Schlichting's talk once the video is on the website, and on many blog posts on the Gocept Blog.

Getting Plone to run on that newest version of Zope (currently Zope 4.0b2) was another story and took a lot of work (some details are in my post here. Finally in Barcelona, in a daring move we merged all the work that had been done for that PLIP https://github.com/plone/Products.CMFPlone/issues/1351 and decided that the result will be called Plone 5.2. But by that time not all tests were green (that's why it was daring). We worked hard to get the tests to pass and to fix some issues we found when testing manually.

By the way: At the same sprint we started to prepare Plone itself for Python 3 by fixing all imports to work in both Python 2 and Python 3. But that is a tale for another blog post.

So, despite out best efforts, even one week after the conference I was not yet able to fix all the tests, and so I created at ticket to track the remaining issues.

Here this story about two erroring tests in Products.CMFFormController actually begins. Here is the spoiler: I did not really solve the issue but finally worked around it. But I still think the approach I took might be of interest to some.

The two breaking tests, test_attacker_redirect and test_regression, were passing when I ran them in isolation or when I ran all test of Products.CMFFormController with ./bin/test -s Products.CMFFormController. To add insult to injury, Products.CMFFormController is basically dead code but is still used by some of our legacy ControllerPageTemplates.

So how could I find the issue since the traceback was not really helpful?

Here is the relevant part of the log from jenkins:

#### Running tests for group Archetypes ####
Running Products.Archetypes.tests.attestcase.Archetypes:Functional tests:

[...]

Running plone.app.testing.bbb.PloneTestCase:Functional tests:
  Tear down Testing.ZopeTestCase.layer.ZopeLite in 0.000 seconds.
  Set up plone.testing.zca.LayerCleanup in 0.000 seconds.
  Set up plone.testing.z2.Startup in 0.101 seconds.
  Set up plone.app.testing.layers.PloneFixture in 9.722 seconds.
  Set up plone.app.testing.bbb.PloneTestCaseFixture in 2.628 seconds.
  Set up plone.app.testing.bbb.PloneTestCase:Functional in 0.000 seconds.


Error in test test_attacker_redirect (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Products.CMFFormController/Products/CMFFormController/tests/testRedirectTo.py", line 97, in test_attacker_redirect
    handle_errors=False,
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/Testing/ZopeTestCase/functional.py", line 43, in wrapped_func
    return func(*args, **kw)
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/Testing/ZopeTestCase/functional.py", line 127, in publish
    wsgi_result = publish(env, start_response)
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/ZPublisher/WSGIPublisher.py", line 254, in publish_module
    with load_app(module_info) as new_mod_info:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/Testing/ZopeTestCase/sandbox.py", line 73, in load_app
    with ZPublisher.WSGIPublisher.__old_load_app__(module_info) as ret:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/ZPublisher/WSGIPublisher.py", line 220, in load_app
    app = app_wrapper()
  File "/home/jenkins/workspace/plone-5.2-python-2.7-at/src/Zope/src/App/ZApplication.py", line 78, in __call__
    return connection.root()[self._name]
  File "/home/jenkins/shiningpanda/jobs/2fa08faf/virtualenvs/d41d8cd9/lib/python2.7/UserDict.py", line 40, in __getitem__
    raise KeyError(key)
KeyError: 'Application'



Error in test test_regression (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
Traceback (most recent call last):

[...]

    raise KeyError(key)
KeyError: 'Application'

  Ran 68 tests with 0 failures, 2 errors and 0 skipped in 1.626 seconds.
Running plone.app.folder.tests.layer.plone.app.folder testing:Integration tests:
  Set up plone.app.folder.tests.layer.IntegrationFixture in 0.027 seconds.
  Set up plone.app.folder.tests.layer.plone.app.folder testing:Integration in 0.000 seconds.
  Ran 27 tests with 0 failures, 0 errors and 0 skipped in 9.033 seconds.

[...]

Tearing down left over layers:
  Tear down zope.testrunner.layer.UnitTests in 0.000 seconds.
Total: 733 tests, 0 failures, 2 errors and 0 skipped in 3 minutes 10.739 seconds.
#### Finished tests for group Archetypes ####

What? Why does connection.root() have no Application? This makes no sense to me, and a pdb there did not help to shed light on it at all.

First I reproduced the error by testing all packages in the test group Archetypes (where the error occurs):

./bin/test \
  -s Products.Archetypes \
  -s Products.CMFFormController \
  -s Products.MimetypesRegistry \
  -s Products.PortalTransforms \
  -s Products.statusmessages \
  -s Products.validation \
  -s plone.app.folder

Then I only used the test layers that actually got set up according to the output:

./bin/test --layer Products.Archetypes.tests.attestcase.Archetypes \
           --layer Products.PortalTransforms.testing.PortalTransformsLayer \
           --layer Testing.ZopeTestCase.layer.ZopeLite \
           --layer plone.app.testing.bbb.PloneTestCase \
           -s Products.Archetypes \
           -s Products.CMFFormController \
           -s Products.MimetypesRegistry \
           -s Products.PortalTransforms \
           -s Products.statusmessages \
           -s Products.validation \
           -s plone.app.folder

That worked, I see the error. But I will not try to read 733 tests and wait for more than 3 minutes each time I think I may have fixed something!

Thus I used the divide-and-conquer strategy to figure out which combination produced the failing tests: remove half of the packages layers and see if it still fails. If they pass, try the other half. Do the same with the layers.

Remember to keep --layer plone.app.testing.bbb.PloneTestCase and -s Products.CMFFormController in order not to skip the tests that expose the issue.

It turned out that the following combination reproduced the issue:

./bin/test \
    --layer Products.Archetypes.tests.attestcase.Archetypes \
    --layer Testing.ZopeTestCase.layer.ZopeLite \
    --layer plone.app.testing.bbb.PloneTestCase \
    -s Products.Archetypes \
    -s Products.CMFFormController

Still way too many tests to have a look, most of them in Products.Archetypes. So I removed (actually, moved the .py files to some temp folder) all python tests and kept the doctests (and their setup). The only reason was that I hate doctests and consequently it must be a doctest that created trouble. I was right.

So I kept only one doctest that produced the issue by commenting out the others in test_doctest.py of Products.Archetypes.

Now I needed to find a combination of three tests from these layers that still exposed the issue. To to that, I added the option -vv to the testrunner to see the names and python path of all tests that still ran.

./bin/test --layer Products.Archetypes.tests.attestcase.Archetypes --layer Testing.ZopeTestCase.layer.ZopeLite --layer plone.app.testing.bbb.PloneTestCase -s Products.Archetypes -s Products.CMFFormController -vv
Running tests at level 1
Running Products.Archetypes.tests.attestcase.Archetypes:Functional tests:
  Set up plone.testing.zca.LayerCleanup in 0.000 seconds.
  Set up plone.testing.z2.Startup in 0.157 seconds.
  Set up plone.app.testing.layers.PloneFixture in 10.252 seconds.
  Set up plone.app.testing.bbb.PloneTestCaseFixture in 1.871 seconds.
  Set up Products.Archetypes.tests.attestcase.ATTestCaseFixture in 0.647 seconds.
  Set up Products.Archetypes.tests.attestcase.Archetypes:Functional in 0.000 seconds.
  Running:
    1/1 (100.0%) /Users/pbauer/workspace/coredev/src/Products.Archetypes/Products/Archetypes/tests/traversal_4981.txt

  Ran 1 tests with 0 failures, 0 errors, 0 skipped in 0.269 seconds.
Running Testing.ZopeTestCase.layer.ZopeLite tests:
  Tear down Products.Archetypes.tests.attestcase.Archetypes:Functional in 0.000 seconds.
  Tear down Products.Archetypes.tests.attestcase.ATTestCaseFixture in 0.010 seconds.
  Tear down plone.app.testing.bbb.PloneTestCaseFixture in 0.009 seconds.
  Tear down plone.app.testing.layers.PloneFixture in 0.065 seconds.
  Tear down plone.testing.z2.Startup in 0.004 seconds.
  Tear down plone.testing.zca.LayerCleanup in 0.001 seconds.
  Set up Testing.ZopeTestCase.layer.ZopeLite in 0.009 seconds.
  Running:
    1/5 (20.0%) test_parseXML_empty (Products.CMFFormController.tests.test_exportimport.CMFFormControllerImportConfiguratorTests)
    2/5 (40.0%) test_parseXML_with_info (Products.CMFFormController.tests.test_exportimport.CMFFormControllerImportConfiguratorTests)
    3/5 (60.0%) test_action_not_unicode (Products.CMFFormController.tests.test_exportimport.Test_importCMFFormController)
    4/5 (80.0%) test_normal (Products.CMFFormController.tests.test_exportimport.Test_importCMFFormController)
    5/5 (100.0%) test_partial (Products.CMFFormController.tests.test_exportimport.Test_importCMFFormController)

  Ran 5 tests with 0 failures, 0 errors, 0 skipped in 0.023 seconds.
Running plone.app.testing.bbb.PloneTestCase:Functional tests:
  Tear down Testing.ZopeTestCase.layer.ZopeLite in 0.000 seconds.
  Set up plone.testing.zca.LayerCleanup in 0.000 seconds.
  Set up plone.testing.z2.Startup in 0.092 seconds.
  Set up plone.app.testing.layers.PloneFixture in 7.227 seconds.
  Set up plone.app.testing.bbb.PloneTestCaseFixture in 2.087 seconds.
  Set up plone.app.testing.bbb.PloneTestCase:Functional in 0.000 seconds.
  Running:
    1/4 (25.0%) testCopy (Products.CMFFormController.tests.testCopyRename.TestCopyRename)
    2/4 (50.0%) testRename (Products.CMFFormController.tests.testCopyRename.TestCopyRename)
    3/4 (75.0%) test_attacker_redirect (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)


Error in test test_attacker_redirect (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/Users/pbauer/workspace/coredev/src/Products.CMFFormController/Products/CMFFormController/tests/testRedirectTo.py", line 97, in test_attacker_redirect
    handle_errors=False,
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/functional.py", line 43, in wrapped_func
    return func(*args, **kw)
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/functional.py", line 127, in publish
    wsgi_result = publish(env, start_response)
  File "/Users/pbauer/workspace/coredev/src/Zope/src/ZPublisher/WSGIPublisher.py", line 254, in publish_module
    with load_app(module_info) as new_mod_info:
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/sandbox.py", line 73, in load_app
    with ZPublisher.WSGIPublisher.__old_load_app__(module_info) as ret:
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/ZPublisher/WSGIPublisher.py", line 220, in load_app
    app = app_wrapper()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/App/ZApplication.py", line 78, in __call__
    return connection.root()[self._name]
  File "/Users/pbauer/workspace/coredev/bin/../lib/python2.7/UserDict.py", line 40, in __getitem__
    raise KeyError(key)
KeyError: 'Application'

    4/4 (100.0%) test_regression (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)


Error in test test_regression (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/Users/pbauer/workspace/coredev/src/Products.CMFFormController/Products/CMFFormController/tests/testRedirectTo.py", line 71, in test_regression
    handle_errors=False,
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/functional.py", line 43, in wrapped_func
    return func(*args, **kw)
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/functional.py", line 127, in publish
    wsgi_result = publish(env, start_response)
  File "/Users/pbauer/workspace/coredev/src/Zope/src/ZPublisher/WSGIPublisher.py", line 254, in publish_module
    with load_app(module_info) as new_mod_info:
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/Testing/ZopeTestCase/sandbox.py", line 73, in load_app
    with ZPublisher.WSGIPublisher.__old_load_app__(module_info) as ret:
  File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/ZPublisher/WSGIPublisher.py", line 220, in load_app
    app = app_wrapper()
  File "/Users/pbauer/workspace/coredev/src/Zope/src/App/ZApplication.py", line 78, in __call__
    return connection.root()[self._name]
  File "/Users/pbauer/workspace/coredev/bin/../lib/python2.7/UserDict.py", line 40, in __getitem__
    raise KeyError(key)
KeyError: 'Application'


  Ran 4 tests with 0 failures, 2 errors, 0 skipped in 0.403 seconds.
Tearing down left over layers:
  Tear down plone.app.testing.bbb.PloneTestCase:Functional in 0.000 seconds.
  Tear down plone.app.testing.bbb.PloneTestCaseFixture in 0.010 seconds.
  Tear down plone.app.testing.layers.PloneFixture in 0.068 seconds.
  Tear down plone.testing.z2.Startup in 0.007 seconds.
  Tear down plone.testing.zca.LayerCleanup in 0.001 seconds.

Tests with errors:
   test_attacker_redirect (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
   test_regression (Products.CMFFormController.tests.testRedirectTo.TestRedirectToFunctional)
Total: 10 tests, 0 failures, 2 errors, 0 skipped in 24.082 seconds.

24 seconds? I can work with that.

Still, I removed tests from each layer until I only had three tests left and reverted my changes to Products.Archetypes.

The result is the following:

./bin/test \
    --layer Products.Archetypes.tests.attestcase.Archetypes \
    --layer Testing.ZopeTestCase.layer.ZopeLite \
    --layer plone.app.testing.bbb.PloneTestCase \
    -s Products.Archetypes \
    -s Products.CMFFormController \
    -t test_parseXML_empty \
    -t traversal_4981 \
    -t test_attacker_redirect \
    -vv

Since more than one test still exposed the issue, I kept only very simple ones because I guessed that the issue is actually in the setup or teardown.

So next I changed the test test_parseXML_empty to a simple return. The error is still there. Trying the same with traversal_4981 makes it go away.

At this point I could skip reducing the layers since I only run three tests from two packages.

It was time to actually read what the remaining tests are doing. I stripped down all tests and their setup to the base minimum that still breaks the test run and could not find anything. I turn edCMFFormControllerImportConfiguratorTests into a ZopeTestCase and a PloneTestCase and realized that the error disappears when it is a PloneTestCase. Bad. Migrating the whole test to PloneTestCase or plone.app.testing would be a lot of work since CMFFormControllerImportConfiguratorTests inherits from Products.GenericSetup.tests.common.BaseRegistryTests and does a lot of additional magic.

So the test layers for the two tests that did not fail or error by themselves but triggered the issue in the failing tests (traversal_4981 and test_parseXML_empty) seemed to be out of the scope of what I could do so I took a closer look at the failing tests themselves. I quickly found that I hate them but what they do is actually quite simple. Why do I hate them? Because they use the publish method of ZopeTestCase.Functional. That method (and its evil doctest-cousin Testing.ZopeTestCase.zopedoctest.functional.http) are way too clever helper methods that make things harder, not easier. I prefer to use restrictedTraverse or the testbrowser any time since both are much closer to what actually happens in the application.

This was the moment when I decided to migrate the tests in question to proper plone.app.testing tests. It took me about 1 hour to create a pull-request which resolves the issue. The rest of the day was spent on a fruitless attempt to find the issue that must still be lurking somewhere between the three tests and their layers.

I hope that monster will never rear its ugly head again until CMFFormController is finally removed from the coredev. The PLIP 2092 by @esteele and me will remove the last remaining ControllerPageTemplates but there are some more left in Archetypes.

I fear it will be quite some time until all ZopeTestCase and PloneTestCase tests are migrated to plone.app.testing. The remaining happy thought is that many will not need to be migrated since they are part of Archetypes and will go awaaaaay with it.