Planet Plone - Where Developers And Integrators Write

Plone now supports Python 3

Posted by Starzel.de on November 29, 2018 09:21 AM

At the Plone Conference 2018 in Tokyo I gave a plenary talk in which I told the story how Plone got to support Python 3. Ten minutes before my talk Plone 5.2a1 was released. That release was the culmination of a 3-year long journey (or even 5 years if you start in Brazil). If you want to learn how we got there you should watch the talk.

Where are we now?

Plone 5.2 supports Python 3.7, 3.6 and 2.7. It is currently in alpha but the plan is to release a final version in February 2019 right after the Alpine City Sprint. You should seriously consider joining that sprint if you plan to migrate your own projects. Working together with the people who ported Plone to Python 3 will give you the know-how to succeed in your own migrations!

But in fact you should not wait until February to start working with Plone on Python 3 but start right now so you'll have enough time to migrate to Python 3 before January 1st 2020 (End of Life of Python 2).

To make testing and working with Python 3 easier some tools and addons were already working on Python 3.

The following essential development-tools are already ported:

Also the three addons that achieved the highest ranking during the latest poll at Ploneconf now work on Python 3:

You can also run the new frontend Volto with Plone on Python 3.

Open tasks

Before the final release of Plone 5.2 scheduled for February 2019 there are still a lot of things to do.

Here are the six most pressing issues:

  1. The upgrade- and porting-guide need to be completed.
  2. ZODB migration needs to be finished and documented.
  3. Much more addons need to be ported and released.
  4. We require performance tests.
  5. People who really need a replacement for FTP and/or WebDAV need to figure out how to do that.

There is still a lot to do. You can help Plone by coming to a sprint, testing it and fixing or reporting a bug.

Migrating to Python 3

From 1.1.2020 on you'll have to inform your clients that you are running their super-secure system on an unsupported version of Python. That might be ok if you're running a small website for a friend but for serious projects this simply will not do. Instead you should start to adopt Python 3 in your Plone projects now. You need to start early and plan ahead!

Here are the six steps you have to follow to upgrade to Python 3. Do not attempt to do all steps at once, instead you should work iteratively and even deploy to production in between steps.

1. Upgrade to Plone 5.2

You should still use Python 2.7 for this step. The changes between Plone 5.1 and Plone 5.2 are not huge but you will gain some great features including a new and much faster dropdown-navigation and a bootstrap-based ZMI.

The Upgrade-Guide for 5.2 still needs much more love. So if you encounter any issues please update that document and/or create a ticket in github.

2. Drop Archetypes and only use Dexterity

Archetypes is still supported on Plone 5.2 but only when running with Python 2.7. It is now officially deprecated and is not ported to Python 3. There has been a built-in migration for default types from Archetypes to Dexterity since 2013, and since 2015 there have been helpers and even a form that allows you to migrate your custom content.

3. Migrate your code to Python 3

Follow the documentation in to migrate your custom code and any addons that you need in your packages.

  • Make sure your code works in Python 2 and Python 3. If you are certain that you will not publish that code as a addon or reuse it in another project you can also drop support for Python 2. That will make porting and testing easier.
  • For most addons: Use python-modernize as described in the documentation.
  • For small addons you could even simply try to startup on Python 3 and and fix whatever fails (e.g. relative imports or invalid syntax)
  • Manually fix whatever python-modernize misses (e.g. relative imports)
  • Do the same with tests. You'll need a test-setup with tox and a test-matrix in travis that tests different Python versions. The setup in collective.ifttt is almost done and might be a good example.
  • If you have complex doctests consider migrating them to python tests since these are easier to get to pass in both Python 2 and 3. If you do want to keep them as doctests, change them so that the output for Python 3 is the default and use a Py23DocChecker to make the tests pass in Python 2 as well. See the example in plone.indexer.
  • Once all seems to work you need to test every single feature manually because even a high test coverage is never a guarantee that your code actually does what it should.

4. Migrate Addons and Dependencies

Now migrate any addons you use to Python 3 as explained above. The main difference is that now everyone who uses this addon benefits. You shoud not drop support for Python 2 for those!

I listed the addons that already run on Python 3 above. I'm sure that list will be outdated very soon. Before you start migrating an addon you need to check if there already is a branch that supports Python 3 and check if there is a ticket like the one for collective.easyform. If not create such a ticket and start the process.

5. Migrate your Database

ZODB itself is compatible with Python 3 but a database that was created in Python 2.7 cannot be used in Python 3 without being modified first. I will not go into details about that. You should read the documentation and see David Glick's talk.

The migration will probably require some downtime and in rare cases you might have to write your own mappings to tell zodbupdate how to handle your data.

6. Deploy on Python 3

Plone 5.2 will use WSGI with waitress as the default WSGI server. Alternatively you can use uwsgi or gunicorn. ZServer can only be used with Python 2.7.

Some deployment tools may not ported yet, others like ZRS or RelStorage already support Python 3. You should test your production setup early. There may be changes that you don't want to struggle with a week before a scheduled launch.

Summing up

Start that whole process as early as possible because you may need to deal with unexpected problems and addons that you did not know still used Archetypes. 2019 might be much busier for you than you expect.

Do not wait for a final release of Plone 5.2! Start the process of planning and testing a migration. Consinder hiring experts to help you if necessary.

Again: You can and you should work iteratively. That means you do not have to do everything in one step and one night:

  • Upgrade to 5.2 with Dexterity on Python 2 and deploy that early in 2019.
  • Then take some time to migrate and test your code and addons.
  • You can then deploy the Python 3-compatible code while still running Python 2.7.
  • Then you can migrate your existing database and deploy on Python 3.

FAQ

Here are some questions you might have:

What about Python 4?

That will not be a big issue since 4.0 will be a upgrade like from 3.7 to 3.8. The Python community simply don't want to release a Python 3.10.

When will Plone drop support for Python 2?

Not before we have a good reason. Maybe Plone 7? It would allow us to remove and clean up some code but we would not gain a lot.

What about Zope 5?

There are no real plans for that yet.

Will Archetypes be migrated to Python 3?

Please don't try to postpone the inevitable. We are too small a community to maintain two Python versions, two frontends and two content type frameworks.

How do we replace FTP/WebDAV?

I don't know, for WebDAV you might be able to use a middleware like WsgiDAV.

Why did you make all that effort and not 'simply' migrate to Guillotina?

Guillotina does not aim to be a replacement for Plone. Also it has no equivalent addon ecosystem, feature equality or migration path.

Can I now use async/await?

You can but it won't really help you since Plone and Zope are still fundamentally synchronous.

Why doesn't the Plone Foundation simply buy support for Python 2.7 from RedHat/IBM for all of us?

Are you kidding me?

Does Python 3 make Plone run faster?

Maybe. We still lack performance tests but my guess is that Plone 5.2 will run faster on Python 3 than on 2.

Can I run the same Database in Python 2 and 3?

No.

Is addon already ported?

No.

Plone supports the latest two major version. Does that extend to Python?

Plone's security update policy supports the latest two major versions (currently, that means the 5.x series and the 4.3.x series) with security updates. That guarantee does not extend to the programming language, the operating systen or the hardware you are running Plone on.

Will this work on Windows?

Very likely. Someone will have to test it.

I need help with migrating our projects to Python 3. Who can help me?

Talk to us at Starzel.de or ask other professionals with experience.

Warm words

Migrating Plone to Python 3 was quite a ride and a great community effort. I had to cope with some brain-melting changes in Zope, fight heartbreaking test-isolation issues and toil through an endless list of packages and tests. But I had the immense privilege and pleasure to do all of that together with some of the smartest and nicest people I know. Thanks to all of you!

Watch the full talk if you want to learn more about how Plone finally landed on Python 3.

Here are all videos from Ploneconf 2018: Day 1, Day 2, Day 3.

Plone finally supports Python 3!

Posted by Starzel.de on November 29, 2018 09:21 AM

At the Plone Conference 2018 in Tokyo I gave a plenary talk in which I told the story how Plone got to support Python 3. Ten minutes before my talk Plone 5.2a1 was released. That release was the culmination of a 3-year long journey (or even 5 years if you start in Brazil). If you want to learn how we got there you should watch the talk.

Where are we now?

Plone 5.2 supports Python 3.7, 3.6 and 2.7. It is currently in alpha but the plan is to release a final version in February 2019 right after the Alpine City Sprint. You should seriously consider joining that sprint if you plan to migrate your own projects. Working together with the people who ported Plone to Python 3 will give you the know-how to succeed in your own migrations!

But in fact you should not wait until February to start working with Plone on Python 3 but start right now so you'll have enough time to migrate to Python 3 before January 1st 2020 (End of Life of Python 2).

To make testing and working with Python 3 easier some tools and addons were already working on Python 3.

The following essential development-tools are already ported:

Also the three addons that achieved the highest ranking during the latest poll at Ploneconf now work on Python 3:

You can also run the new frontend Volto with Plone on Python 3.

Open tasks

Before the final release of Plone 5.2 scheduled for February 2019 there are still a lot of things to do.

Here are the six most pressing issues:

  1. The upgrade- and porting-guide need to be completed.
  2. ZODB migration needs to be finished and documented.
  3. Much more addons need to be ported and released.
  4. We require performance tests.
  5. People who really need a replacement for FTP and/or WebDAV need to figure out how to do that.

There is still a lot to do. You can help Plone by coming to a sprint, testing it and fixing or reporting a bug.

Migrating to Python 3

From 1.1.2020 on you'll have to inform your clients that you are running their super-secure system on an unsupported version of Python. That might be ok if you're running a small website for a friend but for serious projects this simply will not do. Instead you should start to adopt Python 3 in your Plone projects now. You need to start early and plan ahead!

Here are the six steps you have to follow to upgrade to Python 3. Do not attempt to do all steps at once, instead you should work iteratively and even deploy to production in between steps.

1. Upgrade to Plone 5.2

You should still use Python 2.7 for this step. The changes between Plone 5.1 and Plone 5.2 are not huge but you will gain some great features including a new and much faster dropdown-navigation and a bootstrap-based ZMI.

The Upgrade-Guide for 5.2 still needs much more love. So if you encounter any issues please update that document and/or create a ticket in github.

2. Drop Archetypes and only use Dexterity

Archetypes is still supported on Plone 5.2 but only when running with Python 2.7. It is now officially deprecated and is not ported to Python 3. There has been a built-in migration for default types from Archetypes to Dexterity since 2013, and since 2015 there have been helpers and even a form that allows you to migrate your custom content.

3. Migrate your code to Python 3

Follow the documentation in to migrate your custom code and any addons that you need in your packages.

  • Make sure your code works in Python 2 and Python 3. If you are certain that you will not publish that code as a addon or reuse it in another project you can also drop support for Python 2. That will make porting and testing easier.
  • For most addons: Use python-modernize as described in the documentation.
  • For small addons you could even simply try to startup on Python 3 and and fix whatever fails (e.g. relative imports or invalid syntax)
  • Manually fix whatever python-modernize misses (e.g. relative imports)
  • Do the same with tests. You'll need a test-setup with tox and a test-matrix in travis that tests different Python versions. The setup in collective.ifttt is almost done and might be a good example.
  • If you have complex doctests consider migrating them to python tests since these are easier to get to pass in both Python 2 and 3. If you do want to keep them as doctests, change them so that the output for Python 3 is the default and use a Py23DocChecker to make the tests pass in Python 2 as well. See the example in plone.indexer.
  • Once all seems to work you need to test every single feature manually because even a high test coverage is never a guarantee that your code actually does what it should.

4. Migrate Addons and Dependencies

Now migrate any addons you use to Python 3 as explained above. The main difference is that now everyone who uses this addon benefits. You shoud not drop support for Python 2 for those!

I listed the addons that already run on Python 3 above. I'm sure that list will be outdated very soon. Before you start migrating an addon you need to check if there already is a branch that supports Python 3 and check if there is a ticket like the one for collective.easyform. If not create such a ticket and start the process.

5. Migrate your Database

ZODB itself is compatible with Python 3 but a database that was created in Python 2.7 cannot be used in Python 3 without being modified first. I will not go into details about that. You should read the documentation and see David Glick's talk.

The migration will probably require some downtime and in rare cases you might have to write your own mappings to tell zodbupdate how to handle your data.

6. Deploy on Python 3

Plone 5.2 will use WSGI with waitress as the default WSGI server. Alternatively you can use uwsgi or gunicorn. ZServer can only be used with Python 2.7.

Some deployment tools may not ported yet, others like ZRS or RelStorage already support Python 3. You should test your production setup early. There may be changes that you don't want to struggle with a week before a scheduled launch.

Summing up

Start that whole process as early as possible because you may need to deal with unexpected problems and addons that you did not know still used Archetypes. 2019 might be much busier for you than you expect.

Do not wait for a final release of Plone 5.2! Start the process of planning and testing a migration. Consinder hiring experts to help you if necessary.

Again: You can and you should work iteratively. That means you do not have to do everything in one step and one night:

  • Upgrade to 5.2 with Dexterity on Python 2 and deploy that early in 2019.
  • Then take some time to migrate and test your code and addons.
  • You can then deploy the Python 3-compatible code while still running Python 2.7.
  • Then you can migrate your existing database and deploy on Python 3.

FAQ

Here are some questions you might have:

What about Python 4?

That will not be a big issue since 4.0 will be a upgrade like from 3.7 to 3.8. The Python community simply don't want to release a Python 3.10.

When will Plone drop support for Python 2?

Not before we have a good reason. Maybe Plone 7? It would allow us to remove and clean up some code but we would not gain a lot.

What about Zope 5?

There are no real plans for that yet.

Will Archetypes be migrated to Python 3?

Please don't try to postpone the inevitable. We are too small a community to maintain two Python versions, two frontends and two content type frameworks.

How do we replace FTP/WebDAV?

I don't know, for WebDAV you might be able to use a middleware like WsgiDAV.

Why did you make all that effort and not 'simply' migrate to Guillotina?

Guillotina does not aim to be a replacement for Plone. Also it has no equivalent addon ecosystem, feature equality or migration path.

Can I now use async/await?

You can but it won't really help you since Plone and Zope are still fundamentally synchronous.

Why doesn't the Plone Foundation simply buy support for Python 2.7 from RedHat/IBM for all of us?

Are you kidding me?

Does Python 3 make Plone run faster?

Maybe. We still lack performance tests but my guess is that Plone 5.2 will run faster on Python 3 than on 2.

Can I run the same Database in Python 2 and 3?

No.You can neither use a DB create with py2 in py3 nor use one created with py3 in py2.

Is addon xyz already ported?

Most likely not.

Plone supports the latest two major version. Does that extend to Python?

Plone's security update policy supports the latest two major versions (currently, that means the 5.x series and the 4.3.x series) with security updates. That guarantee does not extend to the programming language, the operating systen or the hardware you are running Plone on.

Will this work on Windows?

Very likely. Someone will have to test it.

I need help with migrating our projects to Python 3. Who can help me?

Talk to us at Starzel.de or ask other professionals with experience.

Warm words

Migrating Plone to Python 3 was quite a ride and a great community effort. I had to cope with some brain-melting changes in Zope, fight heartbreaking test-isolation issues and toil through an endless list of packages and tests. But I had the immense privilege and pleasure to do all of that together with some of the smartest and nicest people I know. Thanks to all of you!

Watch the full talk if you want to learn more about how Plone finally landed on Python 3.

Here are all videos from Ploneconf 2018: Day 1, Day 2, Day 3.

Thank you, Plone Conference 2018 Tokyo!

Posted by PLONE.ORG on November 17, 2018 08:57 PM

It is with a full heart that we bid farewell to the just ended Plone Conference 2018 Tokyo, and we offer our enduring gratitude to organizers Manabu Terada, Takeshi Yamamoto, Zenichiro Yasuda, Takanori Suzuki, Sakai, and Kinofumi for their wonderfully warm welcome and flawless event planning at the PiO conference centre.

Attendees came to Tokyo from 27 countries around the world! (Bangladesh, Belgium, Brazil, Catalonia, Denmark, Finland, France, Germany, India, Italy, Jamaica, Japan, Korea, Luxembourg, Mexico, Netherlands, Philippines, Poland, Romania, Slovenia, South Africa, Spain, Switzerland, Taiwan, Thailand, UK, USA) 

Over half of them took at least one of the free training classes covering beginner, intermediate, and expert topics, from Learn Plone to Mastering Plone Development, with by far the most interest focused on the React and Volto (Plone React) classes. 

Each day began with keynotes by Eric Steele, Jim Fulton, and Torajiro Aida, followed by 44 talks in four streams: three for Plone and one daily stream on Python Web, Database, and Frontend.

As is our tradition, the conference also included two days of sprints where again almost half our attendees worked in self-organizing teams to collaboratively contribute to the Plone project and community. (see the shared sprint document to find out what was worked on). At the sprint, it was with great pleasure that we conferred the Most Productive Sprinter trophy on first time sprinter Paul Grunewald!

Work presented at this year's conference showed that Plone's future is bright, with at least three future paths:

  • traditional Plone: Plone 5.2 running on Python 3 is well on its way, thanks to a huge effort by many, including the Zope community
  • headless Plone: a modern JavaScript Volto (Plone React) frontend with the new Pastanaga UI running against a Plone backend via REST API
  • Async Plone: Guillotina CMS

We are left with indelible memories

  • Plonistas making their way through the labyrinth of Tokyo commuter railway stations, phones in hand with open map apps
  • Akihabara's bright lights and towering stores full of anime and electronic goodies
  • Visiting temples, castles, and shrines
  • Poking our heads into tiny bars, restaurants, and karaoke bars

Look for photos and tweets tagged with #ploneconf2018. (talk videos and slides coming soon)

We look forward to seeing you at sprints and events in 2019!

ありがとうございました Arigatō gozaimashita! 

Plone Conference 2019 will be in Ferrara, Italy

Posted by PLONE.ORG on November 12, 2018 06:37 PM

Following on the heels of the successful, inspiring Tokyo Plone Conference 2018, the Plone Foundation Board is very pleased to announce that Plone Conference 2019 will be in Ferrara, Italy!

Located in northern Italy, near Bologna, Venice, and Milan, Ferrara is the home of RedTurtle, a long time provider of Plone solutions.

Stay tuned for further details!

What next(1).jpg

What next(2).jpg

What next(3).jpg

What next(4).jpg

Plone on Python 3 is Closer to Reality Thanks to Huge Sprint

Posted by PLONE.ORG on November 12, 2018 05:31 PM

In October 2018, 34 Plone developers joined forces for a major push to move Plone to Python 3, implement improvements in Zope 4, and bring Plone 5.2 closer to a final release during a sprint held in the gocept offices in Halle, Gemany. 

Given the large number of people on hand for the sprint, a huge list of achievements were reported:

Read the full report here.

The attendees celebrated Plone's 17th birthday with a barbecue and party in true Plone style. Thank you to Philip Bauer for organizing.

The organizers want to thank gocept for the use of their amazing offices and allowing the community to take advantage of the palace-like salon, cafe Kaffej, and the excellent cappuccino.

Our appreciation also goes out to Plone Foundation, and the German Python Software Foundation (PySV) for their support.

Thank you to the amazing attendees for their hard work during this intense 5 day sprint.

Finally, we could not accomplish these things without the sprinters, so a huge thank you to them on behalf of the entire Plone community!

 

Sorrento Sprint on Frontend Modernization

Posted by PLONE.ORG on November 07, 2018 08:12 AM

Our perennial Plone Open Garden sprint returns for 2019!

Date: April 7-14, 2019

Location: Hotel Mediterraneo

Registration deadline: February 28, 2019

Price per day (includes breakfast and dinner):

  • Double room for 1 person: 150 €
  • Double room for 2 people: 160 €
  • Triple room for 3 people: 180 €

The focus of this sprint will be to bring Plone more in line with what front end developers and content editors expect in a CMS these days. We will emphasize efforts that will improve the experience for both traditional Plone (5) and future Plone (Volto, Guillotina, etc.) Wherever possible, we will strive to make features available in both ecosystems. We hope to attract both traditional Plone and future Plone developers and provide a venue and topics that will encourage cross-fertilization. The goal of the sprint is to focus on efforts that will have high impact for both developers/integrators and non-technical users.

Subtopics:

  • Volto aka plone-react
  • Content quality checking
  • Accessibility improvements
  • Contextual image alt text
  • Improved accessibility for overlays, dialogs, and menus
  • ATAG 2.0 Part B compliance: single page and site-wide compliance checkers
  • Incorporate a11y checking into travis/jenkins process
  • Parity between Plone 5 and Volto for accessibility features
  • Supporting service workers for improved performance
  • Plone 5 UX improvements

Additional topic suggestions from the community are welcome!

Follow up discussion will be in our community forum

Random News October 2018

Posted by TestTheDocs on October 24, 2018 11:15 AM
Community Overcome Fragmentation For a long time (since the beginning) we used IRC and later Gitter for our communication. This was mainly because we are used to this tools and never kept in mind that these maybe not the “most user friendly” tools around anymore. We also placed ourself into a corner, because it was hard for others to find or get in contact. Because of this we decided to switch to Slack, we are still not huge fans, but this should make things hopefully a bit easier and less fragmented.

Help Oshane get to the Tokyo Plone Conference

Posted by David "Pigeonflight" Bain on October 24, 2018 03:38 AM

Summary

Oshane Bailey, a talented Plone developer with loads of Plone experience has been selected as a presenter for the 2018 Tokyo Plone Conference. His Japanese visa was just approved.
He will share a streamlined approach to Plone development that he is applying on a Plone project targeted at Jamaican Developers. At the time of writing he has raised about 17% of the funds he needs to get to Tokyo. You can help him get to Japan by contributing to his crowd-funding campaign.

Oct 29, 2018 update
Thanks to generous contributions, Oshane's trip is now 70% funded. You are welcome to join the crowdfund and cover the rest of his trip.

Oct 30, 2018 update

Oshane's trip is now 89% funded. The plane ticket and conference ticket have been purchased. you can still pitch in by joining the crowdfund to cover the rest of his trip.

Since at least 2015, Oshane has worked on Plone projects for teams around the world and in the process has been exposed to varied approaches to the development and ongoing management of Plone sites. Over recent months he has poured his, hard earned, experience, into a side project -- the Jamaican Developers site.  Through this project he has refined a continuous development pipeline based on some of the best techniques used in the Plone community and enhanced with some of his own innovations.

Last year Oshane participated as a Plone Google Summer of Code student and presented his work at the Barcelona conference. He also participated in the after-conference sprints, contributing to efforts to port Plone to Python 3 and also looking into the WebSauna project.

Supporting his trip to Tokyo will serve to enrich PloneConf 2018 in many ways. Here are three that spring immediately to mind:
1) As part of his talk he will share the techniques he is using on the Jamaican Developers site
2) He plans to participate in the after conference sprints.
3) He will bring an important perspective to discussions influenced by constraints common to Jamaican developers.

How to Support Oshane

Appropriately, his crowdfunding campaign is running on the Jamaican Developers site that he built with Plone. His goal is to raise enough to cover his travel and expenses related to the Japan trip.

As we say in Jamaica... "Follow back a me" as I support Oshane's trip to PloneConf2018 in Tokyo.

Python 3, Plone 5.2, Zope 4 and more

Posted by Starzel.de on October 19, 2018 09:14 AM

With over 30 attendees the SaltLabs Sprint was among the bigger sprints in the history of Plone and Zope.

Fortunately, our host Gocept just moved into a new office and had plenty of space to spare. Stylish as we are, the Plone crowd quickly congregated in the palace-like salon with red walls and renaissance paintings. Even better: Throughout the week the cafe Kaffej downstairs was open for us to eat breakfast, lunch and for work. We tried to keep the barista busy producing excellent Cappuccino. Thanks!

Zope 4

The sprint split in two groups, one working on Zope and one working on Plone. A short report on about what happened at the Zope can be found at https://blog.gocept.com/2018/10/11/beta-permission-for-earl-zope-extended . Zope 4.0b6 was released just after the sprint. The new Beta 6 includes the new bootstrap-styled ZMI, support for Python 3.7 and a metric ton of bug fixes compared to 4.0b5 which was released in May this year. By the way: Zope can now be installed without buildout using pipenv

Python 3

The Plone crowd was huge and worked on several tasks. The biggest task - and the main reason to join the sprint - was continuing to port Plone to Python 3. Jens Klein, Peter Mathis, Alessandro Pisa, David Glick (remote) and Philip Bauer as lead worked mainly on fixing all remaining failing tests and getting the jenkins-builds to pass. We dove deep into crazy test-isolation-issues but also added a couple of missing features to Dexterity that were so far only tested with Archetypes.

During the sprint we got very close to a green build but a couple of really nasty test-isolation issues prevented a green build. A week after the sprint we finally got that done and all tests of Plone now pass in Python 2.7, Python 3.6 and Python 3.7 with the same code-base!

With the test passing we're now merging that work and preparing the release of Plone 5.2a1. The tasks for this are discussed in this ticket

Deprecate Archetypes

Archetypes will not be ported to Python 3 but still has to work when running Plone 5.2 in Python 2.7. Since all tests that used PloneTestCase were still using Archetypes Philip changed that to use Dexterity now and created a new testlayer (plone.app.testing.bbb_at.PloneTestCase) that is now used by all the packages using the Archetypes-stack. See https://github.com/plone/plone.app.testing/pull/51. You could use this layer in your archetypes-based addon-packages that you want to port to Plone 5.2.

Addons

Yes, Plone is now able to run on Python 3. But without the huge ecosystem of addons that would be of limited real-world use. Jan Mevissen, Franco Pellegrini (remote) and Philip Bauer ported a couple of addons to Python 3 that Plone developers use all the time:

Being able to run tests against Plone in Python 3, and with these development-tools now available also, it is now viable to start porting other addons to Python 3. To find out how hard that is I started porting collective.easyform. The work is only 97% finished since some tests need more work but it is now useable in Python 3!

ZODB Migration

Harald Friesenegger and Robert Buchholz worked on defining a default migration-story for existing databases using zodbupdate. To discuss the details of this approach we had a hangout with Jim Fulton, David Glick and Sylvain Viollon. They solved a couple of tricky issues and wrote enough documentation. This approach seems to work well enough and the documentation points out some caveats but the ZODB migration will require some more work.

Relevant links:

ZODB Issues

Sune Broendum Woeller worked on a very nasty and complex issue with KeyErrors raising while releasing resources of a connection when closing the ZODB. That happened pretty frequently in our test-suites but was so far unexplained. He analyzed the code and finally was able to add a failing test to prove his theory. Then Jim Fulton realized the problem and wrote a fix for it. This will allow us to update to the newest ZODB-version once it is released. See https://github.com/zopefoundation/ZODB/issues/208 for details.

WSGI-Setup

Thomas Schorr added the zconsole module to Zope for running scripts and an interactive mode. Using WSGI works in Python 2 and Python 3 and will replace ZServer in Python 3.

Frontend and Theming

Thomas Massman, Fred van Dijk, Johannes Raggam and Maik Derstappen looked into various front-end issues, mainly with the Barceloneta theme. They closed some obsolete tickets and fixed a couple of bugs.

They also fixed some structural issues within the Barceloneta theme. The generated HTML markup now has the correct order of the content columns (main, then left portlets, then right portlets) which allows better styling for mobile devices. Also in the footer area we are now able to add more portlets to generate a nice looking doormat. See the ticket https://github.com/plone/plonetheme.barceloneta/pull/163 for screenshots and details.

They also discussed a possible enhancement of the Diazo based theming experience by including some functionality of spirit.plone.theming into plone.app.theming and cleaning up the Theming Control Panel. A PLIP will follow for that.

New navigation with dropdown support

Peter Holzer (remote) continued to work on the new nativation with dropdown-support which was started on collective.navigation by Johannes Raggam. Due to the new exclude_from_nav index and optimized data structures the new navigation is also much faster than other tree based navigation solutions (10x-30x faster based on some quick tests).

Static Resource Refactoring

Johannes Raggam finished work on his PLIP to restructure static resources. With this we no longer carry 60MB of static resources in the CMFPlone repository, it allows to use different versions of mockup in Plone, enable us to release our libraries on npm and will make it easier to switch to a different framework. Skin scripts removal Katja Süss, Thomas Lotze, Maurits van Rees and Manuel Reinhardt worked hard at removing the remaining python_scripts. The work is still ongoing and it would be great to get rid of the last of these before a final Plone 5.2 release. See https://github.com/plone/Products.CMFPlone/issues/1801 for details. Katja also worked on finally removing the old resource registry (for js and css).

Test parallelization

Joni Orponen made a lot of progress on speeding up our test-runs by running different test-layers in parallel. The plan is to get them from 30-60 minutes (depending on server and test-setup) to less than 10 minutes. For a regularly updated status of this work see https://community.plone.org/t/ci-run-speedups-for-buildout-coredev/6225

Documentation and User-Testing

Paul Roeland fought with robot-tests creating the screenshots for our documentation. And won. A upgrade guide to Plone 5.2 and Python 3 was started by several people. Jörg Zell did some user-testing of Plone on Python 3 and documented some errors that need to be triaged.

Translations

Jörg Zell worked on the german translations for Plone and nearly got to 100%. After the sprint Katja Süß did an overall review of the german translation and found some wording issues with need for a discussion

Plone React

Rob Gietema and Roel Bruggink mostly worked on their trainings for React and Plone-react, now renamed to “Volto”. Both will be giving these trainings at Plone Conference in Tokyo. On the second day of the sprint Rob demoed the current state of the new react-based frontend for Plone.

Assorted highlights

  • In a commit from 2016 an invisible whitespace was added to the doctests of plone.api. That now broke our test in very obscure ways. Alessandro used some dark magic to search and destroy.
  • The __repr__ for persistent objects changed breaking a lot of doctests. We still have to figure out how to deal with that. See https://github.com/zopefoundation/zope.site/issues/8 for details.
  • There was an elaborate setup to control the port during robot-tests. By not setting a port at all the OS actually takes care of this to makes sure the ports do not conflict. See https://github.com/plone/plone.app.robotframework/pull/86. Less is sometimes more.
  • Better late than never: We now have a method safe_nativestring in Products.CMFPlone.utils besides our all-time favorites safe_unicode and safe_encode. It transforms to string (which is bytes in Python 2 and text in Python 3). By the way: There is also zope.schema.NativeString and zope.schema.NativeStringLine.
  • We celebrated the 17th birthday of Plone with a barbecue and a generous helping of drinks.

Tokyo Plone Conference 2018 Talks and Training Classes

Posted by PLONE.ORG on October 18, 2018 07:35 PM

See the Tokyo Plone Conference 2018 talks list, covering a wide range of topics:

  • modern JavaScript frontends, including ReactJS, Angular, Volto, GatsbyJS
  • Zope
  • Pyramid
  • Plone on Python 3
  • headless Plone / Plone API / Guillotina / Guillotina CMS
  • testing and continuous integration
  • current practices for Plone development and deployment
  • Plone theming

Training classes are included with your conference registration – see the training class schedule and class descriptions

Join us in Tokyo next month! Registration and details at https://2018.ploneconf.org

A conference sponsorship would be a great way to show your support for Plone and to enhance your visibility in the community! Find out more about conference sponsorships

Hello Kitty

Photo by Miguel Á. Padriñán from Pexels

Plone 5.1.4 has been released

Posted by PLONE.ORG on October 17, 2018 02:29 PM

Plone 5.1.4 has been released, containing bug fixes and updated packages.

To download it, please see the download page.


Changes

plone.recipe.alltests: 1.5 → 1.5.1

Bug fixes:
  • Fix imports when buildout was installed using pip. [davisagli]

plone.recipe.zope2instance: 4.4.0 → 4.4.1

Bug fixes:
  • Make comments in zcml values work, even if not starting at the beginning of the line; before, we had a confusing error message. Fixes #46 [tobiasherp]

plone.releaser: 1.6.0 → 1.7.0

plone.versioncheck: 1.6.5 → 1.6.10

Bug fixes:
  • Better handling of relative paths as entry, like project/foo/dev.cfg. [jensens]

  • If a file does not extend any other file, the return statement was returning only one parameter, while callers expected 2. [gforcada]

  • Fix: Do not trust on setuptools internals. Works now with newest setuptools. [jensens]

  • Fixes another bug in find_relative. [jensens]

  • Fixes bug in find_relative introducued in last release. [jensens]

  • Fixed: Inherited extends with same name showed up as same. Now show relative to basedir if possible, else full. [jensens]

  • Fixed: Relative extends in urls were broken. [jensens]

plone.app.robotframework: 1.2.1 → 1.3.0

Breaking changes:
  • Remove compatibility code for Plone 4.x. [hvelarde]

  • Skip Support for Plone 4.x. [jensens]

Bug fixes:
  • Add support for Python 3. [davisagli]

  • Support for Plone 5.x toolbar, i18n, ... [jensens]

  • Fix RemoteLibrary i18n to work proper with registry based settings. [jensens]

  • Make robotframework work with merged plone.login. [jensens]

  • Fix Travis CI setup. [jensens]

Pillow: 4.3.0 → 5.2.0

python-dateutil: 2.6.1 → 2.7.3

borg.localrole: 3.1.5 → 3.1.6

Bug fixes:
  • Python 2 / 3 compatibility. [ale-rt, pbauer]

  • Fix issue StopIteration raising a Runtimeerror in Python 3.7 See https://www.python.org/dev/peps/pep-0479/ [pbauer]

diazo: 1.2.9 → 1.2.10

Bug fixes:
  • Fix diazo.debug console with Python 2.7. [pbauer]

icalendar: 4.0.2 → 4.0.3

Bug fixes:
  • Categories are comma separated not 1 per line #265. [cleder]

  • mark test with mixed timezoneaware and naive datetimes as an expected failure. [cleder]

mockup: 2.7.4 → 2.7.6

New features:
  • pat-structure: Use more tooltips in the actions menu. [thet]

  • pat-structure: Move breadcrumbs out of folder contents table. [thet]

  • pat-structure: Use the datatables pattern for sorting columns.

  • Include TinyMCE 4.7.13 [erral]

  • pat-structure: Use the datatables pattern for sorting columns [frapell]

  • pat-structure: Add a '250' option for the pagination. [frapell]

  • Add a new pattern, to provide the DataTables functionality. https://datatables.net/ [frapell]

  • Add package-lock.json to git. See https://github.com/npm/npm/blob/v5.0.0/doc/files/package-lock.json.md [sunew]

Bug fixes:
  • Structure pattern: Fix URL for current path for popovers. Refs: https://github.com/plone/mockup/issues/857 [thet]

  • Structure pattern: Fix spacing for attribute-columns popover. [tmassman]

  • TinyMCE: Do not fail if initializing with another language or external script https://github.com/plone/Products.CMFPlone/issues/2425 [frapell]

  • Get rid of obsolete X-UA-Compatible header. [hvelarde]

  • Fix small message typos. [tkimnguyen]

  • Remove patternslib "This pattern without a name attribute will not be registered!" warnings by setting dummy name and trigger properties. [sunew]

  • Fix error when running tests: "Some of your tests did a full page reload!", due to a form submit with no preventDefault. [sunew]

  • Fix WARN [watcher]: Pattern "..../patterns/foo.js" does not match any file, by excluding all requirements ending with '-url', since they point to a folder and not to a .js file. The folder contents are already included by 'patterns/*/'. [sunew]

  • Move installation and config of sinon from bower+requirejs to karma-sinon. [sunew]

  • Remove chai - it is unused. The assertions being used are from 'expect', installed via bower.json [sunew]

  • Upgrade some build and testing tools: grunt, karma, mocha, their plugins, coveralls, and browser launchers. [sunew]

Plone: 5.1.3 → 5.1.4

New features:
  • Release Plone 5.1.4 [esteele]

plone.api: 1.8.4 → 1.9.0

New features:
  • Python 2/3 support. [pbauer]
Bug fixes:
  • Removed allow-hosts from base.cfg, so we can use the new pypi warehouse. Refs https://github.com/plone/plone.api/issues/403 [jaroel]

  • fix typos in doc strings [tkimnguyen]

  • Fix failing AT Collection creation when using api.content.create. [gbastien]

plone.app.caching: 1.2.21 → 1.2.22

New features:
  • Python 3 support [pbauer, MatthewWilkes, ale-rt]

plone.app.content: 3.5.3 → 3.5.4

Bug fixes:
  • Fix namechooser encoding issue [tomgross]

  • fc_workflow view now reindex the object after state change. This fix the problem that some index dates (effective and modified for example) was updated only when the state was changed from the toolbar. [cekk]

  • Fix various issues in with py3. [pbauer]

plone.app.contentmenu: 2.2.3 → 2.2.4

Bug fixes:
  • Fix sorting of portletmanager-menuitems in py3. [pbauer]

plone.app.contenttypes: 1.4.11 → 1.4.15

Bug fixes:
  • Really revert breaking behavior marker interface changes in 1.4.12. [maurits]

  • The Format accessor should actually return the format attribute (see plone/Products.CMFPlone#2540) [ale-rt]

  • Revert breaking changes in 1.4.12 and move them to 2.0.0 branch of plone.app.contenttypes. [esteele]

  • Fix folder layout property migration. The default listing_view layout was always set if a folder didn't have a layout property. Also a default_page property could be inherited from parent folders or the Plone Siteroot, causing 'front-page' default_pages on many folders. Now only a direct layout property is copied and in that case on the local default_page if set is copied again. see issue 444 <https://github.com/plone/plone.app.contenttypes/issues/444> [fredvd]

  • Fixed false implemented Factories and Markers for ILeadImage and IRichText. see issue 457 <https://github.com/plone/plone.app.contenttypes/issues/476> [iham]

  • Fixed Tests for collection and migration. see issue <https://github.com/plone/plone.app.contenttypes/issues/477> [iham]

  • Pinned pydocstyle as it broke buildout. [iham]

  • pep8 cleanup. [iham]

  • Fix various issues in py3 [pbauer]

plone.app.dexterity: 2.4.10 → 2.5.1

New features:
  • Add plone.schema>=1.1.0 as minimal dependency in setup.py. [timo]

  • Add JSONField that has been added to plone.schema 1.1.0 to ALLOWED_FIELDS in types.py. [timo]

Bug fixes:
  • Remove bbb for MessageFactory import. [jensens]

  • Python 3 fixes. [jensens, pbauer]

plone.app.discussion: 3.0.6 → 3.0.8

Bug fixes:
  • Reverted test fix for richtext behavior. The related change was reverted in plone.app.contenttypes 1.4.15. [maurits]

  • Fixed tests when IRichText behavior is used. IRichText -> IRichTextBehavior This is a follow up to issue 476 <https://github.com/plone/plone.app.contenttypes/issues/476>_. [iham]

plone.app.event: 3.1.1 → 3.2.0

New features:
  • Remove Pragma: no-cache header for icalendar exports. The Pragma header is HTTP 1.0 and the caching strategy on icalendar exports should better be defined by integrators. [thet]
Bug fixes:
  • Python 3 compatibility. [pbauer]

  • Make sure to include the 'Content-Length' header for ical exports [frapell]

  • Update plone.app.event resources. Requires plonetheme.barceloneta >= 1.9. [agitator]

plone.app.folder: 1.2.5 → 1.2.6

plone.app.i18n: 3.0.4 → 3.0.5

Bug fixes:
  • Fix tests with py3. [pbauer]

plone.app.iterate: 3.3.8 → 3.3.9

Bug fixes:
  • Some Python 3 fixes. [pbauer]

plone.app.layout: 2.8.1 → 2.8.2

New features:
  • Add test for Revert to Revision action in History Viewlet [davilima6]
Bug fixes:
  • Show document byline for logged-in users. Fixes #160 [erral]

  • Get rid of obsolete X-UA-Compatible header. [hvelarde]

plone.app.linkintegrity: 3.3.6 → 3.3.7

Bug fixes:
  • Fix tests in Python 3 [pbauer]

plone.app.locales: 5.1.9 → 5.1.10

  • Update Traditional Chinese translations. [l34marr]

  • Save Lithuanian plonelocales with utf-8 encoding. Refs #234 [pysailor]

plone.app.multilingual: 5.2.1 → 5.2.3

New features:
  • Make plone.app.controlpanel optional (no longer there in Plone 5.2). [jensens]
Bug fixes:
  • Rerelease, as 5.2.1 was somehow released twice, once in June, once in September. [maurits]

  • Upgrade step to profile version 3 was lost and now recreated. [jensens, 2silver]

  • Do not show deprecation warning when loading migrator code, as it is intended to load old LRF there. [jensens]

  • Don't fail, if multilingual selector is called without query [tomgross]

  • Fix connecting of documents [tomgross]

plone.app.portlets: 4.3.3 → 4.3.4

Bug fixes:
  • Portlet add and edit forms already extend AutoExtensibleForm from plone.autoform. But some portlet addforms fail on creating the Assignment, if there is a FormExtender for the form, and the addform uses Assignment(**data) for creation instead of explicit parameters. Fix this by filtering away data values that does not come from the 'core' schema. [sunew]

plone.app.querystring: 1.4.8 → 1.4.9

Bug fixes:
  • Fix a case, where objects id starts with roots id [eikichi18]

plone.app.testing: 5.0.8 → 5.0.9

Bug fixes:
  • Amended the doctests to work with automatical layer port picking from plone.testing. [Rotonen]

plone.app.theming: 2.0.4 → 2.0.5

New features:
  • Recompiled resource bundles with latest mockup [sunew]

plone.app.upgrade: 2.0.15 → 2.0.17

plone.app.vocabularies: 4.0.6 → 4.0.7

Bug fixes:
  • Python 3 support [ale-rt, pbauer]

plone.app.z3cform: 3.0.5 → 3.0.6

Bug fixes:
  • Prepare for Python 2 / 3 compatibility [pbauer, MatthewWilkes, ale-rt]

plone.autoform: 1.7.5 → 1.8.0

New features:
  • Add support for python 3 [pbauer]

plone.batching: 1.1.3 → 1.1.4

Bug fixes:
  • Fix navlist different in Python2 and Python 3 (Refs. #21) [ale-rt]

  • Python 3 fixes. [thet]

plone.browserlayer: 2.2.1 → 2.2.2

Bug fixes:
  • Fix tests for py3 [pbauer]

plone.cachepurging: 1.0.15 → 1.0.16

Bug fixes:
  • Set default purger backlog size to 0 (infinity) in order to fully invalidate Varnish cache [avoinea refs #11]

  • Tests and Code are Python 3 compatible [pbauer, ale-rt, jensens]

plone.caching: 1.1.2 → 1.2.0

New features:
  • Add support for Python 3. [pbauer]
Bug fixes:
  • Fix caching and tests in python 3 [ale-rt, pbauer]

plone.contentrules: 2.0.7 → 2.0.8

Bug fixes:
  • Fix tests in py3. [pbauer]

plone.dexterity: 2.6.0 → 2.6.1

New features:
  • ZServer is now optional [pbauer]
Bug fixes:
  • Other Python 3 compatibility fixes [ale-rt, pbauer, jensens]

plone.folder: 2.0.0 → 2.0.1

Bug fixes:
  • Stabilize order of unordered items in partial ordering. [davisagli]

  • Fix ordering of content in folder in python 3. [pbauer]

  • Fix tests in py3. [pbauer]

plone.formwidget.namedfile: 2.0.5 → 2.0.6

Bug fixes:
  • Prepare for Python 2 / 3 compatibility [ale-rt, pbauer]

  • On form validation errors after submitting, keep the already uploaded image. [thet]

plone.indexer: 1.0.4 → 1.0.5

Bug fixes:
  • fix https://github.com/plone/Products.CMFPlone/issues/2469: "Subobjects are indexing attributes of parent". Allow only direct attributes and acquired PythonScripts, but not acquired attributes. Indexers and PythonScripts are able to handle this explicitly, because they get the acquisition-wrapped object. [jensens]

  • Fix tests to work in Python 3 [pbauer]

plone.locking: 2.2.2 → 2.2.3

Bug fixes:
  • More Python 2 / 3 compatibility fixes. [thet]

plone.memoize: 1.2.2 → 1.2.3

Bug fixes:
  • Re-enable Travis-CI. This package can be used outside Plone so it should be tested outside, too. [howitz]

plone.portlet.static: 3.1.2 → 3.1.3

Bug fixes:
  • Fix static portlet for py3 [pbauer]

plone.resourceeditor: 2.1.1 → 2.1.2

Bug fixes:
  • Fix functionality and tests in py3 [pbauer]

plone.scale: 3.0.1 → 3.0.2

Bug fixes:
  • Fix cleanup of image scales in py3 [pbauer]

plone.schema: 1.0.0 → 1.2.0

New features:
  • Improve and complete Plone integration of the JSONField (z3c.form, plone.supermodel, plone.schemaeditor) [sneridagh]

  • Add new JSONField field and JSONSchema auto validation. [sneridagh]

plone.schemaeditor: 2.0.19 → 2.0.20

Bug fixes:
  • Fix issue where serializing vocabulary terms into '|' syntax failed when values contained non-ascii characters [datakurre]

  • Allow defaults to be set on dexterity type fields via the web UI (@@fields) [ezvirtual]

  • Fix tests in py3 and some py3 issues. [pbauer]

plone.session: 3.7.0 → 3.7.1

Bug fixes:
  • Python3 compatibility [ale-rt, pbauer]

plone.stringinterp: 1.2.1 → 1.3.0

New features:
  • Add support for Python 3. [pbauer]

plone.subrequest: 1.8.5 → 1.8.6

New features:
  • Fix importsi without ZServer [pbauer]
Bug fixes:
  • More Python 3 compatibility. [ale-rt, thet]

  • Make test dependency on Archetypes optional. [davisagli]

plone.supermodel: 1.3.4 → 1.4.0

New features:
  • Python 3 compatibility. [pbauer, davilima6, dhavlik, ale-rt]

plone.testing: 4.3.1 → 4.3.2

Breaking changes:
  • Default to picking a dynamical port for ZServer layers instead of a static default port. [Rotonen]

plonetheme.barceloneta: 1.8.1 → 1.9.1

New features:
  • Integrated plone.app.event styles and further cleanup. [agitator]
Bug fixes:
  • Do not strip the #edit-bar section from the toolbar. Refs: https://github.com/plone/Products.CMFPlone/issues/2322 [thet]

  • Fix nested navigation tree and icon visibility [petschki]

Products.CMFDiffTool: 3.2.0 → 3.2.1

Bug fixes:
  • Fix diff and tests in py3 [pbauer, hverlarde, jensens]

Products.CMFEditions: 3.2.1 → 3.2.2

Bug fixes:
  • Fix Unauthorized error due to importing six inside Restricted Python Plone issue 2463 <https://github.com/plone/Products.CMFPlone/issues/2463>_ [davilima6]

Products.CMFPlone: 5.1.3 → 5.1.4rc5

New features:
  • Added utils.human_readable_size. https://github.com/plone/Products.CMFPlone/issues/1801 [reinhardt]

  • Update mockup. [thet]

  • Upgrade grunt + plugins to same versions as in mockup https://github.com/plone/mockup/pull/870 [sunew]

  • Include TinyMCE 4.7.13 [erral]

  • Update mockup to latest version. [frapell]

Bug fixes:
  • Fix profile version. [esteele]

  • Reverted test fix for richtext behavior. [maurits]

  • Fix multiple nested #edit-bar section when navigating in folder contents. Fixes: https://github.com/plone/Products.CMFPlone/issues/2322 [thet]

  • added CSS hyphenation support for toolbar for avoiding ugly text wrapping Fixes issue 723 <https://github.com/plone/Products.CMFPlone/issues/723> and issue 2315 https://github.com/plone/Products.CMFPlone/issues/2315. [ajung]

  • Remove the devdependencies from bower.json - they are just used for running tests in mockup, not here. [sunew]

  • Let the combine-bundles import step also work when the IBundleRegistry keyword is not in registry.xml, but in a registry directory. Issue 2520 <https://github.com/plone/Products.CMFPlone/issues/2502>_. [maurits]

  • Get rid of obsolete X-UA-Compatible header. [hvelarde]

  • Fix registration of robots.txt browser view to avoid AttributeError on Zope's root (fixes #2052 <https://github.com/plone/Products.CMFPlone/issues/2052>_). [hvelarde]

  • Fixed tests when IRichText behavior is used. IRichText -> IRichTextBehavior This is a follow up to issue 476 <https://github.com/plone/plone.app.contenttypes/issues/476>_. [iham]

  • Fix plone.app.redirector support for JSON/unspecified requests. [rpatterson]

  • Do not include too new upgrades when upgrading Plone Site. Otherwise the Plone Site ends up at a newer version that the filesystem code supports, giving an error when upgrading, and resulting in possibly missed upgrades later. Fixes issue 2377 <https://github.com/plone/Products.CMFPlone/issues/2377>_. [maurits]

  • Add test for issue #2469. [jensens]

  • Fix toolbar on mobile:

    • Clicking on menu links, submenus are not shown
    • With a opened submenu, html has huge margins and page content disappears [nzambello]
  • Clicking on menu links, submenus are not shown

  • With a opened submenu, html has huge margins and page content disappears

  • Remove last legacy Javascript highlight-searchterms.js. Removes also the skins folder plone_ecmascript. It was broken for almost all use cases (Google, other search engines, own live search); JS worked only when coming from Plone detailed search page. [jensens]

  • Fix failing thememapper robot test after rebuild of thememapper bundle in p.a.theming PR 147 [sunew]

  • Fix Datatables CSS not available #2512 via PR mockup #863 [sunew, thet]

Products.ExtendedPathIndex: 3.3.0 → 3.3.1

Bug fixes:
  • Fix dependencies defined in setup.py. [gforcada]

Products.GenericSetup: 1.8.9 → 1.8.10

Bug fixes:
  • Do not mask KeyError in 'getProfileDependencies' from missing dependency profiles. Refs: https://github.com/plone/Products.CMFPlone/issues/2228 [ida]

Products.MimetypesRegistry: 2.1.4 → 2.1.5

New features:
  • Python 3 compatibility. [rnix]

Products.PortalTransforms: 3.1.3 → 3.1.4

Bug fixes:
  • fix test for python 3 [petschki]

Products.statusmessages: 5.0.3 → 5.0.4

Bug fixes:
  • Use Umlaut ä in tests in order to check string/bytes handling. Refactor test_encoding to unittest. [jensens]

Products.Archetypes: 1.15.3 → 1.15.4

Bug fixes:
  • Switch to new TestCase using AT after PloneTestcase is now DX. Fall back to the old TestCase in case of an older plone.app.testing. [pbauer, maurits]

plone.app.relationfield: 1.4.1 → 1.4.2

Bug fixes:
  • Change import of message factory from dx to non-deprecated way. [jensens]

  • Fix tests in py3 [pbauer]

plone.app.versioningbehavior: 1.3.3 → 1.3.4

Bug fixes:
  • Migrate tests away from PloneTestCase [pbauer]

  • Fix imports for py3 [pbauer]

plone.app.blocks: 4.1.1 → 4.2.0

plone.app.mosaic: 2.1.0 → 2.1.1

plone.app.standardtiles: 2.3.0 → 2.3.1

plone.app.tiles: 3.0.3 → 3.1.0

plone.tiles: 2.0.0 → 2.1.0

Project resources

Learn about Plone

Tokyo Plone Conference 2018 Keynote Speakers Announced

Posted by PLONE.ORG on October 12, 2018 03:56 PM

Our three daily Plone Conference 2018 keynote speakers will be:

  • Eric Steele, Plone's release manager overseeing the Plone 4 and 5 releases, a longtime contributor of Plone add-ons and member of the legendary Penn State WebLion team
  • Jim Fulton, who led Zope development for many years and created many key Zope components and a co-creator of  buildout
  • Torajiro Aida, a Japanese high school student who started programming at the age of 6, whose interest is not only technology itself but also how technology affects our society and ethics. For his art work, he received the New Face Award at the Japan Media Art Festival.

Join us in Tokyo next month! Registration and details at https://2018.ploneconf.org

A conference sponsorship would be a great way to show your support for Plone and to enhance your visibility in the community! Find out more about conference sponsorships

2018 ploneconf keynote speakers

Three Members Join the Plone Foundation

Posted by PLONE.ORG on October 02, 2018 10:20 AM

The Plone Foundation welcomes 3 new members after unanimous confirmation by the Foundation's Board of Directors on September 13, 2018. Membership in the Foundation is conferred for significant and enduring contributions to the Plone project and community. The Plone Foundation Membership Committee overwhelmingly recommended each applicant for their ongoing contributions to Plone.

Katja Süss

Katja Süss

Katja got started in Plone by posting issues, committing features to existing add-ons, adding translations and submitting bug fixes. She has been co-organizer of monthly meetups in Zurich, welcoming integrators and developers and getting insights into customer needs and technical features. In addition to participating in several Plone and Python conferences, she has joined several sprints over the years, including the Innsbrück Alpine City sprint (Plone on Python 3) this past January. Most recently, she has contributed a new add-on to the community: the audio playlist player, collective.playlist.

Davi Lima

Davi Lima

Davi has been working with Plone and participating in the community since 2007, initially building and maintaining Plone 2.5 sites for a group of Brazilian universities and then moving on to bigger projects with several organizations: Simples Consultancy, Unified, Interlegis, Brazil's Presidency of Republic, Enfold Systems and Kitconcept. He has actively participated in community discussions, answering questions on IRC, StackOverflow, GitHub issues and the community forum, in English and Portuguese. He has participated in many sprints, and attended four Plone conferences (volunteering at two of these), where he gave Python and Plone training classes, presented talks, workshops, and lightning talks on XDV/Diazo, Plone React (PyConWeb Munich, EuroPython Edinburgh).

Eric Bréhault

Éric Bréhault

Eric's main contributions are Plomino and Rapido, two add-ons for non-experts to create their own applications in Plone. He has made contributions to Plone core, mainly for Plone 5. More recently, he focused on frontend technology, developing and maintaining plone.restapi-angular, an Angular 2+ toolkit allowing the building of sites and applications based on the Plone REST API. He is an active member of the Framework Team and has given talks, including inspirational keynotes, and provided training at several Plone conferences.

 

 

The Foundation is the trustee for Plone's intellectual property, works to protect and promote Plone, and has over 80 active members.

First Redactor Release

Posted by TestTheDocs on October 01, 2018 04:32 AM
What Is Redactor A work in progress editor-in-chief to highlight potential linguistic and structural issues with your text. Vision The aim of Redactor is to be a tool that helps check text with a variety of best practice based tests. While the test configuration is opinionated, inquisitive users may change and override the defaults. Initially, users run the tool locally, in the future we plan a remote (maybe commercialized) service triggered by user actions such as a CLI command or Git push.

Plone Goes To Pycon

Posted by Jazkarta Blog on September 12, 2018 09:50 PM

Plone booth at Pycon

I’ve just returned from Pycon where I had a great time staffing the Plone Foundation’s booth. It was fun to see old friends and introduce new people to Plone, Python’s open source, enterprise-grade content management system. Pycon attendees are wonderfully friendly and curious, always interested in learning new things. The venue – Cleveland’s downtown convention center – was lovely and the rain mostly held off. We gave away dozens and dozens of Plone-branded neck pillows and water bottles and multi-tools and pins and magnets, not to mention blue M&Ms arranged in the shape of the Plone logo. I arrived with 2 suitcases full of swag and left with 2 empties. Thanks to Witek and Nate from Jazkarta and to Chrissy, Anthony and Carol from Six Feet Up for helping set up and run the booth, and to the Plone Foundation for sponsoring Pycon. Open source rocks!

Announcing collective.siteimprove

Posted by Jazkarta Blog on September 12, 2018 09:47 PM

Screenshot of collective.siteimprove UI, expanded

As I reported back in May, at our last Jazkarta sprint Witek and Alec began work on a Plone add-on, collective.siteimprove, that provides integration with the Siteimprove content quality checking service. I’m pleased to announce that the add-on has now been thoroughly vetted by the folks at Siteimprove and the resulting version is available from Pypi!

What Is Siteimprove

Siteimprove is a respected service for maintaining and improving web content quality. Customers who sign up for the service get automated scans of their websites which check for content quality, accessibility compliance, SEO, data privacy, and performance. Site rollups and per page reports are available via email and from a customizable dashboard at siteimprove.com.

Siteimprove also provides an API which allows for the development of CMS plugins – integrations of the Siteimprove service within content management systems. This allows content editors to get immediate feedback on pages that they publish. This is great because it lets editors see problems while they are in the process of editing a page, instead of getting a report after the fact and needing to click through links to fix things.

Graphic explaining how Siteimprove works

 

Why Siteimprove for Plone

Plone, the premier Python-based open source content management system, is an enterprise scale CMS that is widely used by large organizations with large websites. These are just the types of organizations that can benefit from a tool like Siteimprove, which has a reputation for being an excellent service for maintaining and improving website content.

The Jazkarta team was delighted to be able to contribute to the Plone community by creating an add-on that integrates Siteimprove’s CMS plugin features into the Plone editing process. Now anyone with a Plone website can easily integrate with Siteimprove simply by installing an add-on – all the integration work has been done.

collective.siteimprove

After collective.siteimprove is installed on a Plone site, there will be a new control panel where Siteimprove customers can request and save a token that registers the domain with siteimprove.com. After that, authorized users will see an overlaid Siteimprove button on each page that shows the number of issues found.

Screenshot of the collective.siteimprove UI, collapsed

 

When clicked, the overlay expands to show a summary report of page errors and an overall score, as shown in the image at the top of this post. After an edit, users can click a button on the overlay to request that Siteimprove recheck the page. They can also follow a link to the full page report at siteimprove.com.

Plone+Siteimprove FTW

Now anyone who has a Plone website can easily integrate with the Siteimprove service and take advantage of all of Siteimprove’s enterprise-scale features while they are working on their content!

Plone 4.3.18 has been released

Posted by PLONE.ORG on September 08, 2018 02:26 PM

Plone 4.3.18 has been released, containing bug fixes and updated packages.

To download it, please see the download page.


Changes

Zope2: 2.13.27 → 2.13.28

DocumentTemplate: 2.13.3 → 2.13.4

plone.recipe.zope2instance: 4.3 → 4.4.0

New features:
  • Added support for setting instance-home option. [zupo]

  • Added support for setting CGI environment variables. [zupo]

Bug fixes:
  • Regard 'parsed_version' of setuptools > 38.7.0 does not return iterable anymore, fixes #37. [ida]

zest.releaser: 6.13.4 → 6.13.5

robotframework: 3.0 → 3.0.4

robotframework-selenium2screenshots: 0.7.0 → 0.7.2

selenium: 2.53.5 → 2.53.6

sphinx-rtd-theme: 0.1.5 → 0.1.9

Pillow: 3.3.0 → 3.3.3

importlib: 1.0.3 → 1.0.4

lxml: 3.6.0 → 4.2.1

WebOb: 1.4.1 → 1.4.2

Plone: 4.3.17 → 4.3.18

Products.Archetypes: 1.9.17 → 1.9.18

Products.CMFPlone: 4.3.17 → 4.3.18

Products.CMFUid: 2.2.1 → 2.2.2

Products.GenericSetup: 1.8.8 → 1.8.9

Bug fixes:
  • When metadata.xml parsing fails, show the filename in the ExpatError. Fixes Plone issue 2303 <https://github.com/plone/Products.CMFPlone/issues/2303>_.

  • Require five.localsitemanager less than version 3. Version 3 requires a too new Zope2 version.

Products.PlonePAS: 5.0.15 → 5.1.0

New features:
  • Notify PropertiesUpdated event when member properties are changed [ezvirtual]

collective.monkeypatcher: 1.1.3 → 1.1.5

Bug fixes:
  • Fix import for Python 3 in the tests module [ale-rt]

  • Fix import for Python 3 [pbauer]

five.localsitemanager: 2.0.5 → 2.0.6

  • Don't complain if the site root has no Acquisition parent. [davisagli]

  • Removed zope.site dependency. Using Zope 2.12 it is an indirect dependency and using Zope 2.13 or later it is no longer required. [yuppie]

  • Ensure that the PersistentComponents has no aquisition wrapper before passing to the superclass, to allow the caching of component roots in zope.interface to make a weakref to this root. [MatthewWilkes]

plone.app.layout: 2.3.17 → 2.3.18

plone.app.textfield: 1.2.10 → 1.2.11

Bug fixes:
  • Python 3 fixes [pbauer]

plone.app.upgrade: 1.4.4 → 1.4.5

plone.app.z3cform: 0.7.7 → 0.7.8

plone.cachepurging: 1.0.14 → 1.0.15

Bug fixes:
  • consider purging to be enabled when it's enabled (even if no servers are listed) [skurfer]

plone.folder: 1.0.10 → 1.0.11

New features:
  • Improve logging in case ordered index is not consistent [tomgross]
Bug fixes:
  • Remove ancient buildout config [tomgross]

  • Replace deprecated testing assertion calls [tomgross]

grokcore.security: 1.6.2 → 1.6.3

plone.app.lockingbehavior: 1.0.4 → 1.0.5

Bug fixes:
  • Add coding header on python files. [gforcada]

  • Unskip test for Zope 4, as isolation problems are already fixed. [thet]

plone.app.versioningbehavior: 1.2.0 → 1.2.10

New features:
  • Used plone i18n domain and removed locales folder. [klinger]
Bug fixes:
  • Do not break in the case of dexterity objects with relations migrated from something else (usually Archetypes). [ale-rt]

  • Use zope.interface decorator. [gforcada]

  • Fixes #25: URLs like ${absolute_url}/@@images/${uuid}.png are not converted on @@version-view. [rafaelbco]

  • Updated Traditional Chinese translations. [l34marr]

  • Update Italian translations [ale-rt, cekk]

  • Fixes #10: Views for Image and File versions don't work. [rafaelbco]

  • Update French translations [enclope]

  • Updated basque translation [erral]

  • Correct functional test, it was not checking correct on version1. [bloodbare]

  • Synchronize translations [vincentfretin]

  • provide better description of how new versions are created when in manual mode [vangheem]

  • Ported tests to plone.app.testing. Removed PloneTestCase / p.a.testing compatibility hack. [jone]

  • Remove dependencies on zope.app.container and rwproperty. [davisagli]

  • Added Italian translations. [cekk]

plone.api: 1.8.3 → 1.8.4

Bug fixes:
  • Call processForm with {None: None} dict as values. This prevents processForm using REQUEST.form and overwriting values already set by invokeFactory. Fixes issue 99 <https://github.com/plone/plone.api/issues/99>_. [david-batranu]

  • Simplification/minor speedup: Permissions checks now directly use AccessControl. Technical its now exact the same as before. Before a tool lookup was needed, calling a utility function, calling AccessControl. [jensens]

Project resources

Learn about Plone

Nominations Open for Plone Foundation Board of Directors

Posted by PLONE.ORG on August 29, 2018 03:55 PM

If you have an interest in helping the governance of Plone, and particularly the energy and time to pitch in, please consider nominating yourself to serve on the Plone Foundation board of directors for 2018-2019.

Nomination Process

  1. Log in on plone.org and go here: https://plone.org/foundation/meetings/membership/2018-membership-meeting/nominations
  2. Add a page there with your name in the title.
  3. For the body, discuss:
    • Who you are
    • Why you're interested
    • What you think you can add to the Plone Foundation
    • Most importantly, the name(s) of one or more Plone Foundation members who "second" your nomination
  4. Once ready, click "submit" in the workflow drop-down menu to get a reviewer to look at your nomination.
  5. Nominations will be accepted until October 31 2018, 23.59, UTC. The election will be conducted in conjunction with the annual meeting, which will take place in Tokyo, Japan at the Plone Conference 2018. All active members of the Plone Foundation will be eligible to vote.

About Board Membership

The Plone Foundation is a not-for-profit, public-benefit corporation with the mission to "promote and protect Plone". That has meant that the board is involved in:

  • protecting the trademark, copyrights and other intellectual property, including considering licensing and usage issues;
  • hiring the release manager;
  • working with various committees, including marketing and membership;
  • handling "other stuff in the community" as needed, e.g. helping craft policy on plone.org and plone.com about commercial listings
  • but not: directing Plone development. The board facilitates, but does not direct, the development of Plone itself.

While there's lots of work that happens online, much of the critical business of the board is conducted during video meetings every two weeks — typically, board meetings last about an hour to 90 minutes though occasionally they can run over to handle time-critical issues.  Please consider whether this fits your schedule, since missing more than an occasional meeting severely limits the ability of the board to reach quorum and conduct business.

Historically, board meetings have been organized to occur during daytime hours in America and evening hours in Europe, currently at Thursday nights, 19.00 UTC in northern hemisphere summer and 20.00 UTC in northern hemisphere winter. That can always change with new board members.

In addition, there is a board mailing list (private), where we discuss things in addition to the meetings.

This is a working board. Be ready to regularly take on and complete responsibilities for board business.

The board writes no code and makes no development decisions. It is much more concerned with marketing, budgets, fundraising, community process and intellectual property considerations.

You do not need to be a Foundation member to serve on the board (in fact, board leadership is an excellent way to become a Foundation member). All you need is to get an active Foundation member to second your nomination.

The Plone Foundation is interested in broadening the diversity of our leadership, with regards to gender, ethnicity, and geography.

If you have questions about the nomination process, contact the board: board@plone.org

Plone Conference 2018: Registration and Call for Talks are now open

Posted by PLONE.ORG on August 27, 2018 01:27 PM

See the Plone Conference 2018 website to register and to submit your talk proposals!

Visit 2018.ploneconf.org

The conference will be held in Tokyo, Japan, from November 5-11, 2018.

It will include outstanding training classes, keynotes and presentations, and will be followed by two days of collaborative sprints.

Join the Plone and Python web communities at this annual event, not to be missed!

For more news about the conference, sign up for the announcements list at 2017.ploneconf.org

 Plone Conference 2018 logo

 Photo credit: Eric Montfort, "Tokyo"

Auch bei Vodafone sind nur Sicherheitsdilletanten am Werk

Posted by Andreas Jung/ZOPYX on August 24, 2018 03:35 PM
Offenbar hat auch bei großen Konzernen niemand Ahnung von Security oder man ist ignorant. Wieviele Datenlecks, wieviele Hacks müssen noch passieren bis man wach und sensitiv wird?

Successful Beethoven Sprint in Bonn

Posted by PLONE.ORG on August 23, 2018 05:12 PM

During this three-day sprint at the office of kitconcept in Bonn we finished the second iteration of the new Pastanaga editor for Plone-React. The new editor is based on a tiles implementation for plone.restapi that we built during the sprint. The user can add, edit, and delete text, image, and video tiles and arrange them via drag-and-drop.

Group photo of the Beethoven Sprint 2018

In addition to the new tiles endpoint, the Plone.restapi team worked hard on multiple enhancements that resulted in four Plone.restapi releases and two Plone.rest releases during and right after the sprint. Plone.restapi and Plone.rest now work with Plone 5.2, which uses Python 3. We also ported the JSON field type from Guillotina to plone.schema and released it with plone.schema 1.2.0.

In addition to working on the new Pastanaga editor for Plone-React, we significantly enhanced and polished Plone-React, resulting in the Plone-React 0.7.0 release.

We made Plone-React work with a Guillotina backend and discussed the roadmap to Plone 6.

The sprint was kindly sponsored by the Plone Foundation, the German Python Software Foundation (PySV), The Python Software FoundationCodeSyntaxGanzgraphkitconcept GmbH, and Iskra.

Full daily sprint reports can be found here: day oneday two, and day three.

Announcing the Plone Conference 2019 selection process

Posted by PLONE.ORG on August 18, 2018 12:45 PM

With Plone Conference 2018 drawing near, it is time to begin planning for our next conference in 2019.The annual Plone Conference brings together users, integrators, developers, designers, and other interested folk from throughout the world for a week of training, talks, and sprinting. Plone conferences are also an expression of community spirit: they are organized by a company, user group, or other entity with ties to and a history with the Plone community and are in essence not-for-profit events.

The Plone Foundation is soliciting proposals to host the 2019 Plone Conference. The selection process this year begins in time to allow for final selection of the conference venue during this year's Conference. The extended timeline allows groups and organizations interested in hosting the 2019 Plone Conference (or beyond) to work with the Tokyo team for hands on experience during this year's conference.

Let's revisit where we've been so we can determine where we might want to go: we've traveled the world from New Orleans, Louisiana, USA for the first Plone Conference to:

  • Vienna, Austria
  • Seattle, Washington, USA
  • Naples, Italy
  • Washington, D.C., USA
  • Budapest, Hungary
  • Bristol, UK
  • San Francisco, CA, USA
  • Arnhem, Netherlands
  • Brasilia, Brazil
  • Bucharest, Romania
  • Boston, MA, USA
  • Barcelona, Catalunya, Spain

and this year to Tokyo, the capital of Japan. But, there are many places yet to explore! If you have a place in mind, don't be shy: submit a proposal!

The Plone Foundation will accept proposals beginning September 1 through October 15, 2018.

The Foundation Board of Directors will review proposals and open those that are viable for voting by the Foundation membership between November 1–5, 2018. The winning proposal will be announced at the end of Plone Conference 2018 in Tokyo.

Everything you need to know to submit a proposal, including the full schedule for the process and in-depth requirements for hosting, is outlined in the official Plone Conference 2019: Call for Proposals.

On behalf of the entire Plone community, we look forward to your conference proposals!

TestTheDocs Sprint

Posted by TestTheDocs on August 01, 2018 04:32 AM
Sprinting Time Join us in Szeged the sun city of Southern-Hungary, or online from your favorite spot!

Pastanaga Editor Status Report - Plone Beethoven Sprint 2018

Posted by kitconcept GmbH on July 20, 2018 11:24 AM

During the Plone Beethoven Sprint in Bonn, we worked hard on creating a first version of a new content editor for Plone-React.

Here is a short demo of what the editor looks like right now:

Demo of the Pastanaga Editor with tiles

We already had a first implementation based on DraftJS that allows inline styles (e.g. bold, italic), block styles (headlines, (un)ordered lists), and links to remote URLs.

The new version of the editor is based on a “tiles” backend, that is build by Victor Fernandez de Alba during the sprint and released with plone.restapi 3.2.0.

This allows us adding more complex content elements such as images, videos, and in the future more complex layout elements.

With the new backend in place, Rob Gietema went ahead and implemented the basic editor that Albert Casado designed as part of the new Pastanaga UI for Plone.

pastanaga editor Mobile Pastanaga Editor design by Albert Casado

The user can type in the title, description and the text content of the document without worrying about form fields or be distracted by tabs and fieldsets.

In addition to the standard text editing it is now possible to add an image tile that can be placed on the left or right side, on the center of the page or in full page width.

Rob also added a YouTube tile that allows the editor to add a YouTube video URL and then displays the video within the editor and the page view.

Text, image, and video tiles can be added to a page. They can be deleted and moved up and down to change the order of the elements.

Next Steps

The new editor is a great accomplishment. The tiles endpoint in plone.restapi allows us to further enhance the current version of the editor with more advanced layout variants and tiles.

We plan to continue with our iterative and agile approach of building a useful, fully functional version of the editor with each step, that allows Plone companies to use the editor and Plone-React today in their client projects.

The next steps are polishing the editor and the existing tiles. Work out some UX issues that we found when working with the editor and building more advanced tiles.

Stay tuned for more news and features after the Costa Brava sprint…

IMIO wins European Commission prize, donates to the Plone Foundation

Posted by PLONE.ORG on July 06, 2018 06:53 PM

imio-award2.jpeg

Award and Donation

On March 29, 2018, the European Commission awarded IMIO the €15,000 first prize in the local government category of the Sharing and Reuse Award.

IMIO has generously donated €10,000 of the prize money to the Plone Foundation.

The Sharing and Reuse Award, for interoperable solutions for public administration, businesses and citizens in Europe, recognizes government agencies that have set up and shared IT solutions with broad potential for reuse.

Of the 16 nominated projects, 10 (including IMIO's) are open source.

Governments are able to realize substantial savings and efficiencies when they reuse "cross-functional" components such as authentication systems and electronic invoicing. 

in Sorrento, Italy

IMIO's Statement on Plone

"If IMIO has won the Share and Reuse Award of the European Commission, it is due not only to the support and unconditional collaboration of its members, but also to the innovative aspects of its approach, be it organizational or technical.

This latter element, namely "Plone", is often unknown because the close relationship between IMIO and local governments masks the activity of a host of other private and public actors who created this technology that is the basis for most of IMIO's tools.

However, without them, IMIO would probably not have reached this technological maturity and nor have been able to respond as effectively to the needs of local authorities. Given its limited capabilities, IMIO has not been able to contribute directly to Plone core software. The software is indeed "free": no funds in the form of license fees were spent on this technology. Nevertheless, IMIO contributed to the community in the form of our team's services during international sprints, creation of generic modules, community support (contributions to forums), presentations at conferences, and evangelizing, either directly or through our subcontractors.

The prize money donated by IMIO is a token of appreciation for the hard work of the Plone community and its Foundation, with which we have worked for 11 years. The €10,000 donation will allow the Foundation to continue supporting its activities."

About the Plone Foundation

Plone is 100% funded by sponsorships from forward-thinking organizations and individuals. With the help of these financial contributions, the Plone project has continued to thrive and maintain an astonishing level of activity and innovation for 17 years.

The mission of the Plone Foundation is to protect and promote Plone. Foundation expenses are primarily related to:

  • an ambitious support programme for organizing and attending strategic sprints, conferences and marketing events
  • paying the Plone release manager a travel stipend to evangelize, plus a small stipend for major and minor software releases
  • registration of Plone’s trademark in countries around the world
  • legal work to secure Plone’s intellectual property in the Foundation’s “software conservancy”
  • the development and production of marketing materials
  • provisioning servers for our web sites, community forum, testing and continuous integration processes
  • a stipend to support the Plone marketing/communications team lead's attendance at Plone and related events

In order to continue this work, the Plone Foundation relies on sponsorships to create and sustain a reliable stream of income.

In recent years, we have aggressively supported strategic sprints, which are designated by the Framework Team as being significantly important to the continuing development of Plone. These include projects such as the headless CMS initiative, the porting of Plone to Python 3, the new Pastanaga UI, the integration of Angular and React JavaScript frameworks, and the REST API. Each of these represents important avenues for Plone's future growth, and these have been the primary expenditures of the Foundation, funded by vital sponsorships.

For further information:

Plone Beethoven Sprint 2018 - Sprint Report - Day 3

Posted by kitconcept GmbH on June 30, 2018 06:24 PM
Report of the third day of the Plone Beethoven Sprint 2018 in Bonn, Germany.

The Beethoven Sprint was a “strategic” sprint that took place June 21-25 at the kitconcept office in Bonn, Germany. The focus of the sprint was to work on the Pastanaga editor, Plone-React, plone.restapi, and Guillotina. This is the report of the third day of the sprint.

day3 wrap up all Day three wrap-up meeting at the kitconcept office

UTC DateTime Discussion

After having breakfast and doing a quick stand-up, we started a discussion about how to store datetimes in Plone. This was a discussion that came up when we implemented plone.restapi. Though, this question is more related to plone.app.event. As seen many times in the Plone community, when it comes to complex technical matters, offline discussions work way better than discussing issues back and forth via github comments.

Since we had all the experts at the sprint, we decided to schedule a discussion to solve the issue. In the end we reached an agreement that dates should be stored as UTC and that the local time zone should be stored separately in an additional field, for instance to calculate recurring dates.

plone.restapi

Thomas Buchberger worked on the workflow endpoint and added a feature to change the workflow state recursively, to set the effective and expiration date and to allow workflow comment.

Lukas Graf finished his work on implementing redirects in plone.rest based on plone.app.redirector. The redirects do not only work for simple GET requests but also redirects POST, PATCH, und DELETE requests.

Sune Brøndum Wøller finished the portlets endpoint for plone.restapi.

Mikel Larreategi made plone.restapi return cachable image resources to improve the caching of images in plone.restapi.

Roel finally wrote a PLIP for the IPloneSiteRoot and fixed edge cases and a few bugs in the current implementation.

Plone-React

Victor completed his work on plone.schema, allowing to use a JSON field in Plone that validates the JSON structure within the field and also allows through-the-web editing. He also finished the backend behavior for tiles and blocks to store the tiles and the tiles layout on a Plone content object.

Eric Steele added a backend implementation for the add-ons control panel that he wrote for Plone-React. Eric presented a fully working add-ons control panel that allows to install add-ons via Plone-React by the end of the day.

day3 wrap up eric add ons control panel Plone-React add-ons control panel presented by Eric Steele

David completed his work on the vocabularies endpoint by implementing the frontend widgets that rely on those vocabularies to make sure the endpoint serves its purpose.

Davi Lima and Victor continued to work on the override mechanism for Plone-React (JBOT) to customize widgets and views in a config.js file using a babel plugin.

Pastanaga Editor

day3 wrap up rob editor video Rob presenting the new video tile for the Pastanaga editor

As always, Rob did not give us any time to breath and added a YouTube video tile to the Pastanaga Editor, that shows a YouTube video in the editor itself and on the content view.

Of course that wasn’t enough for a day, so he fixed a few other smaller issues and worked on exposing the sitemap.xml via Plone-React.

Dinner and some late night hacking

Dinner Hans im Glueck Dinner at “Hans im Glück”

Rob might not admit it, but after claiming he could fix all those issues mentioned above in one day, he continued to work on the sitemap during our dinner at a local burger restaurant in the inner part of the city. We’ve been there ourselves, so we didn’t mind, while we were enjoying our tasteful cocktails. After a good meal, some drinks, and having to say good by to Eric, we went back to the office.

We said good bye again to our fantastic Dutch Plone-React team Rob and Roel and then went on to hack a little bit more on the stuff we were (and still are) so enthousiastic about.

Plone Beethoven Sprint 2018 - Sprint Report - Day 2

Posted by kitconcept GmbH on June 29, 2018 06:24 PM
Report of the second day of the Plone Beethoven Sprint 2018 in Bonn, Germany.

The Beethoven Sprint was a “strategic” sprint that took place June 21-25 at the kitconcept office in Bonn, Germany. The focus of the sprint was to work on the Pastanaga editor, Plone-React, plone.restapi, and Guillotina. This is the report of the second day of the sprint.

Guillotina

After having breakfast at the office, we started the day with a stand-up/wrap-up meeting with Ramon Navarro Bosch presenting Plone-React running on Guillotina. The user can add new content, edit existing content, and browse the content with Plone-React and Guillotina, which is a huge acomplishment and a promise for the future of Plone. After the stand-up Ramon had to say good bye to the other sprinters and head home to Barcelona.

Pastanaga Editor

Rob Gietema continued his work on the Pastanaga editor. At the end of the day, he was able to present a working version where the user can add a title and a description tile as well as a text tile with inline styles such as bold, italic, headlines, links, and lists.

IMG 6754 6b052ccf7a9ed291599dc6a7047a71cc Pastanaga Editor with basic text editing

Rob also added an image tile that allows you to upload images. The uploaded images then can be aligned left, right, in the middle and in full width or just be deleted.

IMG 6756 abcbc897d406fc882d25dca889402298 Image tile that can be aligned left, right, center and displayed with full width.

Victor Fernandez de Alba finished his work on the tiles backend for the Pastanaga editor (https://github.com/plone/plone.restapi/pull/547), so Rob could present a fully functional editor that actually stores the content (text and images) in Plone tiles.

Plone-React

Eric Steele continued his work on the add-on control panel and was able to present a first version at the wrap-up at the end of the day.

IMG 6758 144f319424d81d358e6410f2aea3d41f Add-ons control panel written in React by Eric Steele

Carsten Senger worked on implementing the users and groups control panel in Plone-React which now allows to add and delete users.

Victor worked on being able to use Plone-React as a library that allows developers to override React components. The system is supposed to work the same way as JBOT (just-a-bunch-of-templates) works in Plone.

Johannes Raggam and Andrea Cecchi continued their work on the reference widget and were able to present a first prototype. We agreed that we have to put some effort into the UX/UI of that widget, but that was beyond the scope of a sprint and requires the help of a UX specialist.

plone.restapi

Thomas Buchberger finished the pull request for the object create order and started to to work on the workflow endpoint enhancements.

David Glick worked on enhancing the vocabularies endpoint with batching and filtering.

Sune Brøndum Wøller fixed some nasty bugs with time freezing and transactions errors in the Plone 5.2 of plone.restapi (e.g. https://github.com/plone/plone.restapi/pull/562, https://github.com/plone/plone.restapi/pull/561).

Lukas Graf continued his work on the translations of the REST response data, fixed a few bugs and wrote a script that tests the Sphinx-based documentation for warnings and errors and fails the Travis build if there is a problem.

Mikel Larreategi finished the pull request for the history endpoint and documented the Accept-Language headers in plone.restapi. He also started working on making the image scales that plone.restapi returns cachable.

Roel Bruggink continued his quest on the folderish site root and fixed errors in CMFPlone, plone.app.content, and in plone.app.dexterity.

We were able to release plone.rest 1.1.1 and plone.restapi 2.1.0 as well as plone.schema 1.2.0 with a JSON field that is required for the tiles endpoint in plone.restapi.

Hacking Night

Germany was playing at the soccer world cup in the evening, so we decided to order Pizza for dinner and then split up between people that wanted to code and people that wanted to watch the game. As you might have guessed the latter group was rather small. We ended up in a bar because the public viewing was already full. Though I guess the sprinters got a taste of the German soccer culture. After the game we went back to the office to join forces with the others again to wrap up the day and hack the night away.

PloneGov growing in the Basque Country

Posted by CodeSyntax on June 29, 2018 08:11 AM
PloneGov is an international initiative with the goal of getting a powerful on-line eGovernment tool. Most eGovernement needs and requirements are similar and PloneGov wants to satisfy them in a effective and efficient way thanks to its open source project. CodeSyntax is part of PloneGov thanks to its UdalPlone initiative.

Plone Beethoven Sprint 2018 - Sprint Report - Day 1

Posted by kitconcept GmbH on June 28, 2018 06:24 PM
Report of the first day of the Plone Beethoven Sprint 2018 in Bonn, Germany

The Beethoven Sprint was a “strategic” sprint that took place June 21-25 at the kitconcept office in Bonn, Germany. The focus of the sprint was to work on the Pastanaga editor, Plone-React, plone.restapi, and Guillotina. This is the report of the first day of the sprint.

Stand-up

day1 standup Stand-up of the first day

We started the day with a stand-up meeting giving people a heads up on the current state of affairs of plone.restapi, plone-react, and Guillotina.

I started with plone.restapi, which is considered stable and battle tested in production.

Victor Fernandez de Alba then gave a brief introduction to our first Plone-React based project VHS-Ehrenamtsportal, that we successfully shipped to our client a few weeks ago and run without any issues since then.

IMG 4031 565f311943de0d1be7694a4b1c0e79a8 Victor introducing VHS Ehrenamtsportal

After this, Rob Gietema gave a short introduction to the current state of Plone-React.

Last but not least, Ramon Navarro Bosch presented “Guillotina”, an async server written in Python with a Cockroach DB / ElasticSearch backend that adopts some of the core concepts of Zope and Plone.

With a group of 15 sprinters, we decided to split up in four different groups for the main sprint topic. Thomas Buchberger led the plone.restapi group, Rob the Plone-React group, Victor the Pastanaga Editor group, and Ramon the Guillotina group.

Pastanaga Editor

22 06 18 4 Rob Gietema going though the different approaches we discussed during the Tiles planning meeting

Right after the stand-up, we had a longer discussion about the “tiles” endpoint in plone.restapi and the editor implementation in Plone-React. We already reached an agreement of the API design at the Plone-React sprint a few months ago. Though, it turned out that implementing that on top of the existing plone.tiles implementation was harder than we thought and we did not anticipated all the problems that came along with that.

We decided to keep the API design and to write a simple Dexterity behavior that adds a “tiles_layout” field for the layout information and a “tiles” field that holds the actual data of the tiles. Ramon already wrote a JSON-field in the Guillotina code that we decided to re-use for our implementation.

Rob Gietema already wrote a first prototype of the new editor at the Plone-React sprint and he was waiting for the backend code to be implemented. While we were working on the backend implementation, he focused on the prototype.

plone.restapi

22 06 18 24 Planning board with plone.restapi issues

Lukas Graf added missing translations in plone.restapi responses and simplified the test setup and did some clean up on the code (https://github.com/plone/plone.restapi/pull/545).

Thomas Buchberger worked on fixing the plone.restapi object creation logic to behave more like through-the-web object creation (https://github.com/plone/plone.restapi/pull/549) and separated the object creation from firing the events.

Sune Brøndum Wøller cleaned up and upgraded multiple Plone versions in plone.restapi (https://github.com/plone/plone.restapi/pull/537), worked on portlets and portletmanager serialization and fixed a ReadConflictError in the plone.restapi tests for the documentation that was bugging us for quite some time (https://github.com/plone/plone.restapi/pull/543).

David Glick and me worked on Zope 4 compatibility for plone.rest and plone.restapi. It turned out that one of my fixes on plone.rest was already sufficient and that the test failures in plone.restapi were caused by a plone.testing issue that David found and fixed (https://github.com/plone/plone.testing/pull/49).

Roel Bruggink continued his efforts on turning the Plone site root into a Dexterity object. He worked on making the IPloneSiteRoot interface / content object behave more like content and he attached behaviours to the IPloneSiteRoot to edit them without relying on a default page.

Mikel Larreategi finished his work on the translation of the content-type names on the @types endpoint (https://github.com/plone/plone.restapi/pull/540) and the translation of the actions and workflow state and transitions on the @history endpoint (https://github.com/plone/plone.restapi/pull/541).

Plone-React

Rob gave an introduction to the Plone-React codebase and explained the basic concepts and libraries that we use in Plone-React.

Eric Steele started to work on creating the add-ons control panel in React. Carsten Senger took over the work that Rob started before the sprint to bring the users and groups control panel to Plone-React.

Andrea Cecchi. and Johannes Raggam worked on the React-based widget for references in Plone.

Guillotina

After giving an introduction to Guillotina to the sprinters, Ramon went ahead and made Plone-React work on top of Guillotina. Since it origins, plone.restapi and Guillotina were supposed to share the same API to allow us to switch the backend for our new frontend at some point in the future. Ramon was also heavily involved in the API design of plone.restapi and wrote the first version of plone.rest before he decided to invent Guillotina. Over the time both APIs differed because of differences in the underlying implementation.

Ramon and Rob worked on this and by the end of the day they could present a working version that allows basic content editing and browsing.

Roadmap / Plone 6

We had a hangout with Philip Bauer from Munich who is leading the efforts to migrate Plone to Python 3 and Zope 4.

Hangout with Philip Bauer

Philip and I already had a longer discussion about a possible roadmap for Plone 6 and how to bring our efforts on the frontend together with the efforts of the group that works on Python 3 and Zope 4. We discussed the outlined roadmap and the upcoming sprints where we plan the implementation of the roadmap.

Lunch / Dinner / Evening

We went to have lunch in a Vegan cafe (Black Veg) and to a traditional brewery in the old part of the town for dinner (Bierhaus Machold). After dinner and a few drinks, we decided to head back to the office for some late night hacking. Not without stopping by at a local “Kiosk” for some customary buying of beverages for the evening.

World Plone Day 2014: manufacturing internationalization strategy

Posted by CodeSyntax on June 22, 2018 08:47 AM
World Plone Day was last wednesday, april the 30th, and as in previous occasions, we did celebrate it at CodeSyntax's offices, with some customers and Plone users of the Basque Country.

Presenting Buildout at PySS 14

Posted by CodeSyntax on May 29, 2018 01:10 PM
Buildout is a tool we use in all of the development and deployments of our applications, and we have given a talk about it at PySS 14.

Porting Plone to Python 3

Posted by Starzel.de on May 23, 2018 07:26 PM

Since I wrote the proposal to Port Plone to Python 3, so much has happened that a status update is needed.

Let's step back a little: The first steps towards Python 3 were taken during the sprint at the Plone conference in Barcelona. The epic PLIP to update to Zope 4 was merged and we started porting individual packages to Python 3 without being able to run any tests. With the help of sixer and python-modernize we tried to get the most obvious import and syntax issues out of the way. That work was continued at the Alpine City Sprint in Innsbruck and in the end, we treated more than 150 packages like this. Some of this work is documented in the Plone tracker.

Along the way I wrote a best-practice guide on how to port Plone packages this way.

As the PLIP states, there are several approaches to porting Plone to Python 3:

  1. Migrate and test packages that have no dependency to CMFPlone and test them in Python 2 and 3.
  2. Prepare packages for Python 3 without being able to test them in Python 3.
  3. Start up Plone on Python 3 and fix whatever breaks. When start-up works, create a Plone Site and again, fix whatever breaks.
  4. Port plone.testing and plone.app.testing to Python 3 and start running tests. Fix what breaks during the setup of the layers.
  5. Run the tests with Python 3 and fix all broken tests.

At the sprint in Innbruck I started with #3 and kept going after the sprint until I was able to create an instance. At the Plone Tagung in Berlin I was able to demo the creation of a site but nothing was rendered yet.

After that, I kept going and I was finally able to create a site and manage content, which is what a CMS is about. It looked a bit raw but I was able to add and edit some content - yay!

early screenshot of plone with python 3

Work continued at an unsteady pace, and with important contributions from Michael Howitz, Alessandro Pisa and David Glick, things started to get better. Even the theme and js started working. Slowly broken features became the exception, not the rule.

Last week at the Zope 4 Welcome Sprint in Halle we removed all feature blockers that we had found so far. Plone with Python 3 looks and feels like it used to in Python 2. During the sprint there was also a lot of progress on:

  • the wsgi setup
  • logging and tracebacks when using wsgi
  • porting plone.testing
  • a new theme for the ZMI
  • beta releases for many packages.

There was also some progress on the difficult issue of database migrations. It seems like zodbupdate is the best tool to do that but there is probably a lot of work ahead.

Read more about the sprints outcome in the blogpost by Michael Howitz.

There is a Jenkins job and as of today, it is running (not passing) tests for all packages (except Archetypes) with Python 3.

At the moment we run 6594 tests from 115 packages with 257 failures and 315 errors - not bad when you keep in mind that we still focus on features, not on tests. Tests for a couple of packages are green, including plone.api which is great since plone.api uses most of our stack one way or another. Jenkins runs every three hours and here you can see the progress over time.

Here is a screenshot from today that shows Plone doing some pretty non-trivial things: Editing a recurring Event and also Richtext with Images and Links:

screenshot today

Next steps:

  • Create a demo site running on Python 3 that people can use to find broken features. This will happen at the Plonator Sprint in Munich.
  • Fix all tests that look like they fail because of a broken feature.
  • Fix all remaining tests to pass in Python 2 and Python 3. Since there are a gazillion doctests that will take some time.
  • Port plone.app.robotframework to python 3 and fix robottests.
  • Experiment with porting a ZODB with Plone to Python 3.

If you want to participate you can simply look at the failing tests and start fixing them. You can also try to fix one of the open issues.
The setup of a coredev environment with Python 3 is really simple and documented.

Rural Sprinting: Two New Plone Add-ons and Progress on Python 3

Posted by Jazkarta Blog on May 17, 2018 06:56 PM

Jazkarta Team

We returned to my house in rural Massachusetts for our annual sprint this year and had a great time sampling local beers and ciders,

Buying supplies for the sprint

eating big meals together,

Making pizzas at the sprint

admiring the Milky Way,

Nightime view at Sally's house

and working on some fun projects. Normally we work remotely – everyone in their own home office, spread across 3 countries and 2 continents. But it’s really nice to get together in person, and we try to do it once a year.

Here’s what we worked on.

Witek and Alec created 2 new Plone add-ons and released them to Pypi.

  • jazkarta.abtesttile Provides a new Mosaic tile type that can be used for A/B testing parts of a page layout. Managers can define 2 rich text fields on one tile, and a ratio for how often each should be displayed (for example, 70%/30% or 50%/50%). Plone will randomly show users one field or the other in that ratio. Managers can optionally specify Javascript snippets for use in analytics tracking. Managers can also optionally enable a query string variable, which is added in the rendered HTML to links in the rich text fields. This will indicate whether option A’s or B’s rich text was the source of a page visit. A custom permission allows usage of the add-on to be restricted to privileged users.
  • collective.siteimprove Provides integration with siteimprove.com. There is a control panel for requesting and saving a siteimprove.com token that registers the domain with Siteimprove. (You must first sign up for a Siteimprove account.) A Siteimprove button is shown to authorized users on all default views. Publicly visible content shows authorized users a Siteimprove recheck action in the Plone toolbar that checks the individual page. This add-on is essentially done but untested since we have not yet met with the Siteimprove sales person who will provide an account for us to test with. We hope to be able to do that next week.

David, Matthew and Jesse decided to contribute to the ongoing effort to port Plone – Python’s open source enterprise CMS – to Python 3. They were in the porting groove because we recently ported our Dallinger project to Python 3. They made some good progress:

  • David made it possible to run Plone tests without including Archetypes, so that developers can run the tests in Python 3 without worrying about porting Plone’s old content type system. After that he investigated why cookies on Python 3 are preventing logins from staying logged in. He traced it to PAS and create a branch with all tests passing on both Python 2 and 3. Hopefully it will get merged during the Halle sprint this week.
  • Matthew found and fixed some Chameleon problems. This included an old error that had nothing to do with Python 3 where Chameleon puts spurious context into error messages. He also converted plone.namedfile to Python 3 and fixed some tests in plone.protect (CSRF protection). His work resulted in a fairly large pull request, which was approved and merged.
  • Jesse got all tests running for plone.app.workflow and made a bit of progress on the test failures for plone.app.dexterity.

Thanks to all these fixes, we got to the point of being able to save a Plone page on Python 3 and it “sort of” works. (At least it didn’t give an error!)

David also did some evaluation of plone-react with Nate. This is a React-based front-end for Plone that is built on plone.restapi. It’s in early stages of development but looks promising. In the process, Nate ran into a bug in Plone’s unified installer, tested it in Plone 5.1.2 and and filed a ticket for it. Back to his Plone roots!

Plone Welcomes Students for Google Summer of Code 2018

Posted by PLONE.ORG on May 03, 2018 08:07 PM

For 2018, the Plone Foundation has been granted four Google Summer of Code student project slots.

Google Summer of Code is a global program focused on bringing more student developers into open source software development. Students work with an open source organization on a 3 month programming project during their break from school.

After careful selection of the many project proposals presented, we are pleased to announce that the following students will be working on various aspects of Plone:

  • Shriyansh Agrawal (IFTTT plugin for Plone)
  • Akshay (Command Line Plone Tools)
  • Ajay NS (GatsbyJS Integration with Plone)
  • Nilesh Gulia (Create-React-App for Plone-React)

Read more about their projects

To our new students: welcome to the Plone community – we wish you a great learning experience over the coming summer. Congratulations!

Mentoring these students will be:

  • Timo Stollenwerk (former Plone GSoC student!)
  • Rob Gietema
  • Victor Fernandez de Alba
  • Asko Soukka
  • Andrea Cecchi
  • Paul Roeland
  • Alexander Loechel
  • Maik Derstappen
  • Encolpe Degoute
  • Nejc Zupan (former Plone GSoC student!)
  • Sally Kleinfeldt
  • T. Kim Nguyen

To our mentors: thank you for introducing new developers to open source and our community!

Plone-React Sprint Bonn 2018

Posted by kitconcept GmbH on April 23, 2018 04:46 PM

From March 12th to 15th, we hosted a Plone-React sprint at our office in Bonn. The main goal of this three day sprint was to contribute back the work we did for a recent client project. We also planned to upgrade Plone-React to React 16 and improving the error handling.

A small and very focused sprint with the core contributors of Plone-React was the perfect opportunity to do so. Participants were Rob Gietema, Roel Bruggink, Asko Soukka, Victor Fernandez de Alba, Carsten Senger and me.

Sprint Day 1

We started the first day of the sprint by doing a quick stand up and planning meeting. We used the days before the sprint to outline our main goals and objectives, so we could get started right away.

IMG 2572 725ac0c3d0acdf94e31efebb6c23f318 Planning board after the stand up

React 16

On the first day Rob and Victor worked on upgrading plone-react to use React 16 and Webpack 4.

We postponed the upgrade to React router version 4, because some of the dependencies that we need, are still in alpha phase and staying with the old version does not hurt right now.

IMG 2576 4ffa46024dfcc49456d639f9059e5581 Card to upgrade to React 16

Error Boundaries

The main reason why we were eager to upgrade to React 16, was a new feature called “error boundaries”, which allows catching errors in React components and handle them gracefully, without failing the entire app.

Victor implemented error boundaries for client and server side components and for the Redux middleware. It is also possible to pass those errors to Sentry for aggregation and further error handling.

Since we are about to launch our first Plone-React-based project in the next weeks, this was something that was especially important for us.

IMG 2580 d0283e182b2c866ad960a1d3c3a44de3 Error message in plone-react

Token Expiration Middleware

Rob added a token expiration middleware to plone-react that improves the handling of JWT auth token in Plone-React. Working with plone-react on a daily basis revealed a few edge cases where the current authentication failed. The new middleware solves those issues.

plone.restapi

Roel and Carsten focused on plone.restapi. Even though this was not planned to be a major topic for the sprint, we had to fix some issues that were causing troubles in the Plone-React frontend.

Carsten worked on fixing an issue that was preventing to allow the API consumer to reset field values to “None”.

Roel started to work on exposing widgets with tagged values via plone.restapi. This is necessary to support autocomplete or reference widgets in Plone-React.

Test Setup

Asko, who joined the sprint remotely, worked on a brand new end-to-end testing setup with Robot Framework. Because this would be way to easy for a super-smart guy like Asko, he decided to wrap the components into a Docker containers, to make it easier for non-Python devs to set it up. Making Plone PIP-installable is also something that we want to have for quite a while. Asko decided to also mix that in. To make this fun, he decided to add support for Jupyter notebook, which makes it super easy to write Robot Framework tests.

IMG 2579 7764cc8777f896ebb057573f308488c1 Hangout with Asko

I started the day discussing the acceptance testing setup with Asko, quietly listening to Rob and Victor and discussing the plone.restapi issues with Roel and Carsten.

Pastanaga Toolbar

During the first day, we started a discussion how plone.restapi could support a toolbar that automatically adapts to the permissions of the logged in users and shows only the actions that this particular user has permission for.

Before the sprint Victor worked on implementing the new super fancy adaptable Pastanaga toolbar, and we were eager to build a proper backend implementation for this.

Victor's tweet with a short demo of the new Pastanaga toolbar

Sprint Day 2

Create-React-App

On the second day of the sprint, Rob and Victor started to look into building a “create-react-app”-like functionality for Plone-React. create-react-app is a widely popular code skeleton generator by the React team at Facebook. It hides lots of complexity from the user (e.g. Webpack, libraries, configuration) and makes it easy to get started with React. This is super important for the adoption of Plone-React because it also allows to use Plone-React as a library and basis for custom client projects.

Rob and Victor created a proof-of-concept that kind of works but it became clear that this requires a lot more effort before this becomes ready-to-use.

Pastanaga Editor

We scheduled a time slot for a discussion and planning session about the new “Pastanaga editor” user experience. We already implemented basic text editing based on the DraftJS editor from Facebook. The current editor allows basic text editing as well as inline (italic, bold, etc.) and block styles (e.g. headlines, bullet points) and external links.

The next step on our agenda to make this editor based on Tiles to make it extendable and allow the user to add images, videos and other media objects.

We agreed on moving forward with an agile approach of building something useful step-by-step, making sure to build a fully functional and useful editor at any point of the development stage, rather than building the full Mosaic-like functionality at once.

Right after this was settled, we started to draft a tiles endpoint that would build the basis for the next iteration of the Pastanaga editor.

Toolbar

Rob started to work on implementing the context-aware toolbar. This was a challenge because the toolbar needs to adapt to the content that is shown in the main column. This means a component deep down the component hierarchy (content) needs to be able to update a component that lives outside of the DOM hierarchy of the parent component. Luckily for us, “Portals” in React support exactly that use case.

Victor worked on the flexbox styling of the toolbar as well as on optimizations of the Webpack configuration.

plone.restapi, Tiles, and Testing

Roel and Carsten continued to work on plone.restapi issues. Roel continued to work on the tagged values representations and Carsten finished the actions endpoint for the context-aware toolbar.

I followed our established documentation-first approach on plone.restapi and wrote the docs for the new tiles endpoint.

With the basic test setup already in place, Asko struggled with the Travis CI setup running different versions of Node and Python at the same time. Problem is that the ZEO version requires the latest Python 2.7.13 which is not shipped by default with Travis CI.

He also worked on a Docker compose option to support an option to run the API server. This would allow faster development of Robot Framework tests since the API server does not need to restart for each test (iteration).

Sprint Day Three

We started the last day of our sprint discussing the create-react-app use case we worked on the day before. Afterwards, we did a hangout with a student who is interested in working on this during the Google Summer of Code 2018.

IMG 2583 f900eaa919019cf508c8a20e6313de48 Hangout with a possible Google Summer of Code student

Right after this call, Asko gave me a tour of the new, Docker compose based, PIP-installable, Jupyter notebook enhanced test setup.

My head was still spinning from Asko’s amazing work, when Rob asked me for a minute to show me a prototype of the new Medium-like Pastanaga editor.

Summary

The sprint was extremely productive and fun. Having a small and dedicated group of developers with a clear goal and focus really worked well for us.

IMG 6425 1 Group photo in front of the kitconcept office

We contributed back all our re-usable code from our client project, upgraded Plone-React to React 16. We fixed some important issues in plone.restapi and build a super fancy test setup that will allow us to further improve the software quality of Plone-React.

We also laid the groundwork for the next important steps forward: Building the new Pastanaga editor with a tiles-based backend and allowing to use Plone-React as an extensible library with a “create-react-app”-like functionality.

We are already looking forward to the upcoming Beethoven Sprint and the Costa Brava sprint where we will continue to push Plone-React, Pastanaga and plone.restapi.

Plone 5.1 Has Been Released

Posted by PLONE.ORG on March 08, 2018 02:48 AM

As with all Plone minor version releases, 5.1 includes many bug fixes and under-the-hood improvements, with a modest selection of enhancements visible to users, editors, and site administrators.

Plone 5.1 contains these user-facing improvements:

  • Plone pages load faster now with better bundling of JavaScript and CSS resources
  • Plone’s built in search indexing is much faster and CPU-efficient
  • Plone supports the display of higher resolution HiDPI (“Retina”) images

Content editors and site administrators will find these enhancements:

  • Site administrators can create, edit, and manage portal actions (menu items and links) through a new control panel
  • Direct linking to groups’ details from the Sharing tab
  • New configuration registry control panel lets site administrators export, import, and add new records
  • Improved HTML filtering for preventing malicious HTML
  • Fine grained control over thumbnail images in portlets and display views
  • Support for image rotation metadata: the image will show up with the rotation that the photographer intended
  • The default order of search results is now configurable

Plone 5.1 also includes these developer-oriented changes:

  • Ability to conditionally import configuration registry records (by checking on the presence or not of specific features and Plone versions)
  • Removal of OpenID login support from Plone core
  • Removal of the old portal_quickinstaller (may be a breaking change for add-on developers)


To get Plone 5.1, please visit the downloads page

For help using or upgrading to Plone 5.1, see the support page

 

Photo by Airman 1st class Alex Echols

The Mountaineers Keep Climbing

Posted by Jazkarta Blog on February 22, 2018 04:40 PM

The Mountaineers' Website's New Look

Some non-profits fund raise for major technology upgrades, then breathe a sigh of relief and ignore technology until things start breaking. Rinse and repeat. Other non-profits avoid this feast-or-famine approach to technology spending by budgeting for continuous improvements. The Mountaineers is in the latter camp. In 2014 they completed a major technology upgrade and launched a new Plone+Salesforce website. Every year since, they have budgeted for significant improvements to their website and its Salesforce.com back end. They did this through modest support contracts with their technology partners (Jazkarta and Percolator Consulting), plus focused spikes of work that were done through a series of agile iterations. Over the last 3 years we have released enhanced versions of the website 10 times, with an overwhelming number of improvements – you can read all about that on their technology blog. These changes have allowed members, staff, volunteers, instructors, and the organization to get better and better at what they do.

The latest update is the biggest one yet. It has brought mountaineers.org and mountaineersbooks.org together into one, integrated website. The new and improved header and main navigation helps users know where they are and get to where they’re going. It looks great! We’re really proud to have helped bring it into the world.

 

Plone Selected for 2018 Google Summer of Code

Posted by PLONE.ORG on February 17, 2018 09:43 PM

Plone, the secure, enterprise-scale Python web content management system, is one of the organizations that have been selected for the 2018 Google Summer of Code (GSoC).

https://summerofcode.withgoogle.com/organizations/5685175707500544/

GSoC is a global program focused on bringing more student developers into open source software development. Students work with an open source organization on a 3 month programming project during their break from school.

Organizations are selected on the basis of their application, which takes the form of a proposal based either on a student’s own idea or one selected from a list compiled by Plone. Proposals are judged on their utility to Plone, depth of planning and likelihood to succeed.

Cris Ewing, the main organizer behind our GSoC 2017 programme, reprises his role as Plone’s GSoC administrator for 2018. Cris is a former Plone Foundation Board member and co-organizer of the 2016 Boston Plone Conference.

Student developers interested in submitting a GSoC proposal to the Plone Foundation should read the introductory information about Summer of Code at https://developers.google.com/open-source/gsoc/ and take a look at Plone’s ideas list at https://plone.org/gsoc. Applications open on March 12, 2018, but students are welcome to get in touch before then in our forum at community.plone.org for feedback on their ideas.

For more information

 

Three New Members Join the Plone Foundation

Posted by PLONE.ORG on February 17, 2018 02:23 PM

The Plone Foundation welcomes 3 new members after unanimous confirmation by the Foundation's Board of Directors on February 8, 2018. Membership in the Foundation is conferred for significant and enduring contributions to the Plone project and community. The Plone Foundation Membership Committee overwhelmingly recommended each applicant for their ongoing contributions to Plone.

Sven Strack

Sven Strack

Sven has given talks and training at several Plone conferences, was the organizer of the Stroopwafel sprint, the creator of Plone Docker images and of the UI installer. He is best known as Plone's documentation czar, having led the Documentation Team for years with verve. His most recent new responsibilities in the Admin and Infrastructure Team have already resulted in the move of critical Foundation servers to new, more reliable facilities.

David Bain

David Bain

David first presented at the 2008 Plone conference and hasn't stopped giving conference talks and training since then. He is a steadfast advocate for Plone, Python, and Jamaica, having organized Plone meetups and Python events in Jamaica, including PyCon Jamaica. David regularly engages with newcomers and old timers alike in our forum, and has been a key member of Plone's Google Summer of Code participation. David's continuously innovative spirit has resulted in new code and new ways of thinking, particularly in the areas of approachability and onboarding.

Franco Pellegrini

franco pellegrini

Franco first burst onto the public Plone scene with his mind-blowing creation, the PloneIDE. He has authored and contributed at least a dozen add-ons, including the popular collective.cover, is a member of the Framework Team, and a contributor to Plone core and the Mockup project. He has given talks and trainings at Plone symposia, conferences, and local events since 2010.

 

 

The Foundation is the trustee for Plone's intellectual property, works to protect and promote Plone, and has 78 active members.

Random News And Updates

Posted by TestTheDocs on February 11, 2018 12:32 PM
Docs We are happy to announce that docs.testthedocs.org is back ! For the moment there is not much to see, we are starting slowly to add more and more content ! Contribute Also, we are looking for contributions !!! We would like to see more content about testing, tools, tip and tricks, etc, etc !!! Future Planning We are playing with a new way of testing, building and deploying our docs.

Plone News Roundup, February 2018

Posted by PLONE.ORG on February 03, 2018 09:24 PM

Welcome to 2018! Plone has continued its flurry of activity since our wonderful conference in Barcelona last October...

Phew! If we missed something, please let us know!

RoboCon 2018 and Robot Framework Jupyter support

Posted by Asko Soukka on January 27, 2018 06:48 PM

It's already over a week since I got back home from the first Robot Framework conference ever – RoboCon 2018. It was a pleasure to be there, and I really feel privileged that I was accepted there as a speaker.

My RoboCon 2018

RoboCon 2018 was a single day conference about Robot Framework test automation ecosystem, held in the heart of Helsinki, Finland, on 18th of January 2018. The conference venue was quite if not completely full, so there must have been around 250 participants. The event was in English and had pretty good international participation. Yet, most of the participants came from Finland, where Robot Framework has become de-facto standard for test automation.

https://1.bp.blogspot.com/-tuaZhsT3Ves/WmwL97tsinI/AAAAAAAABPE/i-cRg8VQBNIO3E89SFY2Z9A4qmDvYAtSwCKgBGAs/s1600/LND_8D2D2079-6B5F-4DC1-A14C-6D30182A63FB.JPG

Pekka Klärck presenting the history, present and future of Robot Framework

RoboCon 2018 had only a single track, so that had to be packed to include something for everyone in its diverse audience. In addition, there was plenty of time and a separate space for networking with the other participants and conference sponsors. There was also organized social program before and after the conference, but unfortunately, I was unable to attend those at this time.

In my opinion the program was well balanced: The conference started with introductory talks, continued with variety of differnet case studies (Kone, Plone and Texas Instruments), and ended with more technical talks about specific Robot Framework addons (SeleniumLibrary, the most awesome new REST library and pabot). And in the middle of everything, there was my personal favorite: Ed Manlove's talk about building successful open source communities. My presentation was called Robot Framework in Plone CMS Project: a case study, story and some technical details, how Robot Framework got successfully adopted in distributed open source community behind Plone.

The most important part of this conference, of course, was getting a lot of Robot Framework users and developers to meet in the same place at the same time. After all, RoboCon 2018 was the first Robot Framework conference ever. My personal absolute highlight during the conference was meeting a former Plonista, Ed Manlove. He was the one who first introduced me to Selenium testing in San Francisco Plone Conference in 2011, and whom I had not seen after that. Until now. I really hope takes less than seven yeras to see him again...

https://3.bp.blogspot.com/-pHUpSJ725vQ/WmwL91OCKzI/AAAAAAAABPE/DA1nMNfVEPcG8NbGnn0V6FdsahlwK5vhwCKgBGAs/s1600/IMG_6460.JPG

Ed Manlove presenting his talk: The Importance of Open Source Communities

Jupyter kernel for Robot Framework

After the conference on Thursday came the single day conference sprint on Friday. And if the conference was a success, the sprint was even more so: the sprint venue, three large office rooms, was packed full of sprinters, many of them participating their first open source sprint (and got a good introduction to open source development from Ed).

https://4.bp.blogspot.com/-j8En_gW7h08/WmwL931Sz7I/AAAAAAAABPE/AEoDgF84z8YMIm7npuP831Fp4bhnY1aJwCKgBGAs/s1600/IMG_6482.JPG

The sprint facilities were provided by Eficode

Because I had to leave early in Friday, I had planned a very specific sprint goal for myself: a MVP Robot Framework kernel for Jupyter notebook.

Jupyter notebook (previously known as IPython notebook) is an open-source web application for creating and sharing documents that contain live code, equations, visualizations and narrative text. The architecture behind Jupyter notebook separates the notebook application from its language specific ”kernels” that are responsible for executing the code in notebooks. Syntax highlighting in notebooks, on the other hand, is provided by CodeMirrorproject for the interactive frontend, and Pygments for server side generated highlighting.

I'm happy to say that I made it. And the more I use it, the more confident I get on that Jupyter makes a great platform for learning also Robot Framework. And not only for learning by yourself, but also for sharing your notes with others.

Check my example notebooks to judge the kernel by yourself.

https://3.bp.blogspot.com/-M10Tz__nv9g/WmygIgebHLI/AAAAAAAABPc/3-_87CO3LBwvX3ID8STfCo6BdO1jnkiTgCKgBGAs/s1600/Screenshot-2018-1-27%2BNotebook%2Bon%2Bnbviewer.png

Please, note that while these examples are static renderings at http://nbviewer.jupyter.org/, they can be opened and interacted live in any running Jupyter notebook with the new kernel and required Python packages. See the repository for more details.

The main Jupyter Robot Framework support features shown in those examplesare:

  • Support for defining and executing Robot Framework test suite cumulatively step by step in successive notebook cells. The main limitation is that each cell should start with a test suite section header (settings, variables, keywords or test suites) even when the same header was already defined in some cell before.
  • HTML log and report files are linked below the executed cells containing the tests. Both files are actually bundled with the notebook in a way that sharing the notebook also shares the log and report files.
  • Images generated during test execution are shown below the executed cells generating the images. Similarly to HTML logs and reports also images are bundled with the notebook for sharing.
  • Support for %%python module LibaryName ”cell magic” to allow defining custom Robot Framework keyword libraries in Python in fly. Once thell cell with a Python library class definition is executed, it can be imported in a successive Robot Framework code cell.
  • Syntax highlighting. But, unfortunately, until the CodeMirror plugin derived from brackets-robotframework-project is accepted into upstream, it must be manually patched into CodeMirror version shipped with Jupyter notebook-distribution. (I have not yet submitted a pull for it.)
  • If the last keyword of the last test case in the executed cell returns JSON string, it is rendered as cell execution output. I added this quite specific feature to make it more fun to learn RESTinstance library with Jupyter (Output keyword of RESTinstance library returns JSON).

Obviously, while the current versions is already fully functional on Python 3, there's still a lot of work (QA, packaging and Python 2 support) left to polish the code for release. I'm looking forward to finish it during the spring.

Happy hacking! And hopefully see you in RoboCon next year – or whenever it is organized and I'll manage to participate it for the next time! :)

Pastanaga icon system

Posted by kitconcept GmbH on January 25, 2018 07:24 PM

The way we deal with icons in the web has evolved over the years. Images, images sprites, fonts, SVG, SVG sprites… I’ve been looking lately for the current best practice in order to include it in Pastanaga and I wanted to share with you my results. Please note that it’s not closed and I’m open to suggestions. PRs are welcome too!

Abandon font-based systems

It was clear to me that font-based icon systems are a no longer an option today. For several reasons:

  • The font is loaded every single time on the page, regardless if we use all the icons or none of them. This bloats the application size and forces an additional request (in the best case scenario).
  • An existing font is difficult to create, maintain and update. You can use some online (free) services to do that and even you can forge your custom icon font with them but it’s cumbersome and not practical.
  • Forces you to maintain a parallel CSS that maps the icon name with its actual character in the font (which is obscure). The font creation tool helps you with that, but…
  • Extending them with new icons is also complex, especially for newbies, and you need access to the source and reload the source in the same tool that was created.

only to name a few.

Time to move on: inlining SVG

The rise of SVG in modern web developing is for a good reason:

  • SVG is a vector format, so it looks great in HiDPI displays
  • It’s lightweight and portable, since it’s not a binary file
  • It can be styled (and animated) easily using CSS (provided they are inlined, not used with the <img /> tag
  • You can control it via JS

My initial feeling was that using a SVG sprite based system would be the best approach, but I soon was discouraged, after reading to Chris Coyier, CSSTricks: A Pretty Good SVG Icon System

All in to inlining SVGs, then.

So, we need an SVG icon system. Luckily for us, Pastanaga already has a complete set of icons based on SVG organized in one file per icon.

Goals

Our main goal is to provide inline SVGs in our applications, having in mind:

  • It should be performant and small in size
  • Only the used icons should be loaded in the given view, compatible with lazy loading
  • Has to be a no-brainer and clutter-less from the developer point of view
  • You should be able to extend (or override) the default icon set with your own icons easily
  • Valid for all modern frameworks, with focus on Angular and React

Harnessing the power of Webpack and modern JS

As developers we want to use the tooling that we have at our hands in the best possible way. So our icon system should use simple ES6/7/* and TypeScript conventions.

import myIcon from './icons/my-nice-icon.svg';
import Icon from './components/Icon';

and the from JSX:

<Icon name={myIcon} />

or angular template:

<Icon [name]="myIcon"></icon>

or

<div icon [name]="myIcon"></div>

Deconstructing the SVG and put it back together again

According to all the use cases shown in this interesting article by Amelia Bellamy-Royds in CSSTricks: How to Scale SVG the most sensible approach when inlining SVGs is to simply just set the viewBox on your <svg> tag, and set one of height or width to auto. The browser will adjust it so that the overall aspect ratio matches the viewBox. As Amelia points out, that would work for all modern browsers back until 2014. If we have to support older ones, we will need to apply for those the famous padding-bottom hack. Let’s keep things simple for now.

Let’s assume that our SVG is not perfect, and we want to have the all the flexibility that a modern browser can achieve handling SVGs. We will take the existing SVG, deconstruct it and get all the SVG attributes, then the content. We will then put it all together in our components, exactly the way we want it.

The Webpack part

We can accomplish all our goals by using a Webpack loaders combo for loading SVG:

{
    test: /\.svg$/,
    include: path.join(paths.appSrc, 'icons'),
    use: [
        {
        loader: 'svg-loader',
        },
        {
        loader: 'svgo-loader',
        options: {
            plugins: [
            { removeTitle: true },
            { convertPathData: false },
            { removeUselessStrokeAndFill: true },
            { removeViewBox: false },
            ],
        },
        },
    ],
},

We will use svg-loader a super simple inline svg loader that provides you extra flexibility when handling your SVG. Initially I tried the popular Webpack Team’s svg-inline-loader but it was not that flexible at the end. svg-loader returns an object with the contents and the attributes of the svg separatedly that we can later manipulate in our components. We are also filtering the SVG using the well known SVGO utility svgo-loader, we can extend or add more filtering options to optimize our SVGs thanks to it.

We are also restricting this loader to the icons folder, just in case we are handling the other SVGs in our app differently, but of course, you can use it for all SVGs removing the include key.

React

Make it work in React is very straight forward. We need to add the loader to our Webpack config, then add an icons folder and the Icon component.

import React from 'react';
import PropTypes from 'prop-types';

const defaultSize = '100%';

const Icon = ({ name, size, color }) => (
  <svg
    xmlns={name.attributes.xmlns}
    viewBox={name.attributes.viewBox}
    style={{ height: size, width: 'auto', fill: color }}
    dangerouslySetInnerHTML={{ __html: name.content }}
  />
);

Icon.propTypes = {
  name: PropTypes.shape({
    xmlns: PropTypes.string,
    viewBox: PropTypes.string,
    content: PropTypes.string,
  }).isRequired,
  size: PropTypes.string,
  color: PropTypes.string,
};

Icon.defaultProps = {
  size: defaultSize,
  color: null,
};

export default Icon;

That’s it. Our React component takes as props: the name of the imported module of the SVG, the size, and the color. If not given, the SVG will inherit the fill color set in the parent element (or itself). Also, if not specified, the SVG will scale to the parent container height.

Take a look into the JSX of the example

<div style={{ height: '100px' }}>
    <Icon name={Add} />
</div>
<Icon name={Add} size="45px" />
<Icon name={Add} size="45px" color="red" />
<Icon name={Plone} size="60px" color="#1782BE" />
<Icon name={Guillotina} size="60px" color="#EC5528" />

Angular

For the angular icon component we needed the same recipe for the Webpack config and this icon component.

import {
    Component,
    Input,
    HostBinding,
    ViewEncapsulation,
    ChangeDetectionStrategy } from '@angular/core';

import { DomSanitizer, SafeHtml } from '@angular/platform-browser';
import { OnInit } from '@angular/core';

const defaultSize = '100%';

@Component({
  // tslint:disable-next-line:component-selector
  selector: '[icon], icon',
  template: `
  <svg
    [attr.viewBox]="name.attributes.viewBox"
    [attr.xmlns]="name.attributes.xmlns"
    [innerHTML]="svgContent"
    [style.height]="height"
    [style.width]="'auto'"
    [style.fill]="color"
    >
  </svg>
  `,
  encapsulation: ViewEncapsulation.None,
  changeDetection: ChangeDetectionStrategy.OnPush
})
export class IconComponent implements OnInit {

  constructor(private sanitizer: DomSanitizer) {}

  svgContent: SafeHtml;
  defaultSize = defaultSize;
  height: string;

  @Input() color: string;
  @Input() size: string;
  @Input() name;

  ngOnInit() {
    this.svgContent = this.sanitizer.bypassSecurityTrustHtml(this.name.content);
    this.height = this.size ? this.size : defaultSize;
  }

}

We also use the same approach using the component, the Angular template way:

<div icon [name]="Add"></div>
<div icon [name]="Add" color="green"></div>
<icon [name]="Add" color="red" size="45px"></icon>
<icon [name]="Plone" color="#1782BE" size="60px"></icon>
<icon [name]="Guillotina" color="#EC5528" size="60px"></icon>

Our Angular component takes the same three properties as the React one.

In addition, Typescript forces us to overcome some tiny things.

Typings

In order to be able to import the SVG as a module, we need to add this typing to our app:

declare module "*.svg" {
  const content: any;
  export default content;
}

Add the imported SVG object as a Class member

The Angular template won’t be able to use it if the imported SVG object is not a Class member, like:

import { Component } from '@angular/core';
import Add from '../icons/add.svg';
import Plone from '../icons/plone.svg';
import Guillotina from '../icons/guillotina.svg';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  Add = Add;
  Plone = Plone;
  Guillotina = Guillotina;
}

Conclusion

While there are other approaches out there like the Icon component that @angular/material has, they all feel to me like too much and all of them are bloated with lots of options that we don’t really need. I’d like to use a more lightweight and approachable solution like the exposed here that only does what we really need. At the end, it’s not rocket science.

If you have any suggestion, please contact me or open an issue on Github. PRs are welcome!

collective.recipe.backup version 4

Posted by Maurits van Rees on January 25, 2018 12:48 PM

Since the end of 2017, there is a new version 4.0 of collective.recipe.backup. There are lots of changes since version 3.1. Let's see some of the highlights.

Safety and exactness of restore

  • When restoring, first run checks for all filestorages and blobstorages. When one of the backups is missing, we quit with an error. This avoids restoring a filestorage and then getting into trouble due to a missing blobstorage backup.
  • When restoring to a specific date, find the first blob backup at or before the specified date. Otherwise fail. The repozo script does the same. We used to pick the first blob backup after the specified date, because we assumed that the user would specify the exact date that is in the filestorage backup. Note that the timestamp of the filestorage and blobstorage backups may be a few seconds or minutes apart. So now the user should pick the date of the blob backup or slightly later. This date will give the same result with 3.1 and 4.0. But: when you use the new blob_timestamps == true option, these dates are the same.

Blob timestamps

  • Added blob_timestamps option. Default is false. By default we create blobstorage.0. The next time, we rotate this to blobstorage.1 and create a new blobstorage.0. With blob_timestamps = true, we create stable directory names that we do not rotate. They get a timestamp, just like the repozo backup. For example: blobstorage.1972-12-25-01-02-03.
  • When backing up a blobstorage, use the timestamp of the latest filestorage backup. If a blob backup with that name is already there, then there were no database changes, so we do not make a backup.
  • Automatically remove old blobs backups that have no corresponding filestorage backup. We compare the timestamp of the oldest filestorage backup with the timestamps of the blob backups. This can be the name, if you use blob_timestamps = true, or the modification date of the blob backup. This means that the keep_blob_days option is ignored, unless you use only_blobs = true.
  • Note: it is fine to switch to blob_timestamps even when you already have 'old' backups. Restoring those will still work.
  • blob_timestamps = true may become the new default later (maybe 4.1). This may even become the only valid value later (maybe 5.0), removing the creation of blobstorage.0. This would simplify the code. If you don't like this, please speak up and create an issue.

Archiving and compressing blobs

  • Renamed gzip_blob option to archive_blob. Kept the old name as alias for backwards compatibility. This makes room for letting this create an archive without zipping it.
  • Added compress_blob option. Default is false. This is only used when the archive_blob option is true. When switched on, it will compress the archive, resulting in a .tar.gz instead of a tar file. When restoring, we always look for both compressed and normal archives. We used to always compress them, but in most cases it hardly decreases the size and it takes a long time anyway. I have seen archiving take 15 seconds, and compressing take an additional 45 seconds. The result was an archive of 5.0 GB instead of 5.1 GB.
  • Note that with both archive_blob and blob_timestamps set to true, you get filenames like blobstorage.1972-12-25-01-02-03.tar.
  • Added incremental_blobs option. This creates tarballs with only the changes compared to the previous blob backups. This option is ignored when the archive_blob option is false.

Various

  • No longer create the fullbackup script by default. You can still enable it by setting enable_fullbackup to true.
  • Added Python 3 support. The integration with plone.recipe.zope2instance is not tested there, because there is no Python 3 compatible release of it yet.

Upgrading

  • In most cases you can simply use the new version without changes.
  • Adding blob_timestamps = true is highly recommended. If you do this, you can remove the keep_blob_days option, unless you use only_blobs = true.
  • If you want the fullbackup script, enable it by setting enable_fullbackup to true.
  • When you used the gzip_blob option, you should rename this to archive_blob. Maybe enable the compress_blob option, but you are probably better off without this.

plone.restapi 1.0.0 released - A Story of Successful Open Source Collaboration

Posted by kitconcept GmbH on January 19, 2018 07:24 PM

After more than three years of development and 25 alpha and one beta release, we are very happy and proud to announce the release of plone.restapi 1.0.0.

plone.restapi is a RESTful hypermedia API for the Plone Open Source Content Management System. It exposes the unique and powerful features of Plone, including the core content management features as well as dynamic content type creation, workflows, permissions, versioning and more.

plone.restapi builds a bridge between a stable and mature Open Source CMS that has been around for more than 15 years and modern state-of-the-art JavaScript-based solutions like React, Angular, Vue and others.

A Little Bit of History

PLOG 2014

The development of plone.restapi started in beautiful Sorrento, Italy at the Plone Open Garden in 2014 after I gave a talk about building an AngularJS application on top of Plone.

plog2014

A long discussion with Simone Deponti under the Italian sun, about REST API design principles and hypermedia (of course), led to the first commit and the development of a first proof-of-concept implementation.

plone.rest and PLOG 2015

One year later we gathered in Sorrento again. Laurence Rowe, Ramon Navarro Bosch and I spent our days and nights discussing the details of the REST API design and drafted multiple endpoints.

plog2015

One of the main obstacles to building a RESTful API on top of Plone was the missing ZPublisher support for HTTP verbs such as PATCH, PUT or DELETE. In 2015, I sat together with Ramon Navarro Bosch in Sorrento (again) and we (he really did all the heavy lifting) started to build plone.rest, a small package that adds support for HTTP verbs to Plone.

Archetypes and Serializers

We never planned to support Archetypes in plone.restapi. Though, when Thomas Buchberger and Lukas Graf came along and offered to build it, we did not object (of course not, this is Open Source). Their company 4teamwork planned to build a REST api on top of Plone for their OneGov GEVER platform.

Instead of building something on their own, they decided to join forces and share their work and code with the community. Along the way, they heavily refactored the code, added tons of adapters for loose coupling and the ability to customize the JSON serialization.

After this, we were confident to do a first alpha release of plone.restapi on June 14th 2016.

Beethoven Sprint

In March 2017, fourteen Plone developers from eight different countries gathered in Bonn, at the kitconcept office, for the Beethoven Sprint to work on plone.restapi and related topics. In addition to sorting out the last remaining design decision, many exciting new projects were started and announced.

BeethovenSprint065

Angular

At the Beethoven sprint, Eric Brehault started to work on an Angular SDK for plone.restapi. A release followed soon and Eric gave a very successful and crowded training at the Plone Conference 2017 in Barcelona.

Today, Angular SDK is a mature package for Angular 2 that makes it really easy for front-end developers to interact with Plone and a fantastic starting point for newbies.

Eric and I mentored Noel Varghese during last year’s Google Summer of Code to build a Progressive Web App for Plone in Angular 2. Noel gave a nice presentation of his successful project at the Plone Conference in Barcelona.

React

Rob Gietema and Roel Bruggink started to build a React-based front-end on top of plone.restapi at the Beethoven sprint in Bonn. Later that year, they went to Toulouse in September 2017 to implement the Pastanaga CSS together with the Plone Angular team.

In November they visited Bonn again for the Pastanaga Sprint where we started to implement the new Pastanaga UI for plone-react.

BeethovenSprint183

At kitconcept, we started to use plone-react with Pastanaga for an ongoing project. We can’t wait to release our work and contribute it back to the community.

Vue.JS

Inspired by the Angular SDK and plone-react, Kevin Bieri started to build a VueJS plone-vuejs implementation on top of Plone at the Plone Conference 2017 in Bareclona.

Guillotina

Ramon Navarro Bosch and Nathan van Gheem revelead the name of “Guillotina”, a blazing fast async Python framework that shares the public API with plone.restapi at the Beethoven sprint in Bonn.

Successful Open Source Collaboration

plone.restapi started with an idea and discussions. People and companies jumped in and contributed in many ways that haven’t been dreaming about at first.

Simone, Laurence, Ramon and other helped to shape the initial idea. Lukas, Thomas, Roel, Carsten, Victor, Mikel and many others contributed new endpoints, bugfixes, etc.

Eric, Rob, Noel, Kevin, and others started to build frameworks and solutions on top of plone.restapi.

Many companies such as 4teamwork, Code Syntax, Markina Corpus, VNC invested and contributed to plone.restapi.

The Plone Foundation always supported our efforts by funding sprints.

plone.restapi is a true community effort and the joy that we feel when collaborating with wonderful people pays us back for the countless hours we spend on hacking on code.

Future Plans

A Plone Improvement Proportal (PLIP) to ship Plone 5.2 with plone.restapi has been accepted by the Plone Framework Team:

https://github.com/plone/Products.CMFPlone/issues/2177

With plone.restapi considered stable and close to being feature complete, we will continue working on what could become the next Plone…stay tuned.

Configuring the ufw firewall to allow Cloudflare IP addresses

Posted by T. Kim Nguyen on January 03, 2018 01:56 AM

I have a Linode running Ubuntu 16.04, and I use the ufw firewall.

I have a web site running on that server, originally accessible via HTTPS on port 443 from anywhere on the internet.

The domain for that web site is managed via Cloudflare. I want the site to be available only through the domain, and not via the Linode's IP address.

Cloudflare publishes the IP addresses it uses to access your web site: https://www.cloudflare.com/ips/

Here is a page describing the overall idea of using ufw to allow access to your web site only from those Cloudflare IP addresses: https://www.ajsalkeld.com/blog/tutorial/2016/08/01/how-to-use-ufw-to-whitelist-cloudflare-ips-ubuntu-debian-digitalocean/

In this repo https://github.com/Paul-Reed/cloudflare-ufw there is a script that does this: https://github.com/Paul-Reed/cloudflare-ufw/blob/master/cloudflare-ufw.sh

I modified it a bit so that:

  • it uses the /tmp directory
  • it uses a unique filename (containing the current process ID) when retrieving the Cloudflare IP addresses
  • it specifically allows connections only on port 443 (you may want to allow connections on port 80 as well or instead)
  • it just outputs to the screen the commands that it would issue using ufw; If the commands look sane/good to you, copy and paste them into your terminal to run them

Here is my script:

#!/bin/sh
cd /tmp
wget https://www.cloudflare.com/ips-v4 -O ips-v4-$$.tmp
wget https://www.cloudflare.com/ips-v6 -O ips-v6-$$.tmp

for cfip in `cat ips-v4-$$.tmp`; do echo "ufw allow from $cfip to any port 443 proto tcp"; done
for cfip in `cat ips-v6-$$.tmp`; do echo "ufw allow from $cfip to any port 443 proto tcp"; done

Once I ran the script and copied and pasted its output into a terminal, ufw was configured as follows:

# ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 22                         ALLOW IN    Anywhere
[ 2] 443/tcp                    ALLOW IN    103.21.244.0/22
[ 3] 443/tcp                    ALLOW IN    103.22.200.0/22
[ 4] 443/tcp                    ALLOW IN    103.31.4.0/22
[ 5] 443/tcp                    ALLOW IN    104.16.0.0/12
[ 6] 443/tcp                    ALLOW IN    108.162.192.0/18
[ 7] 443/tcp                    ALLOW IN    131.0.72.0/22
[ 8] 443/tcp                    ALLOW IN    141.101.64.0/18
[ 9] 443/tcp                    ALLOW IN    162.158.0.0/15
[10] 443/tcp                    ALLOW IN    172.64.0.0/13
[11] 443/tcp                    ALLOW IN    173.245.48.0/20
[12] 443/tcp                    ALLOW IN    188.114.96.0/20
[13] 443/tcp                    ALLOW IN    190.93.240.0/20
[14] 443/tcp                    ALLOW IN    197.234.240.0/22
[15] 443/tcp                    ALLOW IN    198.41.128.0/17
[16] 22 (v6)                    ALLOW IN    Anywhere (v6)
[17] 443/tcp                    ALLOW IN    2400:cb00::/32
[18] 443/tcp                    ALLOW IN    2405:8100::/32
[19] 443/tcp                    ALLOW IN    2405:b500::/32
[20] 443/tcp                    ALLOW IN    2606:4700::/32
[21] 443/tcp                    ALLOW IN    2803:f800::/32
[22] 443/tcp                    ALLOW IN    2c0f:f248::/32
[23] 443/tcp                    ALLOW IN    2a06:98c0::/29

I tested by browsing to my web site's domain (e.g. https://mysite.com) and it worked. Then I tried to browse to my server's IP address (e.g. https://123.45.67.89) and it did not work, as expected and as intended.


Update: January 3, 2018: Thank you to Florian Schulze who suggested the use of Cloudflare's authenticated origin pulls, described at https://support.cloudflare.com/hc/en-us/articles/204494148-Setting-up-NGINX-to-use-TLS-Authenticated-Origin-Pulls. With this method, you don't have to worry that Cloudflare may have changed its IP addresses (the reason why you would need to update your ufw rules periodically).

There is also TLS client side authentication, a feature described at https://support.cloudflare.com/hc/en-us/articles/115000088491-Cloudflare-TLS-Client-Auth. It is, however, available only to Enterprise Cloudflare customers.

Continuous Performance Analysis with Lighthouse and Jenkins

Posted by kitconcept GmbH on December 22, 2017 06:11 AM

Lighthouse is an open-source, automated tool for improving the quality of web pages by Google. It measures the performance of a website and provides metrics for accessibility, best practices for modern web apps, search engine optimization, and assess web applications for adherence to Progressive Web App standards.

Lighthouse Logo Lighthouse Logo

Together with WebPageTest and Google Page Speed Insights it is an indispensable tool to optimize your website performance.

Installation

Lighthouse can be installed in any JavaScript-based project by just running ‘npm install’:

$ npm install lighthouse -g

If you don’t have a package.json in your project, just install npm and run ‘npm init’ before installing.

Running Lighthouse

You can check the performance of any website by calling the ‘lighthouse’ command with the URL of the website you want to test. Append the --view parameter to show the HTML report, right after the command has finished:

$ lighthouse https://kitconcept.com --view

The report will give you five different ratings about PWA, performance, accessibility, performance best practices, and SEO.

Lighthouse Results Lighthouse Results

Continuous Performance Measurements

If you run your performance test every now and then, you always risk to hurt your website performance without noticing. If a performance regression happens unnoticed, it is usually very hard and time consuming to figure out which change caused the performance regression.

You can easily fix this and save lots of time when you run your performance tests and analysis continuously.

Unfortunately Lighthouse does not allow you to set performance test specifications that your CI system can test against, like WebPageTest or Google Page Speed Insights do (we will cover those tools in later blog posts). Though, it is still very convenient to run the performance test on a regular basis for each commit and include them into your CI report.

Install Lighthouse locally for CI

When it comes to a Continuous Integration, a local installation is prefered over a global one, which is usually harder to manage and to maintain. Especially if you have multiple projects with different sets of package versions on your CI.

Therefore we install Lighthouse locally in our project directory:

$ npm install lighthouse --save-dev

This command will install Lighthouse to your local package.json file. We recommend to use yarn or npm package-lock.json to lock down the package version you are using for a repeatable and stable project build.

For convenience, we add a “lighthouse” script to our package.json:

"scripts": {
  "lighthouse:ci": "node_modules/lighthouse/lighthouse-cli/index.js \
  --output-path=./lighthouse-report.html --quiet \
  --chrome-flags='--headless' https://kitconcept.com"
}

We call the locally installed lighthouse binary and set a static output path (by default, Lighthouse creates a file with the current date/time in the filename which makes it harder to publish on your CI).

We also include the --quiet option and run it on headless chrome, so we don’t need to install and run an X server on our CI system.

At the end, we hard-code our project URL into the command so we do not have to type it manually each time we run this command.

Now we can just run:

$ npm run lighthouse:ci

and it will create a nice HTML report that we can publish in our CI.

Configure Lighthouse for your local development environment

For convenience, we also add a command that you can run locally:

"scripts": {
  "lighthouse": "node_modules/lighthouse/lighthouse-cli/index.js \
  --output-path=./lighthouse-report.html --quiet \
  --chrome-flags='--headless' https://kitconcept.com/blog"
}

The --view parameter will fire up a browser with the report at the end of the performance analysis. This is something we clearly don’t want on our CI system.

Publish Lighthouse Reports in Jenkins CI

Travis and other lightweight CI system usually lack the option to publish any reports except the command line output. Though, if you are using Jenkins CI, you can use the HTML publisher plugin to publish your Lighthouse report.

sh 'npm install'
sh 'npm run lighthouse'
publishHTML (target: [
  allowMissing: false,
  alwaysLinkToLastBuild: false,
  keepAll: true,
  reportDir: '.',
  reportFiles: 'lighthouse-report.html',
  reportName: "Lighthouse"
])

After adding publishHTML to your Jenkins pipeline, you will see a “Lighthouse” link under the ‘Artifacts’ tab:

Link to Lighthouse report in Jenkins Link to Lighthouse report in Jenkins

There is a caveat though. Jenkins 1.641 / 1652.3 introduce the Content-Security-Policy header to static files served by Jenkins. The default header is set to a very restrictive set of permissions to protect Jenkins users from malicious HTML/JS files in workspaces.

To allow Jenkins to display the Lighthouse reports, we have to add the following JAVA_ARGS to the Jenkins startup (for instance by adding the following line to your /etc/default/jenkins file):

JAVA_ARGS="-Dhudson.model.DirectoryBrowserSupport.CSP=\"sandbox
allow-scripts; default-src 'unsafe-inline'; img-src * data:\""

For more details see the Content Security Policy Reference and the Jenkins docs on configuring Content Security Policy.

After you fixed the Content Security Policy of your Jenkins you will see the full report when clicking on the ‘Lighthouse’ link on the ‘Artifacts’ tab on your Jenkins build:

Lighthouse full report in Jenkins Lighthouse Report in Jenkins

Jenkins Declarative Pipeline Stage for Performance Tests

A full declarative pipeline stage for lighthouse looks like this:

stage('Performance Tests') {
  agent {
    label 'master'
  }
  when {
    branch 'master'
  }
  steps {
    deleteDir()
    checkout scm
    sh 'npm install'
    sh 'npm run lighthouse'
  }
  post {
    always {
      publishHTML (target: [
        allowMissing: false,
        alwaysLinkToLastBuild: false,
        keepAll: true,
        reportDir: '.',
        reportFiles: 'lighthouse-report.html',
        reportName: "Lighthouse"
      ])
    }
  }
}

We run the performance test stage on ‘master’ agents and only on the master branch. The steps performed are a simple “npm install” to set up the project build and then we run ‘npm run lighthouse’ to produce the HTML report. If you already have an npm build from a previous step you can of course just unstash the build artifact.

Jenkins pipeline with lighthouse performance tests Jenkins pipeline with Lighthouse performance tests stage

Summary

Lighthouse is a valuable and indispensable tool if you want to deliver a fast and user friendly website. Running the analysis on a continuous basis on your CI is a good idea if you take performance seriously. Setting it up is fast and easy. Maybe in the future Lighthouse will also provide a testspec feature that will allow us to fail a CI build (or mark it as unstable) on performance regressions. Though, if you run WebPageTest or Google Page Speed Insights additionally, this is not really needed.

Jazkarta Sponsors Northwest Youth Leadership Summit

Posted by Jazkarta Blog on December 07, 2017 08:21 PM

NWYLS Group Shot

Jazkarta is pleased to have recently sponsored the North Cascades Institute‘s Northwest Youth Leadership Summit. This event is intended to empower Cascadia’s future leaders in conservation by:

  • Enhancing their skills in preparation for job and college applications
  • ​Connecting with regional environmental organizations and businesses to learn about jobs and internships
  • Learning from like-minded peers about career options available in the conservation, outdoor and environmental fields

More than 220 students participated and are now better equipped to take action towards conservation. The Summit was free to all participants to ensure that underrepresented youth are given opportunities to get involved in the outdoor and environmental fields.

The sponsorship added another dimension to our existing partnership with North Cascades Institute. Just before the summit, we had given the non-profit’s Plone+Salesforce website a mobile refresh to make it work smoothly on phones and tablets. If we say so ourselves, the results are quite beautiful. Kudos to Neal Maher for the designs and to the Jazkarta team (Christine Winckler and David Glick) for a smooth implementation.

North Cascades Institute is not the only environmental non-profit organization that Jazkarta is working with – we created The Mountaineers‘s website and the Washington Trails Association ‘s volunteer management system. Both organizations were involved in the Summit. It was hosted at The Mountaineers’ Seattle Program Center, here is one of the students using the climbing wall.

NWYLS Student on The Mountaineers Climing Wall

Andrew Pringle of the Washington Trails Association led a breakout session titled “Trip Planning 101: An Introduction to Leading Backcountry Adventures”, and both organizations ran booths, talking with participants about activities, internships and employment options for young outdoor leaders.  Here’s Andrew at the WTA booth.

WTA's Andrew Pringle at the NWYLS

We feel very lucky to be helping all of these organizations further their missions.

 

— Photos by North Cascades Institute staff