Planet Plone

This is where developers and integrators write about Plone, and is your best source for news and developments from the community.

October 23, 2014

Asko Soukka: Too many ways to do async tasks with Plone

by Asko Soukka at 2014-10-23T15:13:15Z

Triggering asynchronous tasks from Plone is hard, we hear. And that's actually quite surprising, given that, from its very beginning, Plone has been running on top of the first asynchronous web server written in Python, medusa.

Of course, there exist many, too many, different solutions to run asynchronous task with Plone:

  • plone.app.async is the only one in Plone-namespace, and probably the most criticized one, because of using ZODB to persist its task queue
  • netsight.async on the other hand being simpler by just executing the the given task outside Zope worker pool (but requiring its own database connection).
  • finally, if you happen to like Celery, Nathan Van Gheem is working on a simple Celery-integration, collective.celery, based on an earlier work by David Glick.

To add insult to injury, I've ended up developing a more than one method more, because of, being warned about plone.app.async, being hit hard by the opinionated internals of Celery, being unaware of netsight.async, and because single solution has not fit all my use cases.

I believe, my various use cases can mostly be fit into these categories:

  • Executing simple tasks with unpredictable execution time so that the execution cannot block all of the valuable Zope worker threads serving HTTP requests (amount of threads is fixed in Zope, because ZODB connection cached cannot be shared between simultaneous requests and one can afford only so much server memory per site).

    Examples: communicating to external services, loading an external RSS feed, ...

  • Queueing a lot of background tasks to be executed now or later, because possible results can be delivered asynchronously (e.g. user can return to see it later, can get notified about finished tasks, etc), or when it would benefit to be able to distribute the work between multiple Zope worker instances.

    Examples: converting files, encoding videos, burning PDFs, sending a lot of emails, ...

  • Communicating with external services.

    Examples: integration between sites or different systems, synchronizing content between sites, performing migrations, ...

For further reading about all the possible issues when queing asynchronous tasks, I'd recommend Whichert Akkermans' blog post about task queues.

So, here's the summary, from my simpliest approach solution to enterprise messaging with RabbitMQ:

ZPublisher stream iterator workers


class MyView(BrowserView):

def __call__(self):
return AsyncWorkerStreamIterator(some_callable, self.request)

I've already blogged earlier in detail about how to abuse ZPublisher's stream iterator interface to free the current Zope worker thread and process the current response outside Zope worker threads before letting the response to continue its way towards the requesting client (browser).

An example of this trick is a yet another zip-export add-on collective.jazzport. It exports Plone-folders as zip-files by downloading all those to-be-zipped files separately simply through ZPublisher (or, actually, using site's public address). It can also download files in parallel to use all the available load balanced instances. Yet, because it downloads files only after freeing the current Zope worker instance, it should not block any worker thread by itself (see its browser.py, and iterators.py).

There are two major limitations for this approach (common to all ZPublisher stream iterators):

  • The code should not access ZODB after the worker thread has been freed (unless a completely new connection with new cache is created).
  • This does not help installations with HAProxy or similar front-end proxy with fixed allowed simultaneous requests per Zope instance.

Also, of course, this is not real async, because it keeps the client waiting until the request is completed and cannot distribute work between Zope instances.

collective.futures


class MyView(BrowserView):

def __call__(self):
try:
return futures.result('my_unique_key')
except futures.FutureNotSubmittedError:
futures.submit('my_unique_key', some_callable, 'foo', 'bar')
return u'A placeholder value, which is never really returned.'

collective.futures was the next step from the previous approach. It provides a simple API for registering multiple tasks (which does not need to access ZODB) so that they will be executed outside the current Zope worker thread.

Once all the registered tasks have been executed, the same request will be queued for ZPublisher to be processed again, now with the responses from those registered tasks.

Finally, the response will be returned for the requesting like with any other requests.

collective.futures has the same issues as the previous approach (used in collective.jazzport), and it may also waste resources by processing certain parts of the request twice (like publish traverse).

We use this, for example, for loading external RSS feeds so that the Zope worker threads are freed to process other requests while we are waiting the external services to return us those feeds.

collective.taskqueue


class MyView(BrowserView):

def __call__(self):
taskqueue.add('/Plone/path/to/some/other/view')
return u'Task queued, and a better view could now display a throbber.'

collective.taskqueue should be a real alternative for plone.app.async and netsight.async. I see it as a simple and opinionated sibling of collective.zamqp, and it should be able to handle all the most basic asynchrnous tasks where no other systems are involved.

collective.taskqueue provides one or more named asynchronously consumed task queues, which may contain any number of tasks: asynchronously dispatched simple requests to any traversable resources in Plone.

With out-of-the-box Plone (without any other add-ons or external services) it provides instance local volatile memory based task queues, which are consumed by the other one of the default two Zope worker threads. With redis, it supports persistent task queues with quaranteed delivery and distributed consumption. For example, you could have dedicated Plone instances to only consume those shared task queues from Redis.

To not sound too good to be true, collective.taskqueue does not have any nind of monitoring of the task queues out-of-the-box (only a instance-Z2.log entry with resulted status code for each consumed task is generated).

collective.zamqp


class MyView(BrowserView):

def __call__(self):
producer = getUtility(IProducer, name='my.asyncservice')
producer.register() # bind to successful transaction
producer.publish({'title': u'My title'})
return u'Task queued, and a better view could now display a throbber.'

Finally, collective.zamqp is a very flexible asynchronous framework and RabbitMQ integration for Plone, which I re-wrote from affinitic.zamqp before figuring out any of the previous approaches.

As the story behind it goes, we did use affinitic.zamqp at first, but because of its issues we had to start rewrite to make it more stable and compatible with newer AMQP specifications. At first, I tried to built it on top of Celery, then on top of Kombu (transport framework behind Celery), but at the end it had to be based directly on top of pika (0.9.4), a popular Python AMQP library. Otherwise it would have been really difficult to benefit from all the possible features of RabbitMQ and be compatible with other that Python based services.

collective.zamqp is best used for configuring and executing asynchronous messaging between Plone sites, other Plone sites and other AMQP-connected services. It's also possible to use it to build frontend messaging services (possibly secured using SSL) with RabbitMQ's webstomp server (see the chatbehavior-example). Yet, it has a few problems of its own:

  • it depends on five.grok
  • it's way too tighly integrated with pika 0.9.5, which makes upgrading the integration more difficult than necessary (and pika 0.9.5 has a few serious bugs related to synchronous AMQP connections, luckily not requird for c.zamqp)
  • it has a quite bit of poorly documented magic in how to use it to make all the possible AMQP messaging configurations.

collective.zamqp does not provide monitoring utilities of its own (beyond very detailed logging of messaging events). Yet, the basic monitoring needs can be covered with RabbitMQ's web and console UIs and RESTful APIs, and all decent monitoring tools should have their own RabbitMQ plugins.

For more detailed examples of collective.zamqp, please, see my related StackOverflow answer and our presentation from PloneConf 2012 (more examples are linked from the last slide).

October 22, 2014

Davide Moro: Sqlite array type and Python SQLAlchemy

by davide moro at 2014-10-22T22:22:53Z

I need to write up things just for remembering how I solved a particular issue if occurs in the future.
I need to write up things just for remembering how I solved a particular issue if occurs in the future.

Sqlite (with http://sqlitebrowser.org) is great for rapid prototypes development but it lacks some useful implementations provided by Postgresql (for example the sqlalchemy.dialects.postgresql.ARRAY type).

I solved implementing a SQLAlchemy TypeDecorator with a json serialization:   
Here it is the self-explaining code:
from sqlalchemy.schema import Column
from sqlalchemy.types import (
    Integer,
    String,
    TypeDecorator,
    )
from sqlalchemy import Sequence
from pyramid_sqlalchemy import BaseObject as Base
import json

class ArrayType(TypeDecorator):
    """ Sqlite-like does not support arrays.
        Let's use a custom type decorator.

        See http://docs.sqlalchemy.org/en/latest/core/types.html#sqlalchemy.types.TypeDecorator
    """
    impl = String

    def process_bind_param(self, value, dialect):
        return json.dumps(value)

    def process_result_value(self, value, dialect):
        return json.loads(value)

    def copy(self):
        return ArrayType(self.impl.length)

class Element(Base):
    __tablename__ = 'elements'

    id = Column(Integer(),
                Sequence('element_id_seq'),
                primary_key = True)
    # ...
    myarray = Column(ArrayType())
If you are not using Pyramid just replace the pyramid_sqlalchemy's Base wrapper import with:
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()

October 21, 2014

Starzel.de: Extended Mastering Plone Training in Munich in November

2014-10-21T13:05:00Z

When we prepare a training, we always create more material than what we can walk through in the short amount of time we have available.

We decided that we want to offer you more by extending the training period to 5 days. In 5 days we can cover the material deeper have more exercises and additional topics.

The additional topics include:

  • How to work with relations
  • How to test everything using plone.app.testing and robot-framework
  • Building forms and control-panels with z3c.form

We would love to have you with us.

More information is available on our dedicated training page.

October 19, 2014

Davide Moro: Pyramid starter seed template powered by Yeoman (part 1)

by davide moro at 2014-10-19T21:07:47Z

Book of the month I'm reading this summer: Pylons/Pyramid (http://docs.pylonsproject.org/en/latest).


Pyramid (http://www.pylonsproject.org) is a minimal Python-based web development framework that let you "start small and finish big".

It stole a lot of (good) ideas and concepts from other mature Python web frameworks and it is build with the pluggable and extensible concepts in mind. Read: no need to fork applications.

Furthermore Pyramid is database and template engine agnostic: you are free.

From the very beginning Pyramid allows you to become productive quickly. So why not start with something of useful?

Pyramid + Yeoman

The goal of this experiment is integrate yeoman with Pyramid (or other frameworks like NodeJs/Express with AngularJS or Plone as already did), preserving the yeoman's workflow.

UPDATE 20140926: here you can see a Plone + AngularJS + Yeoman article (collective.angularstarter)

In this article I'll talk about what are the benefits you get integrating your Pyramid app with Yeoman, in future posts I'll discuss how they work under the hood with additional technical details omitted here (each used component deserves an entire blog post).

Yeoman
You might wonder why? Because of the importance of tooling. Since it is very important build an effective developer tooling ecosystem, I want to integrate the simple starter demo app with commonly used tools to help you stay productive. So this simple application prototype it is just an experiment that should help you to integrate with modern web development tools provided by the yeoman workflow stack (http://yeoman.io).

Choosing the right tools is very important for the best develop experience and I cannot work anymore without Yeoman, especially when coding with Javascript.

Grunt
Yeoman it is internally based on three important components (nodejs powered):
  • yo, scaffolding tool like pcreate, paster or zopeskel. It is widely adopted by a large and trasversal community
  • grunt, system used for build, preview and test your software. Gulp is another popular option
  • bower, used for dependency management, so that you no longer have to manually download and manage your scripts
Bower

So with the yeoman's tools you can just code, avoid annoying repetitive tasks and don't worry about:
  • javascript testing setup
  • javascript code linting
  • javascript/css minification and merging
  • image minification
  • html minification
  • switch to CDN versions of you vendor plugins in production mode
  • auto-reload browser
  • much much more
So let's see together what happened to our pyramid starter demo template created with pcreate -t starter integrated with a yeoman's generator-webapp project.

The result will be a Pyramid starter seed project integrated with modern non Python-based web development tools.

Goals

Management of third party assets

You no longer have to manually download and manage your scripts with the Bower package manager.

From http://bower.io:
"""Bower works by fetching and installing packages from all over, taking care of hunting, finding, downloading, and saving the stuff you’re looking for."""
So just type something like: bower install angular-translate --save and you'll get the rigth resource with pinning support.

Tasks automation

Automation, automation, automation.

From http://gruntjs.com:
"""Why use a task runner? In one word: automation. The less work you have to do when performing repetitive tasks like minification, compilation, unit testing, linting, etc, the easier your job becomes. After you've configured it, a task runner can do most of that mundane work for you—and your team—with basically zero effort."""
Examples:
  • grunt serve
  • grunt test
  • grunt build
  • grunt YOUR TASK 
  • etc

Jslint

No more deploy Javascript code with bad indentation, syntax errors or bad code practices.

All syntax errors or bad practise will be found.

Image minification

The build process will detect and minify automatically all your asset images.

Uncss task

Modern (and heavy) UI frameworks like Twitter Bootstrap provide an excellent solution for prototyping your initial project, but most of the times you are using a very minimal subset of their functionalities.

https://twitter.com/davidemoroThis inspiring Addy Osmani's blog post helps you to remove unused css in your pages with a grunt task named grunt-uncss (https://github.com/addyosmani/grunt-uncss):
The original not-minified bootstrap.css weights in at 120 kB before removing unused rule.

Css concat and minification

You can split your css code into different files and then the build process will concat and minify them creating a unique app.css file. This way you write modular and better readable css files, reducing the number of browser requests.

The theme.css file is quite small but in real projects you can save more. In this case:
The configured build pipeline is concat, uncss and cssmin. 122.85 kB (original bootstrap.css) -> 4.64 kB (uncss) -> 3.45 kB (minification)

Automatic CDN-ification

It is handy using unminified versions of third party javascript libraries during development and switch to CDN versions in production mode with well known benefits for your website.

Don't worry: the cdnify task will take care about this boring issue. Automatically.

You save a boring manual and error-prone configuration.

Composable bootstrap.js version

The Pyramid starter project is based on Twitter Bootstrap.

Twitter Bootstrap
Depending on your project you can load the whole Twitter Bootstrap Javascript code at once or including individual plugins.

As you can see the Javascript component of Twitter Bootstrap is very modular: http://getbootstrap.com/javascript. So if you don't use a particular feature, just don't include it.

This way in development mode you will have all individual plugins splitted in different files, in production it will served a unique concatenated and minified Javascript file built automatically.

So if you just need alert.js and dropdown.js you can get a 2.79 kB plugins.js:

The concatenation of alert.js and dropdown.js produces a 7.06 kB, that weight in at 2.79 kB after minification instead of the 8.9 kB (gzipped) bootstrap-min.js corresponding to not gzipped 27.2 kB.

Html (template) minification

Since the ZPT/Chameleon templating language is an extension of HTML with xml syntax,

Brower are able to display unrendered ZPT/Chameleon templates
theorically it can play well with html minificators.

I know, template minification can lead to potential unexpected problems due to minification issues on template files... but this is my personal playground, so let me play please!

So... why not switch to a pre-compiled minified template of your ZPT/Chameleon files when you are in "production mode"?

Obviously during development you will use the original template files.

The interesting side of this approach is that there is no overhead at response time, since the minification task runs just one time before deploying your application. It might be an option if you want just your html minified and you cannot feasibly add to your site or project additional optimization tools at web server level.

Anyway I have tried this mad experiment and... if you don't use too aggressive minification params, it seems to work fine with good results. Try it at your own risk or just disable it. Here you can the effects on the generated index.html used in production:
Template minified (7.62 kB -> 4.16 kB)

Result: a lighter Pyramid

Same results but a lighter Pyramid app:

Let's see how it behave the standard Pyramid starter project:
Standard Pyramid starter project (production.ini)
And the Pyramid starter seed:
Pyramid starter seed (production.ini)
As you can see the seed version is ~38 Kb smaller and more performant.

Useful links

That's all?

No, you can do more, for example:
  • reduce the  number or requests (for example you can merge vendor.css and app.css)
  • create and keep updated css sprites with grunt (https://github.com/Ensighten/grunt-spritesmith)
  • manage and upload all your assets to professional services like Amazon AWS (for example you can serve up all your images, styles and scripts from a S3 bucket + CloudFront). This way Pyramid will be able to handle more requests. Pyramid let you put static media on a separate webserver during production with static_url() in conjunction with add_static_view(), without having to change your templates code
  • generate static gzipped assets with Grunt and let your webserver serve them
  • install and configure dedicated performance modules at webserver level (Apache's mod_pagespeed)
Let me know what you think about that, please. Hope soon I will manage to write the second part of this blog post explaining how I did it. In the mean time you can:

Links

    October 16, 2014

    UW Oshkosh How-To's: How to change versions of Plone in your Vagrant install

    by nguyen at 2014-10-16T21:12:14Z

    New info below! Thanks to Steve McMahon :)

    The Vagrant installer is a very nice, easy way to set up Plone.

    If you recently used the Vagrant install but found Plone 5.0a2 instead of the Plone 4.3.3 you expected, here's how to change it.

    Edit the Vagrantfile and change the line

    UI_URL = "https://launchpad.net/plone/5.0/5.0a2/+download/Plone-5.0a2-UnifiedInstaller.tgz"

    to

    UI_URL = "https://launchpad.net/plone/4.3/4.3.3/+download/Plone-4.3.3-UnifiedInstaller.tgz"

    Save the file.

    Run these commands.  Some of them may not be necessary, depending on whether you've run vagrant recently.

    vagrant halt
    vagrant destroy
    vagrant up

    Alternatively go to Github.com

    Steve McMahon suggests going to Github.com and downloading this directly:

    https://github.com/plone/plonedev.vagrant/archive/4.3-maintenance.zip

    and that this is available via the "Branch" dropdown at

    https://github.com/plone/plonedev.vagrant/

    which takes you to 

    https://github.com/plone/plonedev.vagrant/branches

     

    UW Oshkosh How-To's: Tracking down pesky buildout version pins or requirements

    by nguyen at 2014-10-16T20:51:20Z

    Here are some ways to figure out the cause of conflicting version errors like this when you run buildout:

    While:
      Installing client1.
    Error: There is a version conflict.
    We already have: Products.MimetypesRegistry 2.0.2
    but Products.Archetypes 1.7.14 requires 'Products.MimetypesRegistry>=2.0.3'.

    Buildout cfg files

    First, check any version pins you might have in your *.cfg files.

    In the above example, I would run this:

    grep Products.MimetypesRegistry *.cfg

    in my zinstance or zeocluster directory.  This will tell me which files refer to the offending add-on.

    Extends

    If the only references to the add-on correctly show the version you want (and you can't see where the wrong version is being pinned), next check if your buildout.cfg "extends" values are correctly ordered.  

    If your "known good sets" ("KGS") come *after* your local version pins (in versions.cfg in the example below), you may get conflicts.

    The following shows the correct ordering:

    extends =
        http://dist.plone.org/release/4.1.6/versions.cfg
        http://good-py.appspot.com/release/dexterity/1.2.1
        versions.cfg

    This is exactly what happened to me... I had a KGS for an old version of Dexterity, and it was that KGS that had a version pin that conflicted with my local pins in my versions.cfg.

    Buildout can help

    The way I found out that it was a KGS that was conflicting is that I ran "buildout annotate" so that it told me where its various version pins were coming from:

    bin/buildout annotate

    which provided this output:

    ...
        ...
    Products.MimetypesRegistry= 2.0.2
        http://good-py.appspot.com/release/dexterity/1.0.1?plone=4.1
    ...
        ...

     

     

     

    October 15, 2014

    Tom Gross: Porting tests to plone.app.testing for Plone 5

    by Tom at 2014-10-15T20:00:00Z

    A major version of a piece of software always means to leave behind some burdon. Plone ships with two testing framworks since Plone 4. Now it is time to get rid of one of them: PloneTestCase. With the newer plone.app.testing framework it is possible to specify layers to encapsulate testing scenarios and dependencies. I don't want to compete the excelent plone.app.testing documentation here but provide some tipps for porting your addons from PloneTestCase to plone.app.testing.

    First: Look at some examples! Most of the Plone core packages are already ported to plone.app.testing. If you have a lot of packages with one namespace and a similar setup it probably makes sense to start with a ZopeSkel or mr.bob template for your testing base class.

    Second: Use a Testing base class! Define one or two base classes for all your testing needs in one product. This makes migrations a lot easier. With the help of the PloneTestCase class of plone.app.testing.bbb half the work is done. All you need is a layer which installs your addon and does the other things (create content, etc.) you need for testing.

    Third: Doctests Porting doctests is a little bit more tricky. The way doctests are run changed a little bit with plone.app.testing. You no longer pass a testing class to the test but add a layer. This can be easily done with the layered helper function found in plone.testing. You just pass in the layer you defined for your unittests and you can access it in your doctests. Because there is no test class there is no self.<whatever-method> supported in doctests. Greping 'self' in all doctests and replacing it with layer specific code is usually the way to go.

    >>> self.setRoles(['Manager])
    

    would turn into

    >>> from plone.app.testing import TEST_USER_ID, setRoles
    >>> setRoles(layer['portal'], TEST_USER_ID, ['Manager'])
    

    And you don't need to use any Zope based variant of doctest. Just use plain python doctest and suite.

    Forth: Functionaltests Basically all tests inherited from plone.app.testing.bbb.PloneTestCase are functional tests and support the publish method to publish an object in a testing environment. Sometimes I found kind of unpredictable behaviour using this method. Usually it can be avoided using zope.testbrowser (see next point). You need to use it if you are testing alternate publishing methods (like WebDAV or VirtualHostMonster) which rarly be the case.

    One thing I could track down is a cookie reset if diazo is turned on. This is because of a subrequest which is issued during traversal. You can disable diazo during testing:

    >>> response = self.publish(docpath, basic_auth, env={'diazo.off': "1"})
    

    To debug functional testing you need the following patch in your (failing) test.

    def raising(self, info):
        import traceback
        traceback.print_tb(info[2])
        print info[1]
    from Products.SiteErrorLog.SiteErrorLog import SiteErrorLog
    SiteErrorLog.raising = raising
    

    Fifth: Testbrowser Using zope.testbrowser is supported with plone.app.testing too. There are two main differences: a browser instance is initiated with the application: like this:

    >>> browser = Browser(self.layer['app'])   # in functional test cases
    

    or

    >>> browser = Browser(layer['app'])    # in doctests
    

    You need to commit changes before! you initiate the browser.

    >>> from transaction import commit
    >>> commit()
    

    Sixth: plone.protect

    If you are using a view, which uses CSRF protection via plone.protect you may want to disable this feature in tests temporarily. You can call your view by injecting a CSRF token into the request like this:

    >>> from plone.protect import createToken
    >>> request.form['_authenticator'] = createToken()
    

    The original idea I found in this blog.

    Seventh: Functional doctests In functional doctest sometimes a http function is found. This is the doctest analog of the functional test publish method. Currently it fails with plone.app.testing. I am investigating this and keep you posted, if I found something ...

    And now happy porting to plone.app.testing of your addons. BTW the porting of some products is left for core Plone. If you want to give it a try ... go ahead. :)

    See you on the Plone Conference in Bristol, Tom

    Six Feet Up: 1.9.0 release brings new features and documentation to collective.recipe.plonesite

    by Clayton Parker at 2014-10-15T12:00:00Z

    plone.pngThe 1.9.0 release of collective.recipe.plonesite is now available on PyPi. This release brings several new features and some improvements to the documentation.

    The biggest new feature is the integration with collective.upgrade. This allows for the plonesite recipe to automatically upgrade the Plone portal and run any upgrade steps for installed packages in the site. Credit goes to Ross Patterson for adding this feature.

    A couple more features were also added to the latest release. Fabio Rauber contributed a feature that provides the ability to have the plonesite recipe automatically add mount points. See the documentation for an example of how this works. The last new feature is the ability to run the plonesite part using sudo. This allows the plonesite part to be run when the Zope instance and ZEO server are run under different UIDs. Toni Mueller provided this new option.

    Lastly, the documentation was cleaned up in the source and for the end user. The documentation on PyPi is now grouped by option type so that it is easier to find what you are looking for. The docs were also cleaned up in source control so that it is easier to contribute.

    October 14, 2014

    Four Digits: Announcing the Anniversary Sprint 2015

    by Rob Gietema at 2014-10-14T18:17:33Z

    Four Digits will be 10 years old on the 23rd of June 2015, so what better way to celebrate then a Plone Sprint! So after the succes of the Living Statues Sprint 2010, 2011, the Plone Conf 2012 and Arnhem Sprint 2013 Four Digits will be hosting another sprint in 2015! The sprint will be from Monday to Friday and will finish with a big party on Friday evening.

    anniversary-sprint-logo.png

    When and Where

    22 - 26 June 2015, Jansbinnensingel 26, 6811AL Arnhem, The Netherlands

    Topics

    Topics will be announced.

    Organizer

    Four Digits (+31 26 4422700)

    Travel info

    Travelling to Arnhem after a plainride can be done easiest by train. The office is located at walking distance of the Arnhem train station.

    From Schiphol there is a direct train to Arnhem which takes about an hour.

    From Weeze Airport there is a bus(platform 5) from Nijmegen take a train to Arnhem (about 8 an hour).
    Alternative From Weeze, take the bus direct to Arnhem Gelredome(EUR 18,-). From there take line 5 or 7 (or any other bus) to the city and get out at "Jansbinnensingel". From here you can see our office

    From Eindhoven, take bus 401 to Eindhoven CS (Railway station). From there take one of the following trains:
    - Train to Hertogenbosch('s), from there a train to Arnhem
    - Train to Nijmegen, from there a train to Arnhem

    If you have any questions or want any please don't hesitate to contact us!

    Location

    The sprint will be held at the Four Digits Office, located in the city centre of Arnhem. We have room for more than enough sprinters. There are a lot of hotels/B&B's close to the office. If you need any help finding a suitable hotel let us know and we'll gladly assist you.

    Internet

    Wireless network on a fiber connection.

    Sign up

    If you would like to join us, please sign up for this project, if you want to stay a part of the sprint, please let us know. If you have any questions, don't hesitate to contact us.

    Food

    Everyone can join us for live cooking at the office and breakfast and lunch are included as well for all sprinters.

    Hotels

    "Hotel Arnhem Centraal", "Hotel Haarhuis", "Hotel Molendal", "Hotel Old Dutch", "Hotel Blanc", "Bordeaux B&B" and "Hotel Rembrandt" are all within walking distance of our office (Jansbinnensingel 26, Arnhem). If you can't find a suitable hotel let us know and we'll help you find one

    More information on the CoActivate page

    Plone.org: Announcing the Anniversary Sprint 2015 in Arnhem, June 22-26

    2014-10-14T15:59:34Z

    Four Digits will be 10 years old on the 23rd of June 2015, so what better way to celebrate then a Plone Sprint! So after the succes of the Living Statues Sprint 2010, 2011, the Plone Conf 2012 and Arnhem Sprint 2013 Four Digits will be hosting another sprint in 2015! The sprint will be from Monday to Friday and will finish with a big party on Friday evening.

    Six Feet Up: Learning Plone as a SysAdmin

    by Rob McBroom at 2014-10-14T12:00:00Z

    Plone Code BadgeI recently started learning to develop for the Python-based CMS, Plone. This is my perspective on learning to be a Plone developer with years of experience as a SysAdmin and Python developer.

    More Powerful Than I Thought

    Getting to know Plone, I can really see its appeal for someone who needs a CMS. It can handle anything you want it to. I've been really impressed with the granularity of:

    • Access control
    • Where portlets can be located
    • What types of content can exist in a container
    • How you can override styles and templates at any layer

    It's impressive to me that you can even create your own content types. Plone is packed with features for large organizations and let's you customize it to your requirements.

    There's no Free Lunch

    The flip side of the ability to do whatever you want is that it has a steep learning curve. Figuring out where parts of Plone start and end can be difficult:

    • Where does buildout end and Plone begin?
    • What is a part of GenericSetup vs. Plone?
    • When do you need to use Templer?

    This is further complicated by the fact that almost everything has an "old way" and a "new way".

    Make sure you understand vanilla Plone before you dive into anything custom on an existing site. There are also many versions of Plone in the wild that are still supported from 2 to 4.3, and 5 is just around the corner. Make sure to learn on the version your organization is already using or plans to use.

    How I would learn Plone

    If you need to learn how to develop for Plone, I recommend to start you:

    Learn Buildout First

    With Plone you don't install packages like you traditionally would with `pip install`. You list all the packages in a file that Buildout uses to install everything at once and track their location. You should also understand the difference between a Unified Installer buildout or a manual buildout.

    ZODB and Plone

    Get to know the commands and sequence required to start the ZODB and Plone and shut them down. Then you can get familiar with the command scripts that helps take care of this for you.

    Learn How to Get Into the Debugger

    Next I would get familiar with the Plone debugger which will let you inspect and interact with live objects.
    http://docs.plone.org/develop/plone/getstarted/debug_mode.html


    Explore the Admin Interface

    Once you are comfortable with getting Plone installed, running, and debugged, you can start getting familiar with the user's admin interface inside Plone. Take a look around and get familiar with creating content like blog posts as well as changing system settings.

    Understand the Zope Management Interface (ZMI)

    Once you've played with settings and content in Plone, you can hop over into the ZMI and see how the changes you made in the Plone interface modifies and creates data in Zope.
    http://docs.zope.org/zope2/zope2book/UsingZope.html

    At this point you should be familiar enough with Plone to start looking at how to build the site you want. Then things get real interesting depending on where you go with this powerful CMS.

    Did you enjoy this post? I'll continue to blog on my perspective of Plone and how to modify and manage it.

    October 11, 2014

    UW Oshkosh How-To's: How to hide the title and/or description on a Plone 4 page

    by nguyen at 2014-10-11T04:53:02Z

    Based on information at http://plone.org/documentation/kb/how-to-write-templates-for-plone-4

    Create the view template

    If you want to hide the title and description on a Plone page, you can create a new template that does that.

    • Go to the ZMI -> portal_skins
    • Use the Find tab to look for an object with the ID "document_view". Click on it, then click on the Customize button.  
    • In the breadcrumbs, click on the "custom" link to take you up one level.
    • Check the box next to the document_view object, then click the Rename button at the bottom of the page.  Give it a new name (e.g. document_no_title_or_description_view) then click the OK button.
    • Click on the document_no_title_or_description_view object again to edit it, and replace its contents with the code below.

     

    <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"
          xmlns:tal="http://xml.zope.org/namespaces/tal"
          xmlns:metal="http://xml.zope.org/namespaces/metal"
          xmlns:i18n="http://xml.zope.org/namespaces/i18n"
          lang="en"
          metal:use-macro="context/main_template/macros/master"
          i18n:domain="plone">
    <body>
    <metal:content-title fill-slot="content-title">
        <metal:content-title define-macro="content-title">
         </metal:content-title>
    </metal:content-title>
    <metal:content-description fill-slot="content-description">
        <metal:content-description define-macro="content-description">
         </metal:content-description>
    </metal:content-description>
    <metal:content-core fill-slot="content-core">
        <metal:content-core define-macro="content-core">
            <metal:field use-macro="python:context.widget('text', mode='view')">
                Body text
            </metal:field>
        </metal:content-core>
    </metal:content-core>
    </body>
    </html>

    The difference between the above and the default view template is that both the content-title and the content-description macros no longer do anything.

    Next, you have to enable this new customized view template for pages ("Documents").

    Add the new view template to the page/Document type

    • Go to ZMI -> portal_types -> Document
    • Add one more line to the "Available View Methods" field, containing document_no_title_or_description_view, and press the Save Changes button.

     

    Using the new view template

    • Go to the page on which you wanted to hide the title and description.  
    • The Display drop down menu should now include a new choice "document_no_title_or_description_view".  Select it, and the page should now no longer show its title nor its description.

     

    October 10, 2014

    Espen Moe-Nilssen: A different approach to mobile theming.

    by admin at 2014-10-10T13:55:00Z

    Goals

    • One theme for mobile, another for others....
    • Keep old theme (don't mess it up)
    • Only make new theme for mobile
    • Do not send "full scale images" to mobile

     

    Installation

    • zettwerk.mobiletheming
    • medialog.simpleslider
    • Some mobile theme ( try zettwerk.mobiletheme, or customize medialog.mobilethemeThree)

     

    Setup

    • Go to http://yoursite/@@mobile-theming-controlpanel
    • Change hostname to the mobile hostname (for example http://m.mydomain.com
    • Select mobile theme
    • Set up your DNS with an entry for m.mydomain.com and probably a new rewrite rule (on apache or whatever you use)

     

     

    How does i work?

    • When someone visits your site, they will see your "nomal theme"
    • This could be a diazo theme, or an "old-style theme"
    • If the site is visited from a mobile device, they will be redirected to the mobile URL
    • The mobile site will show the mobile theme.
    • This is done with a javascript

     

     

    The same site visited via two different domains

     

     

    The slider part

    (this part is still a bit experimental)

     

    • Go to the page where you want your slider (fader)
    • Enable the simple slider
    • Edit the slider settings

     

     

     

     

     

     

     

    • Notice: you can write the path to the images or select a tag (keyword) to search for the images (but it will only sort those selected by tag)
    • You should set image size (scale) for "normal" and mobile (see screenshot)

     

     

    Troubleshooting

    • You might have to disable/enable the themes (or restart)
    • The slider is a viewlet, so you can enable/disable it with /@@manage-viewlets
    • The "fader" is responsive, so its size might be the same if you change the scale (but it will be less kB)

    October 09, 2014

    UW Oshkosh How-To's: How to export and import Plone users

    by nguyen at 2014-10-09T16:10:32Z

    I found this very useful information which shows how to copy the usernames and passwords from one Plone site to another, by exporting the credentials into a file then reading the file into another site:

    http://play.pixelblaster.ro/blog/archive/2011/03/09/export-import-users-in-and-out-of-plone

    import cPickle
    def export(self):
        pas = self.acl_users
        users = pas.source_users
        passwords = users._user_passwords
        result = dict(passwords)
        f = open('/tmp/out.blob', 'w')
        cPickle.dump(result, f)
        f.close()
        return "done"
    def import_users(self):
        pas = self.acl_users
        users = pas.source_users
        f = open('/tmp/out.blob')
        res = cPickle.load(f)
        f.close()
        for uid, pwd in res.items():
            users.addUser(uid, uid, pwd)
        return "done"

    However, I didn't want to use a file since I have a small number of users to copy. Instead, I used the ZEO client in "debug" mode to copy the username & password dictionary.

     

    Getting the credentials from the source site

    On the source site I did started the instance in debug mode (you could do this with a ZEO client too if you have a ZEO setup by using bin/client1 for example):

    $ bin/instance debug

    >>> mysite=app.MYSITEID

    >>> pas=mysite.acl_users

    >>> users=pas.source_users

    >>> passwords=users._user_passwords

    >>> dict(passwords)

    {'user1': '{SSHA}hashedpw1', 'user2': '{SSHA}hashpw2'}

    I copied the last line (the passwords dictionary contents) to my terminal clipboard.

    Applying the credentials on the destination site

    Then on the destination site I launched the instance in debug mode and navigated back down to the same spot.  

    Then I defined a variable called "passwords" and set it to the text I'd copied into my clipboard above. That's the line that starts with "passwords =".

    Then, using that new "passwords" variable, I made a for loop and called the users.addUser() method with each set of credentials in the passwords dictionary.

    Finally I committed the ZODB transaction to make the new users persist in the database.

    $ bin/instance debug

    >>> newsite=app.NEWSITEID

    >>> pas=newsite.acl_users

    >>> users=pas.source_users

    >>> passwords = dict({'user1': '{SSHA}hashedpw1', 'user2': '{SSHA}hashpw2'})

    >>> for u in passwords:

    ...   users.addUser(u, u, passwords[u])

    ... 

    >>> import transaction

    >>> transaction.commit()

    >>>

     

    See Also

    This documentation explains "Exporting and importing member passwords":

    http://docs.plone.org/develop/plone/members/member_basics.html?highlight=password#exporting-and-importing-member-passwords