18. Programming Plone#

Backend chapter

In this part you will:

Learn about the recommended ways to do something in backend code in Plone.
Learn to debug

Topics covered:

plone.api
Portal tools
Debugging

18.1. plone.api#

The most important tool nowadays for plone developers is the add-on plone.api that covers 20% of the tasks any Plone developer does 80% of the time. If you are not sure how to handle a certain task, be sure to first check if plone.api has a solution for you.

The API is divided in five sections. Here is one example from each:

Content: Create content
Portal: Send E-Mail
Groups: Grant roles to group
Users: Get user roles
Environment: Switch roles inside a block

plone.api is a great tool for integrators and developers that is included when you install Plone, though for technical reasons it is not used by the code of Plone itself.

In existing code you'll often encounter methods that don't mean anything to you. You'll have to use the source to find out what they do.

Some of these methods are replaced by plone.api:

Products.CMFCore.utils.getToolByName() -> api.portal.get_tool()
zope.component.getMultiAdapter() -> api.content.get_view()

18.2. portal tools#

Some parts of Plone are very complex modules in themselves (e.g. the versioning machinery of Products.CMFEditions). Most of them have an API of themselves that you will have to look up when you need to implement a feature that is not covered by plone.api.

Here are a few examples:

portal_catalog: unrestrictedSearchResults() returns search results without checking if the current user has the permission to access the objects.; uniqueValuesFor() returns all entries in an index
portal_setup: runAllExportSteps() generates a tarball containing artifacts from all export steps.
Products.CMFPlone.utils: is_product_installed() checks if a product is installed.

Usually the best way to learn about the API of a tool is to look in the interfaces.py in the respective package and read the docstrings. But sometimes the only way to figure out which features a tool offers is to read its code.

To use a tool, you usually first get the tool with plone.api and then invoke the method.

Here is an example where we get the tool portal_membership and use one of its methods to logout a user:

mt = api.portal.get_tool('portal_membership')
mt.logoutUser(request)

Note

The code for logoutUser() is in Products.PlonePAS.tools.membership.MembershipTool.logoutUser(). Many tools that are used in Plone are actually subclasses of tools from the package Products.CMFCore. For example portal_membership is subclassing and extending the same tool from Products.CMFCore.MembershipTool.MembershipTool. That can make it hard to know which options a tool has. There is an ongoing effort by the Plone Community to consolidate tools to make it easier to work with them as a developer.

18.3. Debugging#

Here are some tools and techniques we often use when developing and debugging. We use some of them in various situations during the training.

tracebacks and the log

The log (and the console when running in foreground) collects all log messages Plone prints. When an exception occurs, Plone throws a traceback. Most of the time the traceback is everything you need to find out what is going wrong. Also adding your own information to the log is very simple.

import logging

logger = logging.getLogger(__name__)

def do_vote(user, vote):
    logger.info(f"User {user} voted with {vote}")

pdb

The Python debugger pdb is the single most important tool for us when programming. Just add import pdb; pdb.set_trace() in your code and debug away! The code execution stops at the line you added import pdb; pdb.set_trace(). Switch to your terminal and step through your code.

pdbpp

A great drop-in replacement for pdb with tab completion, syntax highlighting, better tracebacks, introspection and more. And the best feature ever: The command ll prints the whole current method.

ipdb

Another enhanced pdb with the power of IPython, e.g. tab completion, syntax highlighting, better tracebacks and introspection. It also works nicely with Products.PDBDebugMode. Needs to be invoked with import ipdb; ipdb.set_trace().

Products.PDBDebugMode

An add-on that has two killer features.

Post-mortem debugging: throws you in a pdb whenever an exception occurs. This way you can find out what is going wrong.

pdb view: simply adding /pdb to a url drops you in a pdb session with the current context as self.context. From there you can do just about anything.

Interactive debugger

Start your instance in debug mode with

venv/bin/zconsole debug instance/etc/zope.conf

You have an interactive debugger at your fingertips. app.Plone is your Plone instance object which you can inspect on the command line.

To list the ids of the objects in a folderish object:

>>> app.Plone.talks.keys()
['whats-new-in-python-3.10', 'plone-7', 'zope', 'betty-white', 'new-years-day', 'journey-band']

To list the items of a folderish object:

>>> from zope.component.hooks import setSite
>>> setSite(app.Plone)
>>> app.Plone.talks.contentItems()
[('whats-new-in-python-3.10', <Talk at /Plone/talks/whats-new-in-python-3.10>), ('plone-7', <Talk at /Plone/talks/plone-7>), ('zope', <Talk at /Plone/talks/zope>), ('betty-white', <Talk at /Plone/talks/betty-white>), ('new-years-day', <Talk at /Plone/talks/new-years-day>), ('journey-band', <Talk at /Plone/talks/journey-band>)]

Setting the site with setSite is necessary because the root object when starting the interactive debugger is app, which can contain more than just one Plone instance object. We choose our Plone instance with id Plone. With setting the site to operate on, we have access to the Zope component registry. The component registry is needed for methods like contentItems which look up utilities and other components like in this case the ITypesTool.

Stop the interactive shell with ctrl d.

plone.app.debugtoolbar

An add-on that allows you to inspect nearly everything. It even has an interactive console, a tester for TALES-expressions and includs a reload-feature like plone.reload.

plone.reload

An add-on that allows to reload code that you changed without restarting the site. Open http://localhost:8080/@@reload in your browser for plone.reloads UI. It is also used by plone.app.debugtoolbar.

Products.PrintingMailHost

An add-on that prevents Plone from sending mails. Instead, they are logged.

verbose-security = on

An option for the recipe plone.recipe.zope2instance that logs the detailed reasons why a user might not be authorized to see something.

Sentry

Sentry is an error logging application you can host yourself. It aggregates tracebacks from many sources and (here comes the killer feature) even the values of variables in the traceback. We use it in all our production sites.

18.4. Exercise 1#

Knowing that venv/bin/zconsole debug instance/etc/zope.conf basically offers you a Python prompt to inspect your Plone instance, how would you start to explore Plone?

Solution

Use locals() or locals().keys() to see Python objects available in Plone

You will get notified that app is automatically bound to your Zope application, so you can use dictionary-access or attribute-access as explained in What is Plone? to inspect the application:

18.5. Exercise 2#

The app object you encountered in the previous exercise can be seen as the root of Plone. Once again using Python, can you find your newly created Plone site?

Solution

app.keys() will show app's attribute names - there is one called Plone, this is your Plone site object. Use app.Plone to access and further explore it.

>>> app
<Application at >
>>> app.keys()
['browser_id_manager', 'session_data_manager', 'error_log', 'temp_folder', 'virtual_hosting', 'index_html', 'Plone', 'acl_users']
>>> portal = app["Plone"]
>>> from zope.component.hooks import setSite
>>> setSite(portal)
>>> portal
<PloneSite at /Plone>
>>> portal.keys()
['portal_setup', 'MailHost', 'caching_policy_manager', 'content_type_registry', 'error_log', 'plone_utils', 'portal_actions', 'portal_catalog', 'portal_controlpanel', 'portal_diff', 'portal_groupdata', 'portal_groups', 'portal_memberdata', 'portal_membership', 'portal_migration', 'portal_password_reset', 'portal_properties', 'portal_registration', 'portal_skins', 'portal_types', 'portal_uidannotation', 'portal_uidgenerator', 'portal_uidhandler', 'portal_url', 'portal_view_customizations', 'portal_workflow', 'translation_service', 'mimetypes_registry', 'portal_transforms', 'portal_archivist', 'portal_historiesstorage', 'portal_historyidhandler', 'portal_modifier', 'portal_purgepolicy', 'portal_referencefactories', 'portal_repository', 'acl_users', 'portal_resources', 'portal_registry', 'HTTPCache', 'RAMCache', 'ResourceRegistryCache', 'talks', 'sponsors']
>>> app['Plone']['talks']
<Folder at /Plone/talks>
>>> app.Plone.talks
<FolderishDocument at /Plone/talks>

Note

Plone and its objects are stored in an object database, the ZODB. You can use venv/bin/zconsole debug instance/etc/zope.conf as a database client (in the same way e.g. psql is a client for PostgreSQL). Instead of a special query language (like SQL) you simply use Python to access and manipulate ZODB objects. Don't worry if you accidentally change objects - you would have to commit your changes explicitly to make them permanent.

The Python code to do so is:

>>> import transaction
>>> transaction.commit()

You have been warned.

18.6. Exercise 3#

Create a new BrowserView callable as /@@demo_content in a new file demo.py.
The view should create five talks each time it is called.
Use the documentation at Content to find out how to create new talks.
Use plone.api.content.transition to publish all new talks. Find the documentation for this method.
Only managers should be able to use the view (the permission is called cmf.ManagePortal).
Display a message about the results (see Show notification message).
For extra credits use the library requests and Wikipedia to populate the talks with content.
Use the utility methods cropText from Producs.CMFPlone to crop the title after 20 characters. Use the documentation at Global utilities and helpers to find an overview of plone_view helpers.

Note

Do not try everything at once, work in small iterations, restart your Plone instance to check your results frequently.
Use pdb during development to experiment.

Solution

Add this to browser/configure.zcml:

<browser:page
  name="create_demo_talks"
  for="*"
  class=".demo.DemoContent"
  permission="cmf.ManagePortal"
  />

This is browser/demo.py:

from Products.Five import BrowserView
from plone import api

from plone.protect.interfaces import IDisableCSRFProtection
from zope.interface import alsoProvides

import json
import logging
import requests

logger = logging.getLogger(__name__)


class DemoContent(BrowserView):
    def __call__(self):
        portal = api.portal.get()
        self.create_talks()
        return self.request.response.redirect(portal.absolute_url())

    def create_talks(self, amount=3):
        """Create some talks"""

        alsoProvides(self.request, IDisableCSRFProtection)
        plone_view = api.content.get_view("plone", self.context, self.request)
        wiki_content = self.get_wikipedia_content_of_the_day()
        for data in wiki_content[:amount]:
            talk = api.content.create(
                container=self.context,
                type="talk",
                title=plone_view.cropText(data["titles"]["normalized"], length=20),
                description=data["description"],
                type_of_talk="talk",
            )
            api.content.transition(talk, to_state="published")
            logger.info(f"Created talk {talk.absolute_url()}")
        api.portal.show_message(f"Created {amount} talks!", self.request)

    def get_wikipedia_content_of_the_day(self):
        wiki = requests.get(
            "https://en.wikipedia.org/api/rest_v1/feed/featured/2022/01/02"
        )
        return json.loads(wiki.text)["mostread"]["articles"]

Some notes:

Since calling view is a GET and not a POST we need alsoProvides(self.request, IDisableCSRFProtection)() to allow write-on-read without Plone complaining. Alternatively we could create a simple form and create the content on submit.
Transition has two modes of operation: The documented one is api.content.transition(obj=foo, transition='bar'). That mode tries to execute that specific transition. But sometimes it is better to use to_state which tries to to find a way to get from the current state to the target state. Follow the link to the method description in documentation to find more information on transition.
To use methods like cropText from another browser view, you can get the view for your context with api.content.get_view("view_name", self.context, self.request)

Here the description of the talk is set. To add text, you need to create an instance of RichTextValue and set it as an attribute:

from plone.app.textfield.value import RichTextValue
talk.details = RichTextValue(data["extract"], 'text/plain', 'text/html',)

Programming Plone

Contents

18. Programming Plone#

18.1. plone.api#

18.2. portal tools#

18.3. Debugging#

18.4. Exercise 1#

18.5. Exercise 2#

18.6. Exercise 3#