24. Upgrade steps – Mastering Plone 6 development

Upgrade steps

24. Upgrade steps#

In this part you will:

  • Write code to update, create and move content

  • Create custom catalog indexes

  • Create criteria for search and listing blocks

  • Enable features with upgrade steps

Tools and techniques covered:

  • upgrade steps

Backend chapter

Checkout ploneconf.site at tag "schema":

git checkout schema

The code at the end of the chapter:

git checkout upgrade_steps

More info in The code for the training

24.1. Upgrade steps#

You recently changed existing content, when you added the behavior ploneconf.featured or when you turned talks into events in the chapter Turning Talks into Events.

When projects evolve you sometimes want to modify various things while the site is already up and brimming with content and users. Upgrade steps are pieces of code that run when upgrading from one version of an add-on to a newer one. They can do just about anything. We will use an upgrade step to enable the new behavior instead of reinstalling the add-on.

We will create an upgrade step that:

  • runs the typeinfo step, i.e. loads the GenericSetup configuration stored in profiles/default/types.xml and profiles/default/types/... so we don't have to reinstall the add-on to have our changes from above take effect and

  • cleans up existing talks that might be scattered around the site in the early stages of creating it. We will move all talks to a (folderish) page talks (unless they already are there).

Upgrade steps can be registered in their own ZCML file to prevent cluttering the main configure.zcml.

Update the upgrades/configure.zcml:

  <genericsetup:upgradeSteps
      profile="ploneconf.site:default"
      source="1000"
      destination="1001"
      >
    <genericsetup:upgradeStep
        title="Update types"
        description="Enable new behaviors et cetera"
        handler="ploneconf.site.upgrades.v1001.update_types"
        />
    <genericsetup:upgradeStep
        title="Clean up site structure"
        description="Move talks to to their page"
        handler="ploneconf.site.upgrades.v1001.cleanup_site_structure"
        />
  </genericsetup:upgradeSteps>

The upgrade steps bumps the version number of the GenericSetup profile of ploneconf.site from 1000 to 1001. The version is stored in profiles/default/metadata.xml.

Change it to

<version>1001</version>

GenericSetup now expects the code as a method cleanup_site_structure() and update_types() in the file upgrades/v1001.py. Let's create it.

upgrades/v1001.py

 1from plone import api
 2from plone.app.upgrade.utils import loadMigrationProfile
 3
 4import logging
 5
 6
 7default_profile = "profile-ploneconf.site:default"
 8logger = logging.getLogger(__name__)
 9
10
11def reload_gs_profile(setup_tool):
12    """Load default profile"""
13    loadMigrationProfile(
14        setup_tool,
15        default_profile,
16    )
17
18
19def update_types(setup_tool):
20    setup_tool.runImportStepFromProfile(default_profile, "typeinfo")
21
22
23def cleanup_site_structure(setup_tool):
24    # Load default profile including new type info
25    # This makes 'update_types' superfluous.
26    reload_gs_profile(setup_tool)
27
28    portal = api.portal.get()
29
30    # Create the expected site structure
31    if "training" not in portal:
32        api.content.create(
33            container=portal, type="Document", id="training", title="Training"
34        )
35
36    if "schedule" not in portal:
37        schedule_folder = api.content.create(
38            container=portal, type="Document", id="schedule", title="Schedule"
39        )
40    else:
41        schedule_folder = portal["schedule"]
42    schedule_folder_url = schedule_folder.absolute_url()
43
44    if "location" not in portal:
45        api.content.create(
46            container=portal, type="Document", id="location", title="Location"
47        )
48
49    if "sponsors" not in portal:
50        api.content.create(
51            container=portal, type="Document", id="sponsors", title="Sponsors"
52        )
53
54    if "sprint" not in portal:
55        api.content.create(
56            container=portal, type="Document", id="sprint", title="Sprint"
57        )
58
59    # Find all talks
60    brains = api.content.find(portal_type="talk")
61    for brain in brains:
62        if schedule_folder_url in brain.getURL():
63            # Skip if the talk is already somewhere inside the target folder
64            continue
65        obj = brain.getObject()
66        # Move talk to the folder '/schedule'
67        api.content.move(source=obj, target=schedule_folder, safe_id=True)
68        logger.info(f"{obj.absolute_url()} moved to {schedule_folder_url}")

We create the required site structure if it does not exist yet making extensive use of plone.api as discussed in the chapter Programming Plone.

Have a look at ZMI import steps http://localhost:8080/Plone/portal_setup/manage_importSteps to find the upgrade step id for the type upgrade.

Import steps

Import step ids for runImportStepFromProfile#

After restarting the site we can run the upgrade step:

  • Go to the Add-ons control panel http://localhost:3000/controlpanel/addons. The add-on ploneconf.site should now be marked with an Upgrade label and have a button to upgrade from 1000 to 1001.

  • Run the upgrade step by clicking on it.

On the console you should see logging messages like:

2024-09-15 11:26:14,114 INFO    [ploneconf.site.upgrades.v1001:83][waitress-0] http://localhost:3000/talks/test-talk moved to http://localhost:3000/schedule

Alternatively you can also select which upgrade steps to run like this:

  • In the ZMI go to portal_setup

  • Go to the tab Upgrades

  • Select ploneconf.site from the dropdown and click Choose profile

  • Run the upgrade step.

See also

https://5.docs.plone.org/develop/addons/components/genericsetup.html#upgrade-steps

24.2. Add catalog index#

For the next chapter we need to search for sponsors and get the results with the values of field 'url' and 'level. We add a metadata column for these fields to not wake up objects on search request. This would be OK, but time consuming. The search request gets an attribute from catalog brains unless the attribute is not available, then fetches the real object.

Add the new meta data columns 'level' and 'url' to profiles/default/catalog.xml

  <column value="level" />
  <column value="url" />

While we are at it, we also add some more indexes and criteria for fields of type talk. With these indexes and criteria we can create listing and search blocks with facets.

<?xml version="1.0" encoding="utf-8"?>
<object name="portal_catalog">
  <index meta_type="BooleanIndex"
         name="featured"
  >
    <indexed_attr value="featured" />
  </index>
  <column value="featured" />

  <index meta_type="KeywordIndex"
         name="type_of_talk"
  >
    <indexed_attr value="type_of_talk" />
  </index>
  <column value="type_of_talk" />


  <index meta_type="FieldIndex"
         name="speaker"
  >
    <indexed_attr value="speaker" />
  </index>
  <index meta_type="KeywordIndex"
         name="audience"
  >
    <indexed_attr value="audience" />
  </index>
  <index meta_type="FieldIndex"
         name="room"
  >
    <indexed_attr value="room" />
  </index>

  <column value="speaker" />
  <column value="audience" />
  <column value="room" />


  <column value="level" />
  <column value="url" />
</object>

This adds new indexes for the three fields we want to show in the listing. Note that audience is a KeywordIndex because the field is multi-valued, but we want a separate index entry for every value in an object.

A reinstallation of the add-on would leave the new catalog indexes empty. Therefore we write an upgrade step to not only add indexes and criteria, but also reindex all talks:

src/ploneconf/site/upgrades/v1001.py:

def update_indexes(setup_tool):
    # Indexes and metadata
    setup_tool.runImportStepFromProfile(default_profile, "catalog")
    # Criteria
    setup_tool.runImportStepFromProfile(default_profile, "plone.app.registry")
    # Reindexing content
    for brain in api.content.find(portal_type=["talk", "sponsor"]):
        obj = brain.getObject()
        obj.reindexObject()
        logger.info(f"{obj.id} reindexed.")

src/ploneconf/site/upgrades/configure.zcml:

<configure
    xmlns="http://namespaces.zope.org/zope"
    xmlns:genericsetup="http://namespaces.zope.org/genericsetup"
    >

  <genericsetup:upgradeSteps
      profile="ploneconf.site:default"
      source="1000"
      destination="1001"
      >
    <genericsetup:upgradeStep
        title="Update types"
        description="Enable new behaviors et cetera"
        handler="ploneconf.site.upgrades.v1001.update_types"
        />
    <genericsetup:upgradeStep
        title="Clean up site structure"
        description="Move talks to to their page"
        handler="ploneconf.site.upgrades.v1001.cleanup_site_structure"
        />
    <genericsetup:upgradeStep
        title="Update catalog"
        description="Add and populate new indexes. Add criteria."
        handler="ploneconf.site.upgrades.v1001.update_indexes"
        />
  </genericsetup:upgradeSteps>

</configure>

From time to time you may want to update the catalog manually. To do so, go to http://localhost:8080/Plone/portal_catalog/manage_catalogIndexes, select the new indexes and click Reindex. You can also rebuild the whole catalog by going to the Advanced tab and clicking Clear and Rebuild.

24.3. Add collection criteria#

The following additional criteria allow us to create a search block constrained to talks with facets to filter for audience, speaker and room.

profiles/default/registry/querystring.xml

  <records interface="plone.app.querystring.interfaces.IQueryField"
           prefix="plone.app.querystring.field.speaker"
  >
    <value key="title">Speaker</value>
    <value key="enabled">True</value>
    <value key="sortable">True</value>
    <value key="operations">
      <element>plone.app.querystring.operation.string.is</element>
      <element>plone.app.querystring.operation.string.contains</element>
    </value>
    <value key="group"
           i18n:translate=""
    >Metadata</value>
  </records>

  <records interface="plone.app.querystring.interfaces.IQueryField"
           prefix="plone.app.querystring.field.audience"
  >
    <value key="title">Audience</value>
    <value key="enabled">True</value>
    <value key="sortable">False</value>
    <value key="operations">
      <element>plone.app.querystring.operation.selection.any</element>
      <element>plone.app.querystring.operation.selection.all</element>
      <element>plone.app.querystring.operation.selection.none</element>
    </value>
    <value key="group"
           i18n:translate=""
    >Metadata</value>
    <value key="vocabulary">ploneconf.audiences</value>
  </records>

  <records interface="plone.app.querystring.interfaces.IQueryField"
           prefix="plone.app.querystring.field.room"
  >
    <value key="title">Room</value>
    <value key="enabled">True</value>
    <value key="sortable">False</value>
    <value key="operations">
      <element>plone.app.querystring.operation.selection.any</element>
      <element>plone.app.querystring.operation.selection.all</element>
      <element>plone.app.querystring.operation.selection.none</element>
    </value>
    <value key="group"
           i18n:translate=""
    >Metadata</value>
    <value key="vocabulary">ploneconf.rooms</value>
  </records>

See also

For a full list of all existing QueryField declarations see plone/plone.app.querystring.

For a full list of all existing operations see plone/plone.app.querystring.

24.4. Apply the new criterion to create a search block for talks#

As soon as you run the upgrade steps, you can now add a search block to your 'schedule' page that provides facets to filter for audience, et cetera.

search block

search block#

24.5. Summary#

  • You wrote your first upgrade step

    • to enable changes on types

    • update content

    • prepare your site for search and listing blocks for your custom types

Contents