Overview over all blog items

I spent this week three days at the Linuxtag in Berlin and presented Zope and Plone to visitors at the DZUG booth. Many people asked about the Plone dependencies and how to install it on their systems. Some argued that the PHP-based CMSes are much easier to deal with since the hosters usually provide PHP and MySQL as part of a webspace package. On the other hand it is not so easy possible to install those PHP-based CMS on a local server or even on your notebook for just trying it out.  As far as I could figure out,: neither Joomla nor Typo3 nor Drupal provide installers for installing the CMS within more or less complex manual interaction on a local system. Most visitors were amazed that you can install Plone directly and hussle-free by using the unified installer.

1:0 for Plone


30.05.2008

@cryptic :${modifie#rs}

Python indentation bashing

Take a seat and read this if you are in the mood and if you can stand some Python bashing.


26.02.2008

New kids on the block: haufe.eggserver and haufe.releaser

Tools for building up a local (and private) egg repository

One of my customer is currently switching to zc.buildout and egg-based deployment in the mid-term. Since a lot of code is not intended for the public there was a need for a local egg repository. So far we have been using a filesystem containing the eggs that was made accessible through Apache. This worked fine with buildout and easy_install. However this solution lacked a nice web interface and the possibility for uploading eggs directly from setuptools into our own repository. This is  where haufe.eggserver and haufe.releaser comes in.

haufe.eggserver

haufe.eggserver is a Grok-based implementation and provides a browsable  frontend to the local  distribution directory. Currently it provides only a listing for the packages and their distribution files. Support for showing the metadata and release notes is planned

haufe.releaser

haufe.releaser provides a new local_upload command for setuptools. local_upload  uploads your  eggs  or source code archives directly into a running haufe.eggserver instance.

References:


15.02.2008

PyPI - the package toilet of the Python world

PyPI is becoming more and more the huge Python module toilet.


Why?

Lots of upload packages often don't contain a description or don't contain any URL to the project homepage or SVN repository.

So what is the value of uploading such packages to PyPI?

  • For the uploader: a convenient place for making poor releases.
  • For the PyPI user/visitor: none

What I do expect from a quality release on PyPI:

  • The metadata must contain a reasonable description
  • The metadata should include the README.txt or at least contain a direct link to the documentation. Nothing is more annoying than having to step to repositories in order to find the proper documentation. The documentation must be only one mouse-click away
  • The metadata should contain the release notes or at least a direct link to the release notes
  • The metadata must  contain direct links to the source code repository and to the project homepage
  • I am uncertain about if development packages (modules released with _some_ internal SVN revision number) should be placed on PyPI. I would  prefer having only official releases following the X.Y.Z version numbering schema on PyPI

On common request I uploaded and export/import script to our public repository. The code is not a working solution for you. Consider it as a starting point for writing your own migrations. The code is unsupported - use it at your own risk, pleasure and pain:


26.12.2007

When the Plone migration fails - doing content-migration only

The standard Plone migration often fails - especially for more or less customized sites and for sites running on some pre-historic or unreleased Plone version. Plone traditionally performs an in-place migration however you often want to create a new Plone site from scratch having the need to move your old content somehow to the new site.

Over the last few month I had the pleasure (or pain) migrating several educational Plone sites to current Plone 3 technology. There were some Plone 2.0 beta-something sites, some Plone 1-ish sites...more or less stock Plone instances with a small list on add-ons. Unfortunately the standard Plone migration never worked. Big problem? Perhaps..or perhaps not.

After running a site for three or four years you collect a lot of trash on your site and you often want to start from scratch when it comes to design and site-structure.  Over this period of time you see people come and go that is reflected within a Plone site as a huge garbage of stale content, messed up permissions and so on. So the general approach is to create a new Plone3 site from scratch and having some mechanism to get your core content over into the new sites with the most important set of metadata. This means you want to migrate the content only and basically don't care about much about any kind site configuration of your old system.

The idea is pretty simple and not too complicated to implement. You have to write a script that performs a catalog search per content type
and exports the content. For my own migrations I created a simple INI-style format containing the metadata for a particular content-type. A generated documents.ini looks like this:

[document-abt6/weigel/resources/plasmids/Alc/pam58]
path = abt6/resources/plasmids/Alc/pam58
group = abt6
filename = exports/documents/tmpchvyF2
text-format= structured-text
content-type = text/plain
id = pam58
title = pAM58_(AlcA_LFY_pAM54)
Description =
owner = admin
review-state = published
created = 1099242080.130000
effective = 1099242866.000000
expires = 253370674800.000000
The information is directly extract from the original objects by calling the related accessor methods. Since all content-types share the same metadata (dublin core) you can re-use the code for extracting the metadata for all content-types and have to write only specific code for each content-type (e.g. for extracting the start and end date for a news item).

Some remarks to the export format:
  • one .ini file per content-type
  • binary content like image data or the text body or file data is stored in dedicated file on the filesystem and referenced by the filename
  • dates are exported as timestamp
After creating a set of .ini files you can write a small import script that iterates of the list of file and parses them (trivial using the
ConfigParser module of Python). Recreating the content within the new sites is also pretty simple...you call invokeFactory() for creating a new content object and you call the related mutator methods for restoring the field values. If needed you have to call some other Plone API methods for restoring the ownership or for setting the workflow state properly...but this almost pretty easy if you have some experiences with Python, Plone and Archetypes.

This approach is pretty much generic and extensible and made me feel much better in case of failing Plone migration. I am pretty sure that the standard Plone migration machinery has some code for iterating over content or restoring it somehow but I found this simple .ini file format approach very robust and extensible.  The standard Plone migration is getting better from version to version but depending on the level of customizations and third-party products it will always fails at some point. Other alternatives for doing content-migrations with Plone 2.5 or higher are for example tools like GSXML that exports a whole folder structure as XML (which seems to work pretty well and appears pretty much stable). You might also look at XMLForest (which is supposed to work with Plone 2.1 or higher) but I always had some issues.

AND: Never use Export/Import through the ZMI
- NEVER - for doing a migration. Export/import is known to work (and designed) for moving stuff between identical installations only!

There is now a buildout.cfg zc.buildout configuration file available that simplifies the creation of an educational Plone site. The buildout contains support for

  • CMFBibliographyAT
  • FacultyStaffDirectory 2.0
  • Reflecto
  • TextIndexNG 3.2.1
  • FileSystemStorage (enabled for File type)
  • ZWiki
  • FCKEditor
  • PloneFormGen
  • LinguaPlone

Since ages it has been impossible to build Python from the sources with readline support out-of-the-box. The reason seems to be an outdated or broken version of libreadline that ships with MacOSX. To resolve this issue you have to install Darwinports and install readline from the sources. However configure won't pick up this installation automatically. Usually it is sufficient to specify the location of additional header files and libraries through the CFLAGS and LDFLAGS environment variables however that also did not work. I ended up hacking the generated Makefile. After modifiying the CC definition to

CC = gcc -I/opt/local/include -L/opt/local/lib

I was able to compile and install Python 2.4 and 2.5 with working readline support. It's a hack - but a hack that actually works.


Update for Leopard:
  • after running configure you have to edit setup.py
  • search for "if do_readline:" 
  • add do_readline = True before the if-statement


The Plone 3.0 TTW customization story is obviously better than I thought. Tom Lazar explains in this blog how to customize template-based views (very similar to portal_skins) using the new portal_view_customization tool...pretty cool.  This seems to be one of the best hidden secrets on Plone 3. So with this knowledge my former blog entry is just obsolete (but perhaps it was worth getting some clearification ). Thanks Tom!


18.10.2007

Plone 3 annoyances

Some minor ranting about Plone 3

First, Plone 3 is a great release, it is pretty much stable since the 3.0 release but:

  • customizing stuff like header, footer or so became too complicated. Performing trivial customizations is no longer easily possible - neither for a scripter nor for a developer. By moving stuff from portal_skins to browser views or viewlets we lost a lot of flexibility for making customizations. Some Plone 3 customers already complained about that - good for the solution provider, bad for the customer. Although I appreciate the cleaner architecture, it introduces too much complexity.  Example:  the recent portlet  is only available for  logged-in user.  If you want to make it available for anonymous users you have to subclass it and override its available() method. Sorry but that's the wrong way.
I think, that's it for now