MOAI is a server platform, but to be a server you need stuff to be served. This is where the ContentProviders come into play. They provide content, that can come from different sources.

A content provider can be asked to update it’s list of ids. Afterwards, the actual content can be retrieved by id.

Interface: moai.interfaces.IContentProvider

Object that provides the content used to build the database that is used to serve the actual oai data

Method: count()
Returns number of content objects in the repository returns None if number is unknown, this should not be called before update is called
Method: set_logger(self, log)
Set the logger instance for this class
Method: get_content_by_id()
Return content of a specific id
Method: update(from_date=None)
Harvests new content added since from_date returns a list of content_ids that were changed/added, this should be called before get_contents is called
Method: get_content_ids()
returns a list/generator of content_ids


Content itself should implement the IContentObject interface. This makes sure that every content object has some specific attributes that are needed to be able to serve them in an oai feed.

Besides these required attributes, the CotentObject also has a field_names and get_values method that can be used to return additional field names and their values.

If you need to serve content coming from your custom XML files, or database schemas, then you will probably need to create an implementation of this object.

Interface: moai.interfaces.IContentObject

Attribute: deleted
Boolean that tells if object is deleted or not
Method: update(data, provider)
Called by IContentProvider, to fill the object with data
Attribute: when_modified
Modification date of the object
Attribute: provider
ContentProvider instance that created this object
Method: get_assets()

Return a list of python dictionaries, each dictionary contains at least the following keys:

  • url - Url of the asset, this will be used in feeds - filename - The filename

of the asset - md5sum - md5 checksum of the asset - mimetype - mimetype of the asset - absolute_uri - file:/// or http:/// uri referencing the file - metadata - dictionary with lists of strings as values holding additional metadata

Attribute: label
Name of the object
Attribute: content_type
Type of the content object
Attribute: sets
A list of ids from sets that this object belongs to
Attribute: is_set
Boolean indicating if this object is a set
Method: field_names()
Return a list of field names, used in this object
Attribute: id
Id of the content object
Method: get_values(field_name)
Return a list of python objects (string/int/etc) from a specific field name


Interface: moai.interfaces.IDatabaseUpdater

Method: update_database_iterate(validate=True, supress_errors=False)

Update the database with the content_provider this will update the content_provider, optionally validate the content objects, and add everything to the database. If supress_errors is True, this method should never return an error, instead it should yield tuples containing the following values:

(count, total, provider_id, exception)

count: current content object number total: total number of objects provider_id: id used by provider exception: if an error occurs, exception should be a moai.ContentError, or a moai.DatabaseError otherwise this value will be None

After the database update is finished, the flush_update method of the database is called. This allows the database to implement a batching strategy

Method: set_logger(logger_instance)
Make the updater use a specific custom logger (will probably be set automaticly in __init__)
Method: update_database(validate=True, supress_errors=False)
Iterates through update_database_iterate in a loop, returns the number of errors that occured (int)
Method: set_content_provider(content_provider)
Make the updater use a specific ContentProvider (will probably be set automaticly in __init__)
Method: update_provider_iterate(from_date=None)
Updates the provider from a specific date, yields the ids that where updated
Method: update_provider(from_date=None)
Iterates through update_provider_iterate in a loop, returns a list of updated ids
Method: set_content_class(self, content_object_class)
Sets the class to be used to create the content objects from the provider data
Method: set_database(database)
Make the updater use a specific (new) database (will probably be set automaticly in __init__)


Interface: moai.interfaces.IDatabase

Method: oai_sets(offset=0, batch_size=20)

Used by queries from the OAI server. Format returned should be the following:

[{‘id’: <string>, ‘name’: <string>, ‘description’: <string>}]

Method: flush_update()
Called once by the database updater at the end of the update proces This allows the database to implement a batching strategy
Method: get_set(id)

Returns a dictionary of set info containing - id - name - description

If the id does not exist, None is returned

Method: get_assets(id)

Returns a list of dictionaries describing the assets Each dictionary contains the following fields: - filename - url - mimetype - md5 - absolute_uri - metadata

Where metadata is a dictionary with additional lists of string values

Method: add_content(id, sets, record_data, meta_data, assets_data)
Add content to the database, supplying an id and 3 dictionaries, of data. The dictionaries should contain at least the keys that are needed for generating the get_record, get_metadata and get_keys requests. Returns a boolean to indicate if the insertion was succesful
Method: remove_content(id)
Remove all the content of a given id, returns a boolean to indicate if the removal was succesful
Method: get_record(id)

Returns a dictionary of data that is available from the object with the specific id. The dictionary should contain at least the following data:

{‘id’: unicode, ‘when_modified’: dateTime, ‘deleted’: boolean, ‘sets’: list of strings, }

If the id does not exist, None is returned

Method: add_set(id, name, description=None)
Add a set to the database Returns a boolean to indicate if the insertion was succesful
Method: get_sets(id)
Returns a list of set ids for a specific id,
Method: get_metadata(id)

Returns a dictionary with additional data. Keys are always a string, values are always lists of python objects.

If the id does not exist, None is returned

Method: remove_set(id)
Remove set from the database Returns a boolean to indicate if the removal was succesful
Method: oai_query(offset=0, batch_size=20, sets=[], not_sets=[], filter_sets=[], from_date=None, until_date=None, identifier=None)

Used by queries from the OAI server. Format returned should be the following:

[{‘record’: <dict similar to get_record() output>, ‘metadata’: <dict similar to get_metadata() output>, ‘assets’: <dict similar to get_assets() output>} ]


Interface: moai.interfaces.IFeedConfig

Method: get_asset_path(internal_id, asset)
Return an absolute path to an asset given an internal id the asset data dict containing filename, md5, url and metadata
Attribute: log
Logger instance that logs activity and errors
Attribute: sets_allowed
Objects served must have one of these sets
Attribute: url
Base URL of the OAI Server (for identify)
Method: get_oai_id(internal_id)
Rename internal id into oai_id
Method: get_setspec_id(internal_set_id)
Rename internal set id into a setspec id
Attribute: delay
number of miliseconds to delay the feed
Attribute: sets_disallowed
Objects served must not have one of these sets
Attribute: admins
List of email addresses that can be contacted, for questions about the feed
Attribute: filter_sets
Objects served must have one of these sets, besides theconforming to the (dis-)allowed sets
Attribute: content_type
Type of content objects being served
Attribute: metadata_prefixes
List of metadataPrefixes this server can handleby default the list has ‘oai_dc’ included
Attribute: scope
Only serve objects with a specific (or lower) scope
Method: get_internal_id(oai_id)
Rename oai_id into internal id
Method: get_internal_set_id(oai_setspec_id)
Rename setspec id into internal set id
Attribute: id
Id of the OAI Server instance
Attribute: name
Name of this OAI Server instance (for identify)


Interface: moai.interfaces.IServerRequest

Method: redirect(url)
Redirect to this url
Method: url()
Return the current url
Method: query_dict()
Return a dictionary with QueryString values of the request
Method: write(data, mimetype)
Write data back to the client
Method: send_file(path)
Send the file located at ‘path’ back to the user
Method: send_status(code, msg='', mimetype='text/plain')
Return a status code to the user


Interface: moai.interfaces.IServer

Method: is_asset_url(url, config)
Is this url pointing to an asset (returns bool)
Method: handle_request(req)
Serve this request this method goes through the following steps: 1. check if url is valid 2. try to get ServerConfig for this url 3. test if this is an asset url, if so check if download is allowed, and download asset 4. if not asset url, get the oai server through the OAIServerFactory 5. call the handleRequest method on the oai server, and return the result
Method: download_asset(req, url, config)
Download an asset from a url
Method: get_config(id)
Get a ServerConfig by id
Method: allow_download(url, config)
Is user allowed to download this asset (returns bool)
Method: add_config(config)
Add a ServerConfig to the server