Maaate plugin module guide ========================== Last modified: Dec 06 2001 Authors: Silvia Pfeiffer Conrad Parker Thomas Vincent Maaate home: http://www.cmis.csiro.au/Maaate/ 1. Overview ----------- Maaate`s MPEG parser implements the parsing of an MPEG audio stream and provides convenience functions to access the fields of an MPEG audio frame. Maaate's tier 1 implement the API and an access to pre-processed information useful for further analysis. To conduct signal analysis using the contents of the fields, analysis routines have to be implemented. Tier 2 provides generic data structures to support the analysis task. In addition, it provides a module interface to plugin analysis routines. Modules are routines that provide some analysis function to Maaate. They get compiled separately from Maaate and linked into their own shared library, which implies that they are developed separately from Maaate. They are however dynamically loaded into Maaate. An application will thus tell Maaate to load those libraries that contain the modules it requires. After loading they are available to the application. Some advantages of the plugin interface are: * The Maaate libraries can be extended by analysis modules without ever having to recompile it. * The boundary between Maaate code and analysis module code is explicit. An author of a plugin module does not have to add any code into Maaate for his module to be used with Maaate. * The separation of Maaate and the modules simplifies determination of legal ownership of code. This documentation explains what we understand by a module and a plugin library and how to write Maaate modules and create Maaate plugin libraries. There exists a plugin library in the Maaate source archive that provides some modules. They can be found in the src/plugins/ directory and may be useful to inspect. 2. Module specification ----------------------- Inspect the file src/tier2/module.H to find the definitions related to this section. 2.1 What is a module? --------------------- A module is a collection of functions that provide some augmented functionality on top of Maaate. Each module contains * an init-function (required), which sets up the basic information of the module such as its name, author, or description, and the input and output parameter specification. * a default-function (required), which sets default values for input parameters and returns the input parameter list. * a suggest-function (optional, recommended), which takes an input parameter list, suggests parameter values based on information provided by other parameters, and changes constraints of input parameters as required. * a reset-function (optional), which provides the possibility to reset a module, e.g. internal processing values or parameter values. * an apply-function (required), which takes an input parameter list, performs its analysis function and returns the calculated output parameters. * a destroy-function (optional), which cleans up memory allocated within the module and deletes parameter specifications. In addition, a function to construct the module and add it to the list of available modules must be provided. // xxx: missing: function to unload module! Examples of modules are * feature extraction modules such as energy, spectral centroid or spectral bandwidth modules. Such modules usually have to make use of the tier 1 field and pre-processed information access functions and store their results in one of the convenience containers supplied by tier 2. An example of such a module is the sumscf module which can be inspected in the Maaate source archive under src/plugins/sumscf.cc. * feature analysis modules that use the extracted features for some further (usually statistical) analysis such as clustering, segmentation or histogram modules. These modules usually make use of a filled container and store their results in another convenience container. An example of such a module is the segmentation module which can be inspected in the Maaate source archive under src/plugins/segmentation.cc. * content analysis modules that calculate higher level information using feature extraction and analysis modules, such as silence / music / speech determination. Suchd modules usually call other modules to calculate their results, which again may be stored in convenience containers. An example of such a module is the silence segmentation module which can be inspected in the Maaate source archive under src/plugins/silences.cc. A module is an instance of the Module class, which also provides convenience functions to get information on the instantiated module, handle input and output parameters, check constraints on parameters and call the module functions. 2.2 Module parameters --------------------- The apply-function of a module contains the implementation of the analysis functionality provided to Maaate. It takes as input a list of parameters and produces as a result of its processing a list of output parameters. Module parameters are handled by an application as follows: The init-function sets up the list of parameter specifications for input and output parameters. Thereafter, the application has the possibility to set default values for input parameters by calling the default-function. The application sets the input parameter values as it requires. Then it has the possibility to call the suggest-function which will take care of setting further parameter values and parameter constraints based upon internal module knowledge and the provided parameter values. It will also check provided parameter values for sanity and change them appropriately. Now, the application may call the apply-function. The first step within the apply-function is to check parameter values again for being within constraints (such as acceptable numeric ranges or predefined values). So, if an application has decided not to use the default- and suggest-functions, this still ensures that provided parameter values are sane. Constraints are hard limits, i.e. the apply-function will only be called if the parameters satisfy their constraints. The apply-function will return a list of output parameter values, which contain the results of the module execution. 2.3 Module parameter data types ------------------------------- Data types of parameters are either basic types or complex types. They are all enumerated in the type MaaateType. A parameter is an instance of the ModuleParam class, which also provides convenience functions to handle module parameters. 2.3.1 Basic types ----------------- The Maaate module interface provides the following basic types for parameters to be passed to a module or resulting from a module: * a boolean type: MAAATE_TYPE_BOOL, * an integer type: MAAATE_TYPE_INT, (this type may also be used to pass file descriptors to or from the module) * a real type: MAAATE_TYPE_REAL, and * a string type: MAAATE_TYPE_STRING. 2.3.2 Complex types ------------------- The Maaate module interface provides the following complex types for parameters to be passed to a module or resulting from a module: * a pointer to an opened sound file SOUNDfile * (constructed using tier 1; see file src/tier1/SOUNDfile.H): MAAATE_TYPE_SOUNDFILE, * a pointer to a segment data structure SegmentData *, which is provided by tier 2 (see file src/tier2/segmentData.H) and contains for a certain specified time period a matrix of values: MAAATE_TYPE_SEGMENTDATA, and * a pointer to a segment table SegmentTable *, which is provided by tier 2 (see file src/tier2/segmentTable.H) and contains a collection of SegmentData structures thus covering several time periods: MAAATE_TYPE_SEGMENTTABLE. 2.4 Parameter constraints ------------------------- Modules are provided with input parameter values via the input parameter list. A module usually requires parameter values to be within a specific range of allowed values. This is what we call parameter constraints. There are three types of constraints: * no constraints (MAAATE_CONSTRAINT_NONE): parameter values are allowed to take on any value of the parameter's data type, * a (list of) single value(s) (MAAATE_CONSTRAINT_VALUE): parameter values are allowed to take on any value of a list of provided values of the parameter's data type, * a (list of) value range(s) (MAAATE_CONSTRAINT_RANGE): parameter values are allowed to take on any value contained within any of the ranges in a range list. A single constraint is an instance of the class ModuleParamConstraint being either a single value (ModuleParam) or a single range (ModuleParamRange). For one parameter, there is usually a list of constraints, which is realized by instantiating the MaaateConstraint class. This class also provides convenience functions to handle constraints such as adding constraints or checking if values satisfy constraints. 2.5 Parameter specifications ---------------------------- A module is a generic interface to an analysis function. Every analysis function however requires different input parameters and produces different output parameters. The generic interface therefore specifies that a list of (generic) parameters be accepted by a module and a list of parameters be output from a module. The exact specification of the parameters can only be produced by the module itself. This is performed in the module's init-function. A parameter specification is an instance of the ModuleParamSpec class, which contains the specification of a single parameter. The specification consists of the parameter's name, a description, its data type (see 2.3 for possible data types), default value and constraints (see 2.4 for constraint descriptions). The default value that is provided to a parameter must be of the data type of its parameter specification. 3. Plugin libraries ------------------- Inspect the file src/tier2/plugins.H to find the definitions related to this section. A plugin library is a separately compiled, shared library. It may contain one or many modules. Several such libraries may be dynamically loaded into Maaate. Single modules or complete libraries may also be unloaded from Maaate as required by the application. The modules are all administrated within one instance of the Plugins class. This class therefore provides functionality to load and unload single modules and whole plugin libraries, and administrates the list of available modules. It also provides functions to access modules by their name. 4. API versioning ----------------- Maaate's libraries are strictly versioned: both tier1 and tier2 have a separate version numbering that tracks changes to their API. Here, we concentrate on the API of tier2, the plugin API. Versioning of the plugin API tracks the interface provided by tier2. The interface consists of the classes defined in segmentData.H, segmentTable.H, module.H and plugins.H. Tracking changes to the interface is required because applications which are actively linking themselves against plugin libraries, must look for plugins which implement versions they (or rather their Maaate code) understand. Versions are given in the form Major.Minor.Revision (M.m.R). The API version implemented by a particular release of Maaate tier2 is defined in by the values MAAATE_PLUGIN_API_MAJOR, MAAATE_PLUGIN_API_MINOR and MAAATE_PLUGIN_API_REVISION. The version information is only ever updated upon a release of Maaate. The version information V = M.m.R is updated to V' as follows: 1. If any interfaces have been removed or modified by a release (ie. backwards compatability has been broken) then the Major number is incremented and Minor and Revision are set to 0. (V' = M+1.0.0) 2. Otherwise, if any interfaces have been added since the last release the Minor number is incremented and Revision is set to 0. (V' = M.m+1.0) 3. Otherwise, if the application code has changed at all since the last release the Revision is incremented. (V' = M.m.R+1) A plugin library 'plugin' must be built with a name of the form libplugin.so.M.m.R where M, m, and R correspond to the Major, Minor and Revision provided by the particular version of Maaate you are developing against, or an earlier Minor and Revision which are known to work. The current behaviour of Maaate is to attempt to load all plugins with a Major number it understands, and fail silently if the plugin makes use of interfaces added in subsequent Minor API versions. Thus it is not absolutely necessary to find the earliest API version which will work with your plugin. Additionally, symlinks should be created for the plugin as follows: libplugin.so -> libplugin.so.M libplugin.so.M -> libplugin.so.M.m libplugin.so.M.m -> libplugin.so.M.m.R These symlinks are created automatically when using libtool to build plugin libraries by including the line libplugin_la_LDFLAGS = -version-info M:m:R in the Makefile.am within the plugin's build directory (if using automake; otherwise the flag "-version-info M:m:R" needs to be passed in some other way to libtool). Note that this flag contains colons, not dots. 5. Writing a module ------------------- Inspect the file src/plugins/sumscf.cc to find an example module implementation. The following header file has to be included in files containing module code: . It includes all required Maaate header files including tier1 and tier2 header files. When you want to write a module, you have to provide the functions that specify a module: an init-function, a default-function, a suggest-function, a reset-function, an apply-function, and a destroy-function (see also section 2.1). The names you give to these functions are arbitrary, however we propose to call them init_, default_ etc. as we have done with our modules. They should be static functions in order not to appear in the library's symbol table. Here are the generic layouts of these functions: // init-function: only takes a Module pointer typedef void (*ModuleInitFunc) (Module *); // default-function: takes a Module pointer and // returns the input parameter list typedef list (*ModuleDefaultFunc) (Module * m); // suggest-function: takes a Module pointer and a pointer to the // input parameter list which it might change typedef void (*ModuleSuggestValues) (Module * m, list * paramsIn); // apply-function: takes a Module pointer and the input parameter list // and returns the output parameter list typedef list (*ModuleApplyFunc) (Module * m, list * paramsIn); // destroy-function: only takes a Module pointer typedef void (*ModuleDestroyFunc) (Module *); // reset-function: only takes a Module pointer typedef void (*ModuleResetFunc) (Module *); In the init-function, you need to set up the module's specification consisting of its description (modName, modDesc, modAuthor, modCopyright, modUrl) and the input and output parameter specifications (modParamInSpecs and modParamOutSpecs), possibly with constraints. The modName must be a short string without blanks which is used to identify the module. The modDesc should be a somewhat longer ASCII text describing what this module does. The modAuthor filed should be a comma separated list of names of authors of this procedure, each optionally providing an email address in angle brackets. The modCopyright field should be a string such as string("(c) 2000 Sue Clancy"). The modUrl field can optionally give the URL of a web page describing the module in more detail. The module parameters have also to be specified. When adding another parameter specification, it is required to specify an identifying name for the parameter, a description, the parameter's type, its default value and possibly constraints. To that purpose, if you want your module to work with Bewdy, you need to set the first parameter as either the SOUNDfile or the SegmentData the module is taking as an input. The second parameter must be the start time of the analysis and the third one the end time. You also should specify a default-function, which creates a parameter list from the parameter specification given in the init-function and sets default values for the parameters. Input and output parameters of modules are stored in a list (list). The parameter values have to adhere to the parameter specification defined during the init-function of the module (list) and come in the correct order. You don't need to specify a suggest-, reset- or destroy-function, except if you would like to provide their functionality to an application. We recommend to at least specify the suggest-function, which changes constraints and parameter values according already selected values. It also assures that constraints are met (see also section 2.2). The apply-function is the function the provides the functionality of the module to an application. If you would like to share information between the functions of a Module or between different calls of a function, it is best to declare static global variables within the plugin file and communicate values through these variables. Thus, it is best to specify all the module functions within one file. Remember to specify the functions and global variables as static to prevent them from appearing in the library's symbol table. Compiling a module is best done using automake and libtool, which will take care to produce the correct shared libraries. See the next section for an example Makefile.am. 6. Building plugin libraries ---------------------------- When building a shared library to contain one or more modules, a function called loadModules has to be supplied, which instantiates the modules of the library and returns their list to Maaate (see src/plugins/loadModules.cc for an example). Similarly an unloadModules function has to be supplied, which deletes the instantiated modules. These are the only names that should be exposed in the library's symbol table; declare all other functions and global variables as static. In order to load several modules contained within one plugin library, it is best to specify within each module's file another function (let's call it apple :) that instantiates a Module with the given functions and returns it. Then, there has to be a separate file that contains the loadModules function, which calls all the apple functions, pushing the instantiated Modules onto a Module list which gets returned by the loadMoudles function (see /src/plubins/sumscf.cc and src/plugins/loadModules.cc for an example). The build environment for a plugin library is simple if Maaate is installed and you are using automake and libtool. Makefile.am: ----------------------------------------------------------------------- ## Process this file with automake to produce Makefile.in CXXFLAGS = -O2 CFLAGS = # install plugin libraries in PACKAGE_PLUGIN_DIR # which should be ${prefix}/lib/Maaate/ libdir = @PACKAGE_PLUGIN_DIR@ # Libraries to build lib_LTLIBRARIES = libExample.la #per-Library objects & includes libExample_la_SOURCES = loadModules.cc example1.cc example2.cc libExample_la_LDFLAGS = -version-info 1:0:0 ----------------------------------------------------------------------- The version info (1:0:0 in the example above) should consist of the numbers MAAATE_PLUGIN_API_MAJOR, MAAATE_PLUGIN_API_MINOR and MAAATE_PLUGIN_API_REVISION listed in , colon separated. Copy the numbers in manually, do not attempt to generate them from the values defined. These numbers should only be changed when your plugin code changes to use features only available in newer versions of the plugin API. Note that the plugin API (the one for tier2) is maintained independently of the distribution version and that it may have a different version to the API of tier1. 6. Using plugins in programs ---------------------------- If an application wants to dynamically load modules, it first requires to create an instance of the Plugins class and then load the shared module libraries with any of the loading functions defined within the Plugins class: // open known plugin libraries Plugins * plugins = new Plugins(); plugins->AddLibrary(string("libMaaateM.so")); The AddLibrary call also creates an instance of the PluginLibrary class which loads all the contained modules into Maaate and sets up input and output parameter specifications for all of them. It searches them in the default directory under the paths given in the environment variable MAAATE_PATH and the standard paths ${prefix}/lib/Maaate/ and /usr/local/lib/ . The application may then call the following functions in sequence to use a module's functionality (see test/analyseMPaudio.cc for an example): // returns a pointer to the requested module Module * m = plugins->GetModule("modulename"); // creates the input parameter list for a module from its specification list paramsIn = m->defaultValues(); // application makes changes to parameter values // possibly call suggestValues: // suggests further parameter values and changes constraints m->suggestValues(¶msIn); // application makes possibly further changes to parameter values // performs operations using input parameter values and stores // results in output parameters list paramsOut = m->apply(¶msIn); // possibly call reset() // resets internal module values m->reset(); // possibly call apply() again etc. ~PluginLibrary(); -> deletes the module after calling its destroy function and unloads all the modules of the library from Maaate Here is an example Makefile.am for an application using Maaate's plugin interface with automake and libtool: ## Process this file with automake to produce Makefile.in CXXFLAGS = -O2 # Executables to build bin_PROGRAMS = application application_SOURCES = application.cc application_LDADD = libMaaateP.la libMaaateA.la -ldl Acknowledgements ---------------- This API was constructed after careful studying of plugin APIs from many other free software projects, including sweep, SANE, fame and the Gimp.