Get configuration and interpretation into builders going.

Author: brettviren

doc/configuration.org

@@ -6,42 +6,66 @@ The GGD system is layered and allows the user/programmer access to each layer.
 
 * Configuration File Syntax
 
-The GGD configuration file syntax is parsed by Python's =configparser= module.  This syntax is also sometimes known as "INI".  It consists of a number of named sections which hold a number of key/value pairs.
+The GGD configuration file syntax is parsed by Python's =configparser= module.  This syntax is also sometimes known as "INI".  It consists of a number of named sections each holding a number of key/value pairs.
 
 #+BEGIN_EXAMPLE
-[section name]
+[name]
 key1 = value1
 #+END_EXAMPLE
 
 ** Syntax Extensions
 
-The GGD configuration layer will evaluate each section of key/value pairs in a limited Python environment.  This allows any given value to reference a key from the same section in order to use its value.  It also allows expression of complex data structures, units, arithmetic.  In principle this allows a high degree of programming and a balance should be struck.  In order to keep the configuration simple, complex programming should be put into the builder.
+The GGD configuration layer will process the file in a series of steps that add some features beyond the basic syntax.  Care should be exercised in exploiting these features.  If the configuration becomes complex consider if it would be better to move this complexity into the [[./builders.org][builder]] code.  
+
+*** Evaluation
+
+The values in the parsed configuration file are evaluated by the Python interpreter in order.  Within one section, prior keys may be referenced in subsequent values.  For example:
+
+#+BEGIN_EXAMPLE
+[mybuilder]
+x = 21
+y = x*2
+#+END_EXAMPLE
+
+results in =y= being 42.  Referencing a variable before it is defined is an error. 
 
 *** Specifying units
 
-All numerical quantities that have units must have them explicitly given in the configuration.  This is done with the =Q()= function ("Q" for "quantity").
+All numerical quantities that have units *must* have them explicitly given in the configuration.  If not, they will be caught as an error during geometry generation.  Units are specified with the =Q()= function ("Q" is for "quantity").  This is an abbreviation for the [[http://pint.readthedocs.org/en/0.5.1/tutorial.html][=pint.UnitRegistry.Quantity=]] method and as such can be "called" in the configuration file in any manner it could be called directly from Python.
 
 #+BEGIN_EXAMPLE
-[builder mydetector]
+[mybuilder]
 width = Q("10m")
-height = 10*width
-depth = 10*height
+height = Q(10, 'meter')
+depth = Q("10*m")
 #+END_EXAMPLE
 
+*** Data structures
 
+Because values are evaluated by the Python interpreter the configuration values may take the form of any valid Python data structure.  One particularly useful one is a list to give a vector of values.  For example:
 
-The content of the configuration file is parsed into a Python data structure by GGD and then interpreted following these conventions.  
+#+BEGIN_EXAMPLE
+[mybuilder]
+offset = [Q("1m"), Q("2m"), Q("3m")]
+#+END_EXAMPLE 
+
+* Interpretation
+
+After the configuration data is parsed and evaluated as above it is interpreted by GGD.  
 
 ** Builder sections
 
-Sections starting with "=[builder NAME]=" are interpreted as being for a builder.  Any key/value pairs given will be passed to the builder named "=NAME=" if it is created.  Certain keys are reserved by GGD:
+Each configuration section corresponds to a single instance of a builder object (see [[./builders.org][Builders]]) of the same name.  Any key/value pairs given will be passed to the builder for that name, if it is created.  Builder creation starts at a given builder section or the first section if none is given.
+
+Certain keys are reserved by GGD for interpretation:
 
 - =class= :: The fully qualified class name (eg "=module.submodule.MyBuilderClass=").
-- =subbuilders= :: A space or comma separated list of builder names to be given to the builder as sub-builders.  
+- =subbuilders= :: A list of builder instance names to be given to the builder as sub-builders.  
 
 Beyond these reserved keys each builder is free to expect its own set of keys.
 
-** Example
+
+* Examples
 
 A working example is in the source at [[../python/gegede/examples/lar.cfg][lar.cfg]].
 

python/gegede/__init__.py

@@ -1,3 +1,5 @@
 import pint
 units = pint.UnitRegistry()
 Quantity = units.Quantity
+
+

python/gegede/configuration.py

@@ -2,12 +2,74 @@
 '''
 gegede configuration
 '''
+import os
+from collections import OrderedDict
 
 def parse(filenames):
-    '''Parse the filename, return the parser object.'''
+    '''Parse configuration files.
+
+    Return the parser object.'''
+
     try:                from ConfigParser import SafeConfigParser
     except ImportError: from configparser import SafeConfigParser
     cfg = SafeConfigParser()
     cfg.optionxform = str       # want case sensitive
-    cfg.read(filenames)
+    
+    if isinstance(filenames, type("")):
+        filenames = [filenames]
+    for fname in filenames:
+        if not os.path.exists(fname):
+            raise ValueError, 'No such file: %s' % fname
+        cfg.read(fname)
     return cfg
+
+
+def cfg2pod(cfg):
+    '''
+    Convert a ConfigParser object to a plain-old-data structure
+    '''
+    pod = OrderedDict()
+    for secname in cfg.sections():
+        secdat = OrderedDict()
+        for k,v in cfg.items(secname):
+            secdat[k] = v
+        pod[secname] = secdat
+    return pod
+
+def make_class(fqclass):
+    mod, cls = fqclass.rsplit('.',1)
+    exec('import %s' % mod) # better way?
+    return eval(fqclass)
+
+def make_value(v, **kwds):
+    from . import Quantity
+    return eval(v, globals(), dict(Q=Quantity, **kwds))
+
+
+def evaluate(pod):
+    '''
+    Evaluate and replace each value.
+    '''
+    ret = OrderedDict()
+    for secname, secdat in pod.items():
+        newdat = OrderedDict()
+        for k,v in secdat.items():
+            if k == 'class':
+                newdat[k] = make_class(v)
+                continue
+            newdat[k] = make_value(v, **newdat)
+        ret[secname] = newdat
+    return ret
+
+def configure(filenames):
+    '''
+    Return evaluated configuration 
+    '''
+    cfg = parse(filenames)
+    assert cfg.sections()
+    pod = cfg2pod(cfg)
+    assert pod
+    dat = evaluate(pod)
+    assert dat
+    return dat
+

python/gegede/examples/builders.py

@@ -12,10 +12,12 @@ class SimpleBuilder(gegede.builder.Builder):
     '''
 
     def configure(self, dx='1m', dy='1m', dz='1m', mat='Air', **kwds):
+        #print 'Configuring "%s"' % self.name
         self.box_mat = mat      # assume made somewhere else
         self.box_dim = (dx,dy,dz)
 
     def construct(self, geom):
+        #print 'Constructing "%s"' % self.name
         shape = geom.shapes.Box(self.name + '_box_shape', *self.box_dim)
         lv = geom.structure.Volume(self.name+'_volume', material=self.box_mat, shape=shape)
         self.add_volume(lv)

python/gegede/examples/lar.cfg

@@ -1,11 +1,10 @@
-#+BEGIN_EXAMPLE
-[builder world]
+[world]
 subbuilders = site
 class = gegede.examples.lar.WorldBuilder
-size = 50m
+size = Q("50m")
 material = Rock
 
-[builder site]
+[site]
 subbuilders = tank
 class = gegede.examples.lar.TankBuilder
-#+END_EXAMPLE
+

python/gegede/interp.py

@@ -0,0 +1,28 @@
+#!/usr/bin/env python
+'''Some functions that to the interpretation of evaluated
+configuration data into builder objects.
+'''
+
+def get_builder_config(dat, name):
+    '''
+    Return the builder's configuration section of the given <name>.
+
+    If <name> is None, return first builder in <dat>.
+    '''
+    if name is None:
+        return dat.items()[0]
+    return name, dat[name]
+
+def make_builder(dat, name = None):
+    '''Return the a builder object of the given name from the
+    configuration data <dat> and recursively call on any subbuilders.
+    If no name is given return the first found.
+    '''
+    bname, bdat = get_builder_config(dat, name)
+    bobj = bdat['class'](bname)
+    subbuilders = bdat.get('subbuilders',list())
+    for sbname in subbuilders:
+        sb = make_builder(dat, sbname)
+        bobj.builders.append(sb)
+    return bobj
+    

python/gegede/main.py

@@ -0,0 +1,22 @@
+#!/usr/bin/env python
+'''
+High level main functions.
+'''
+import gegede.configuration
+import gegede.interp
+import gegede.builder
+
+
+
+def generate(filenames, world_name = None):
+    '''
+    Return a geometry object generated from the given configuration file(s).
+    '''
+    assert filenames
+    cfg = gegede.configuration.configure(filenames)
+    assert cfg
+    wbuilder = gegede.interp.make_builder(cfg, world_name)
+    gegede.builder.configure(wbuilder, cfg)
+    geom = gegede.construct.Geometry()
+    gegede.builder.construct(wbuilder, geom)
+    return geom

tests/test_configuration.cfg

@@ -1,5 +1,21 @@
-[builder mybuilder]
+[mybuilder]
 class = gegede.examples.builders.SimpleBuilder
+subbuilders = ["sb1", "sb2"]
+nickname = "Digger Dan"
 width = Q("10m")
 height = 10*width
 depth = height*1.23
+dims = [width, height, depth]
+offset = [Q("1m"), Q("2m"), Q("3m")]
+
+[sb1]
+class = gegede.examples.builders.SimpleBuilder
+subbuilders = ["ssb1", "ssb2"]
+
+[sb2]
+class = gegede.examples.builders.SimpleBuilder
+
+[ssb1]
+class = gegede.examples.builders.SimpleBuilder
+[ssb2]
+class = gegede.examples.builders.SimpleBuilder

tests/test_configuration.py

@@ -1,7 +1,18 @@
 #!/usr/bin/env python
 
+import os
 import gegede.configuration as ggdconf
 
 def test_read():
-    cfg = ggdconf.parse(__file__.replace('.py','.cfg'))
-    print cfg
+    fname = os.path.splitext(__file__)[0]+'.cfg'
+    cfg = ggdconf.parse(fname)
+    pod = ggdconf.cfg2pod(cfg)
+    dat = ggdconf.evaluate(pod)
+    assert dat
+
+
+def test_configure():
+    fname = os.path.splitext(__file__)[0]+'.cfg'
+    cfg = ggdconf.configure(fname)
+    worldname, worldcfg = cfg.items()[0]
+    assert 2 == len(worldcfg['subbuilders'])

tests/test_main.py

@@ -0,0 +1,14 @@
+#!/usr/bin/env python
+
+import os
+import gegede.main
+
+def test_main():
+    fname = os.path.dirname(os.path.realpath(__file__))+'/test_configuration.cfg'
+    g = gegede.main.generate(fname)
+
+    assert g
+    assert g.store.shapes
+    if not g.store.matter:
+        print "You still haven't implemented this, huh?"
+    assert g.store.structure