salt.renderers.pydsl

A Python-based DSL

maintainer:Jack Kuan <kjkuan@gmail.com>
maturity:new
platform:all

The pydsl renderer allows one to author salt formulas (.sls files) in pure Python using a DSL that's easy to write and easy to read. Here's an example:

1
2
3
4
5
6
7
8
9
#!pydsl

apache = state('apache')
apache.pkg.installed()
apache.service.running()
state('/var/www/index.html') \
    .file('managed',
          source='salt://webserver/index.html') \
    .require(pkg='apache')

Notice that any Python code is allow in the file as it's really a Python module, so you have the full power of Python at your disposal. In this module, a few objects are defined for you, including the usual (with __ added) __salt__ dictionary, __grains__, __pillar__, __opts__, __env__, and __sls__, plus a few more:

__file__

local file system path to the sls module.

__pydsl__

Salt PyDSL object, useful for configuring DSL behavior per sls rendering.

include

Salt PyDSL function for creating Include declaration's.

extend

Salt PyDSL function for creating Extend declaration's.

state

Salt PyDSL function for creating ID declaration's.

A state ID declaration is created with a state(id) function call. Subsequent state(id) call with the same id returns the same object. This singleton access pattern applies to all declaration objects created with the DSL.

state('example')
assert state('example') is state('example')
assert state('example').cmd is state('example').cmd
assert state('example').cmd.running is state('example').cmd.running

The id argument is optional. If omitted, an UUID will be generated and used as the id.

state(id) returns an object under which you can create a State declaration object by accessing an attribute named after any state module available in Salt.

state('example').cmd
state('example').file
state('example').pkg
...

Then, a Function declaration object can be created from a State declaration object by one of the following two ways:

  1. by calling a method named after the state function on the State declaration object.
state('example').file.managed(...)
  1. by directly calling the attribute named for the State declaration, and supplying the state function name as the first argument.
state('example').file('managed', ...)

With either way of creating a Function declaration object, any Function arg declaration's can be passed as keyword arguments to the call. Subsequent calls of a Function declaration will update the arg declarations.

state('example').file('managed', source='salt://webserver/index.html')
state('example').file.managed(source='salt://webserver/index.html')

As a shortcut, the special name argument can also be passed as the first or second positional argument depending on the first or second way of calling the State declaration object. In the following two examples ls -la is the name argument.

state('example').cmd.run('ls -la', cwd='/')
state('example').cmd('run', 'ls -la', cwd='/')

Finally, a Requisite declaration object with its Requisite reference's can be created by invoking one of the requisite methods (see State Requisites) on either a Function declaration object or a State declaration object. The return value of a requisite call is also a Function declaration object, so you can chain several requisite calls together.

Arguments to a requisite call can be a list of State declaration objects and/or a set of keyword arguments whose names are state modules and values are IDs of ID declaration's or names of Name declaration's.

apache2 = state('apache2')
apache2.pkg.installed()
state('libapache2-mod-wsgi').pkg.installed()

# you can call requisites on function declaration
apache2.service.running() \
               .require(apache2.pkg,
                        pkg='libapache2-mod-wsgi') \
               .watch(file='/etc/apache2/httpd.conf')

# or you can call requisites on state declaration.
# this actually creates an anonymous function declaration object
# to add the requisites.
apache2.service.require(state('libapache2-mod-wsgi').pkg,
                        pkg='apache2') \
               .watch(file='/etc/apache2/httpd.conf')

# we still need to set the name of the function declaration.
apache2.service.running()

Include declaration objects can be created with the include function, while Extend declaration objects can be created with the extend function, whose arguments are just Function declaration objects.

include('edit.vim', 'http.server')
extend(state('apache2').service.watch(file='/etc/httpd/httpd.conf')

The include function, by default, causes the included sls file to be rendered as soon as the include function is called. It returns a list of rendered module objects; sls files not rendered with the pydsl renderer return None's. This behavior creates no Include declaration's in the resulting high state data structure.

import types

# including multiple sls returns a list.
_, mod = include('a-non-pydsl-sls', 'a-pydsl-sls')

assert _ is None
assert isinstance(slsmods[1], types.ModuleType)

# including a single sls returns a single object
mod = include('a-pydsl-sls')

# myfunc is a function that calls state(...) to create more states.
mod.myfunc(1, 2, "three")

Notice how you can define a reusable function in your pydsl sls module and then call it via the module returned by include.

It's still possible to do late includes by passing the delayed=True keyword argument to include.

include('edit.vim', 'http.server', delayed=True)

Above will just create a Include declaration in the rendered result, and such call always returns None.

Special integration with the cmd state

Taking advantage of rendering a Python module, PyDSL allows you to declare a state that calls a pre-defined Python function when the state is executed.

greeting = "hello world"
def helper(something, *args, **kws):
    print greeting                # hello world
    print something, args, kws    # test123 ['a', 'b', 'c'] {'x': 1, 'y': 2}

state().cmd.call(helper, "test123", 'a', 'b', 'c', x=1, y=2)

The cmd.call state function takes care of calling our helper function with the arguments we specified in the states, and translates the return value of our function into a structure expected by the state system. See salt.states.cmd.call() for more information.

Implicit ordering of states

Salt states are explicitly ordered via Requisite declaration's. However, with pydsl it's possible to let the renderer track the order of creation for Function declaration objects, and implicitly add require requisites for your states to enforce the ordering. This feature is enabled by setting the ordered option on __pydsl__.

Note

this feature is only available if your minions are using Python >= 2.7.

include('some.sls.file')

A = state('A').cmd.run(cwd='/var/tmp')
extend(A)

__pydsl__.set(ordered=True)

for i in range(10):
    i = str(i)
    state(i).cmd.run('echo '+i, cwd='/')
state('1').cmd.run('echo one')
state('2').cmd.run(name='echo two')

Notice that the ordered option needs to be set after any extend calls. This is to prevent pydsl from tracking the creation of a state function that's passed to an extend call.

Above example should create states from 0 to 9 that will output 0, one, two, 3, ... 9, in that order.

It's important to know that pydsl tracks the creations of Function declaration objects, and automatically adds a require requisite to a Function declaration object that requires the last Function declaration object created before it in the sls file.

This means later calls (perhaps to update the function's Function arg declaration) to a previously created function declaration will not change the order.

Render time state execution

When Salt processes a salt formula file, the file is rendered to salt's high state data representation by a renderer before the states can be executed. In the case of the pydsl renderer, the .sls file is executed as a python module as it is being rendered which makes it easy to execute a state at render time. In pydsl, executing one or more states at render time can be done by calling a configured ID declaration object.

#!pydsl

s = state() # save for later invocation

# configure it
s.cmd.run('echo at render time', cwd='/')
s.file.managed('target.txt', source='salt://source.txt')

s() # execute the two states now

Once an ID declaration is called at render time it is detached from the sls module as if it was never defined.

Note

If implicit ordering is enabled (ie, via __pydsl__.set(ordered=True)) then the first invocation of a ID declaration object must be done before a new Function declaration is created.

Integration with the stateconf renderer

The salt.renderers.stateconf renderer offers a few interesting features that can be leveraged by the pydsl renderer. In particular, when using with the pydsl renderer, we are interested in stateconf's sls namespacing feature (via dot-prefixed id declarations), as well as, the automatic start and goal states generation.

Now you can use pydsl with stateconf like this:

#!pydsl|stateconf -ps

include('xxx', 'yyy')

# ensure that states in xxx run BEFORE states in this file.
extend(state('.start').stateconf.require(stateconf='xxx::goal'))

# ensure that states in yyy run AFTER states in this file.
extend(state('.goal').stateconf.require_in(stateconf='yyy::start'))

__pydsl__.set(ordered=True)

...

-s enables the generation of a stateconf start state, and -p lets us pipe high state data rendered by pydsl to stateconf. This example shows that by require-ing or require_in-ing the included sls' start or goal states, it's possible to ensure that the included sls files can be made to execute before or after a state in the including sls file.

Importing custom Python modules

To use a custom Python module inside a PyDSL state, place the module somewhere that it can be loaded by the Salt loader, such as _modules in the /srv/salt directory.

Then, copy it to any minions as necessary by using saltutil.sync_modules.

To import into a PyDSL SLS, one must bypass the Python importer and insert it manually by getting a reference from Python's sys.modules dictionary.

For example:

#!pydsl|stateconf -ps

def main():
    my_mod = sys.modules['salt.loaded.ext.module.my_mod']
salt.renderers.pydsl.render(template, saltenv='base', sls='', tmplpath=None, rendered_sls=None, **kws)