salt.pillar.git_pillar¶

Use a git repository as a Pillar source¶

Note

This external pillar has been rewritten for the 2015.8.0 release. The old method of configuring this external pillar will be maintained for a couple releases, allowing time for configurations to be updated to reflect the new usage.

This external pillar allows for a Pillar top file and Pillar SLS files to be sourced from a git repository.

However, since git_pillar does not have an equivalent to the pillar_roots parameter, configuration is slightly different. A Pillar top file is required to be in the git repository and must still contain the relevant environment, like so:

base:
  '*':
    - foo

The branch/tag which maps to that environment must then be specified along with the repo's URL. Configuration details can be found below.

Important

Each branch/tag used for git_pillar must have its own top file. This is different from how the top file works when configuring States. The reason for this is that each git_pillar branch/tag is processed separately from the rest. Therefore, if the qa branch is to be used for git_pillar, it would need to have its own top file, with the qa environment defined within it, like this:

qa:
  'dev-*':
    - bar

Additionally, while git_pillar allows for the branch/tag to be overridden (see here, or here for Salt releases before 2015.8.0), keep in mind that the top file must reference the actual environment name. It is common practice to make the environment in a git_pillar top file match the branch/tag name, but when remapping, the environment of course no longer matches the branch/tag, and the top file needs to be adjusted accordingly. When expected Pillar values configured in git_pillar are missing, this is a common misconfiguration that may be to blame, and is a good first step in troubleshooting.

Configuring git_pillar for Salt releases before 2015.8.0¶

For Salt releases earlier than 2015.8.0, GitPython is the only supported provider for git_pillar. Individual repositories can be configured under the ext_pillar configuration parameter like so:

ext_pillar:
  - git: master https://gitserver/git-pillar.git root=subdirectory

The repository is specified in the format <branch> <repo_url>, with an optional root parameter (added in the 2014.7.0 release) which allows the pillar SLS files to be served up from a subdirectory (similar to gitfs_root in gitfs).

To use more than one branch from the same repo, multiple lines must be specified under ext_pillar:

ext_pillar:
  - git: master https://gitserver/git-pillar.git
  - git: dev https://gitserver/git-pillar.git

To remap a specific branch to a specific Pillar environment, use the format <branch>:<env>:

ext_pillar:
  - git: develop:dev https://gitserver/git-pillar.git
  - git: master:prod https://gitserver/git-pillar.git

In this case, the develop branch would need its own top.sls with a dev section in it, like this:

dev:
  '*':
    - bar

The master branch would need its own top.sls with a prod section in it:

prod:
  '*':
    - bar

If __env__ is specified as the branch name, then git_pillar will first look at the minion's environment option. If unset, it will fall back to using branch specified by the master's gitfs_base:

ext_pillar:
  - git: __env__ https://gitserver/git-pillar.git root=pillar

The corresponding Pillar top file would look like this:

{{env}}:
  '*':
    - bar

Note

This feature was unintentionally omitted when git_pillar was rewritten for the 2015.8.0 release. It was added again in the 2016.3.4 release, but it has changed slightly in that release. On Salt masters running 2015.8.0 through 2016.3.3, this feature can only be accessed using the legacy config described above. For 2016.3.4 and later, refer to explanation of the __env__ parameter in the below section.

Versions 2016.3.0 through 2016.3.4 incorrectly check the master's environment config option (instead of the minion's) before falling back to gitfs_base. This has been fixed in the 2016.3.5 and 2016.11.1 releases (2016.11.0 contains the incorrect behavior).

Configuring git_pillar for Salt releases 2015.8.0 and later¶

Note

In version 2015.8.0, the method of configuring git external pillars has changed, and now more closely resembles that of the Git Fileserver Backend. If Salt detects the old configuration schema, it will use the pre-2015.8.0 code to compile the external pillar. A warning will also be logged.

Beginning with Salt version 2015.8.0, pygit2 is now supported in addition to GitPython (Dulwich will not be supported for the foreseeable future). The requirements for GitPython and pygit2 are the same as for gitfs, as described here.

Important

git_pillar has its own set of global configuration parameters. While it may seem intuitive to use the global gitfs configuration parameters (gitfs_base, etc.) to manage git_pillar, this will not work. The main difference for this is the fact that the different components which use Salt's git backend code do not all function identically. For instance, in git_pillar it is necessary to specify which branch/tag to be used for git_pillar remotes. This is the reverse behavior from gitfs, where branches/tags make up your environments.

See here for documentation on the git_pillar configuration options and their usage.

Here is an example git_pillar configuration:

ext_pillar:
  - git:
    # Use 'prod' instead of the branch name 'production' as the environment
    - production https://gitserver/git-pillar.git:
      - env: prod
    # Use 'dev' instead of the branch name 'develop' as the environment
    - develop https://gitserver/git-pillar.git:
      - env: dev
    # No per-remote config parameters (and no trailing colon), 'qa' will
    # be used as the environment
    - qa https://gitserver/git-pillar.git
    # SSH key authentication
    - master git@other-git-server:pillardata-ssh.git:
      # Pillar SLS files will be read from the 'pillar' subdirectory in
      # this repository
      - root: pillar
      - privkey: /path/to/key
      - pubkey: /path/to/key.pub
      - passphrase: CorrectHorseBatteryStaple
    # HTTPS authentication
    - master https://other-git-server/pillardata-https.git:
      - user: git
      - password: CorrectHorseBatteryStaple

The main difference between this and the old way of configuring git_pillar is that multiple remotes can be configured under one git section under ext_pillar. More than one git section can be used, but it is not necessary. Remotes will be evaluated sequentially.

Per-remote configuration parameters are supported (similar to gitfs), and global versions of the git_pillar configuration parameters can also be set.

To remap a specific branch to a specific Pillar environment, use the env per-remote parameter:

ext_pillar:
  - git:
    - production https://gitserver/git-pillar.git:
      - env: prod

If __env__ is specified as the branch name, then git_pillar will use the branch specified by git_pillar_base:

ext_pillar:
  - git:
    - __env__ https://gitserver/git-pillar.git:
      - root: pillar

The corresponding Pillar top file would look like this:

{{env}}:
  '*':
    - bar

Note

This feature was unintentionally omitted when git_pillar was rewritten for the 2015.8.0 release. It was added again in the 2016.3.4 release, but it has changed slightly in that release. The fallback value replaced by {{env}} is :conf_master: is git_pillar_base, while the legacy config's version of this feature replaces {{env}} with gitfs_base.

On Salt masters running 2015.8.0 through 2016.3.3, this feature can only be accessed using the legacy config in the previous section of this page.

The same issue which affected the behavior of the minion's environment config value using the legacy configuration syntax (see the documentation in the pre-2015.8.0 section above for the legacy support of this feature) also affects the new-style git_pillar syntax in version 2016.3.4. This has been corrected in version 2016.3.5 and 2016.11.1 (2016.11.0 contains the incorrect behavior).

2016.3.4 incorrectly checks the master's environment config option (instead of the minion's) before falling back to the master's git_pillar_base.

With the addition of pygit2 support, git_pillar can now interact with authenticated remotes. Authentication works just like in gitfs (as outlined in the Git Fileserver Backend Walkthrough), only with the global authenication parameter names prefixed with git_pillar instead of gitfs (e.g. git_pillar_pubkey, git_pillar_privkey, git_pillar_passphrase, etc.).

Note

The name parameter can be used to further differentiate between two remotes with the same URL and branch. When using two remotes with the same URL, the name option is required.

salt.pillar.git_pillar.ext_pillar(minion_id, repo, pillar_dirs)¶: Checkout the ext_pillar sources and compile the resulting pillar SLS