charmhelpers.contrib.bigdata.utils¶
- class charmhelpers.contrib.bigdata.utils.DistConfig(filename='dist.yaml', required_keys=None)¶
Bases: object
This class processes distribution-specific configuration options.
Some configuration options are specific to the Hadoop distribution, (e.g. Apache, Hortonworks, MapR, etc). These options are immutable and must not change throughout the charm deployment lifecycle.
Helper methods are provided for keys that require action. Presently, this includes adding/removing directories, dependent packages, and groups/users. Other required keys may be listed when instantiating this class, but this will only validate these keys exist in the yaml; it will not provide any helper functionality for unkown keys.
Parameters: - filename (str) – File to process (default dist.yaml)
- required_keys (list) – A list of keys required to be present in the yaml
- Example dist.yaml with supported keys:
vendor: ‘<name>’ hadoop_version: ‘<version>’ packages:
- ‘<package 1>’
- ‘<package 2>’
- groups:
- ‘<name>’
- users:
- <user 1>:
- groups: [‘<primary>’, ‘<group>’, ‘<group>’]
- <user 2>:
- groups: [‘<primary>’]
- dirs:
- <dir 1>:
- path: ‘</path/to/dir>’ perms: 0777
- <dir 2>:
- path: ‘{config[<option>]}’ # value comes from config option owner: ‘<user>’ group: ‘<group>’ perms: 0755
- ports:
- <name1>:
- port: <port> exposed_on: <service> # optional
- <name2>:
- port: <port> exposed_on: <service> # optional
- add_dirs()¶
- add_packages()¶
- add_users()¶
- exposed_ports(service)¶
- path(key)¶
- port(key)¶
- remove_dirs()¶
- remove_packages()¶
- remove_users()¶
- exception charmhelpers.contrib.bigdata.utils.TimeoutError¶
Bases: exceptions.Exception
- charmhelpers.contrib.bigdata.utils.disable_firewall(*args, **kwds)¶
Temporarily disable the firewall, via ufw.
- charmhelpers.contrib.bigdata.utils.environment_edit_in_place(*args, **kwds)¶
Edit the /etc/environment file in-place.
There is no standard definition for the format of /etc/environment, but the convention, which this helper supports, is simple key-value pairs, separated by =, with optionally quoted values.
Note that this helper will implicitly quote all values.
Also note that the file is not locked during the edits.
- charmhelpers.contrib.bigdata.utils.jps(name)¶
Get PIDs for named Java processes, for any user.
- charmhelpers.contrib.bigdata.utils.normalize_strbool(value)¶
- charmhelpers.contrib.bigdata.utils.re_edit_in_place(filename, subs)¶
Perform a set of in-place edits to a file.
Parameters: - filename (str) – Name of file to edit
- subs (dict) – Mapping of patterns to replacement strings
- charmhelpers.contrib.bigdata.utils.read_etc_env()¶
Read /etc/environment and return it as a dict.
- charmhelpers.contrib.bigdata.utils.run_as(user, command, *args, **kwargs)¶
Run a command as a particular user, using /etc/environment and optionally capturing and returning the output.
Raises subprocess.CalledProcessError if command fails.
Parameters: - user (str) – Username to run command as
- command (str) – Command to run
- args (list) – Additional args to pass to command
- env (dict) – Additional env variables (will be merged with /etc/environment)
- capture_output (bool) – Capture and return output (default: False)
- class charmhelpers.contrib.bigdata.utils.verify_resources(*which)¶
Bases: object
Predicate for specific named resources, with useful rendering in the logs.
Parameters: *which (str) – One or more resource names to fetch & verify. Defaults to all non-optional resources.
- charmhelpers.contrib.bigdata.utils.wait_for_hdfs(timeout)¶
- charmhelpers.contrib.bigdata.utils.xmlpropmap_edit_in_place(*args, **kwds)¶
Edit an XML property map (configuration) file in-place.
This helper acts as a context manager which edits an XML file of the form:
- <configuration>
- <property>
- <name>property-name</name> <value>property-value</value> <description>Optional property description</description>
</property> ...
</configuration>
This context manager yields a dict containing the existing name/value mappings. Properties can then be modified, added, or removed, and the changes will be reflected in the file.
Example usage:
- with xmlpropmap_edit_in_place(‘my.xml’) as props:
- props[‘foo’] = ‘bar’ del props[‘removed’]
Note that the file is not locked during the edits.