Creating Retention Policies¶
Lavatory policies are implemented as Python plugins. Each policy is a .py
file named
after an Artifactory repository.
Each plugin represents one repository. The file name should match the repository name,
replacing -
with _
.
For example, the repository yum-local
should have a retention policy named yum_local.py
Anatomy of a Policy¶
Each policy needs to provide one function, purgelist()
.
This function takes one argument, artifactory
, which is an instance of the
lavatory.utils.artifactory.Artifactory
class. This argument handles all communication
with artifactory.
This function needs to return a list of artifacts to delete.
Docstring Description¶
The docstring following the function definition will be used as the policy description. This gets used in logging, as well as generating a list of all active policies.
Return Value¶
The return value of the policy should be a list of artifacts to delete.
The artifacts are a dictionary that at minimum needs a path
and name
key.
These keys are used by the delete function to remove the artifact.
path
: path to artifact in the repository
name
: Name of the artifact
Example Minimal Return:
[{ 'path': '222', 'name': 'Application-10.6.0-10.6.0.07-9cd3c33.iso'}]
This will delete artifact <repo_name>/222/Application-10.6.0-10.6.0.07-9cd3c33.iso
Policy Helpers¶
Below are some helper functions to assist in writing policies. These include easy ways to do time-based retention, count-based retention, or searching with AQL.
Time Based Retention¶
This policy will purge any artifact in the repository older than 120 days.
def purgelist(artifactory):
"""Policy to purge all artifacts older than 120 days"""
purgable = artifactory.time_based_retention(keep_days=120)
return purgable
-
Artifactory.
time_based_retention
(keep_days=None, time_field='created', item_type='file', extra_aql=None)[source]¶ Retains artifacts based on number of days since creation.
extra_aql example: [{“@deployed”: {“$match”: “dev”}}, {“@deployed”: {“$nmatch”: “prod”}}] This would search for artifacts that were created after <keep_days> with property “deployed” equal to dev and not equal to prod.Parameters: - keep_days (int) – Number of days to keep an artifact.
- time_field (str) – The field of time to look at (created, modified, stat.downloaded).
- item_type (str) – The item type to search for (file/folder/any).
- extra_aql (list) –
Returns: List of artifacts matching retention policy
Return type: list
Count Based Retention¶
This policy will retain the last 5 artifacts of each project in a repository.
def purgelist(artifactory):
"""Policy to keep just the 5 most recent artifacts."""
purgable = artifactory.count_based_retention(retention_count=5)
return purgable
-
Artifactory.
count_based_retention
(retention_count=None, project_depth=2, artifact_depth=3, item_type='folder', extra_aql=None)[source]¶ Return all artifacts except the <count> most recent.
Parameters: - retention_count (int) – Number of artifacts to keep.
- project_depth (int) – how far down the Artifactory folder hierarchy to look for projects.
- artifact_depth (int) – how far down the Artifactory folder hierarchy to look for specific artifacts.
- item_type (str) – The item type to search for (file/folder/any).
- extra_aql (list) –
Returns: List of all artifacts to delete.
Return type: list
AQL Filtering¶
You can also use AQL to search for artifacts if you need more control than the count-based retention or time-based retention helps.
def purgelist(artifactory):
"""Policy to purge artifacts with deployed property of dev and not prod."""
aql_terms = [{"@deployed": {"$match": "dev"}}, {"@deployed": {"$nmatch": "prod"}}]
extra_fields = ['property.*']
purgable = artifactory.filter(terms=aql_terms, fields=extra_fields, depth=None, item_type="any")
return purgable
All of the terms in aql_terms
will be ANDed
together and searched.
The above policy would use the below full AQL to search for artifacts.
items.find({"$and": [{"@deployed": {"$match": "dev"}},
{"@deployed": {"$nmatch": "prod"}}, {"path": {"$nmatch": "*/repodata"}},
{"repo": {"$eq": "yum-local"}}, {"type": {"$eq": "any"}}]}).include("stat", "property.*")
-
Artifactory.
filter
(terms=None, depth=3, sort=None, offset=0, limit=0, fields=None, item_type='folder')[source]¶ Get a subset of artifacts from the specified repo. This looks at the project level, but actually need to iterate lower at project level
This method does not use pagination. It assumes that this utility will be called on a repo sufficiently frequently that removing just the default n items will be enough.
Parameters: - terms (list) – an array of jql snippets that will be ANDed together
- depth (int, optional) – how far down the folder hierarchy to look
- fields (list) – Fields
- sort (dict) – How to sort Artifactory results
- offset (int) – how many items from the beginning of the list should be skipped (optional)
- limit (int) – the maximum number of entries to return (optional)
- item_type (str) – The item type to search for (file/folder/any).
Returns: List of artifacts returned from query
Return type: list