Folder

Checks the contents of a folder for size, age and count. Folder based checks are useful in a number of scenarios:

• Verifying that backups have been uploaded and are the appropriate size
• Checking that logs or other temporary files are being cleaned up
• For batch processes:
  • Checking if files are being processed (and/or produced)
  • Checking the size of queue processing backlog
  • Checking if any error (.err or .log) files have been produced.

folder-check.yaml

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: folder-check
spec:
  interval: 30
  folder:
    - path: /etc/
      name: folder-check-min
      description: Checks if there are at least 10 files in the folder
      minCount: 10

Field	Description	Scheme	Required
path	A local folder path or a remote folder smb://, sftp://, s3:// or gcs://	string	Yes
sftpConnection	Connection details	SFTPConnection	Yes
gcpConnection	Connection details for GCP	GCPConnection
awsConnection	AWS Access credentials	AWSConnection
smbConnection	SMB connection details	SMBConnection
filter	Filter objects out before checking if they are valid	FolderFilter
minCount	The minimum number of files inside the path	int
maxCount	The maximum number of files inside the path, can be used in conjunction with filter.regex to detect error files	int
minAge	The youngest age a file can be	Duration
maxAge	The oldest age a file can be, often used to check for unprocessed files or files that have not been cleaned up	Duration
minSize	The minimum file size, can be used to detect backups that did not upload successfully	Size
maxSize	The maximim file size	Size
name	Name of the check, must be unique within the canary	string	Yes
description	Description for the check	string
icon	Icon for overwriting default icon on the dashboard	string
labels	Labels for check	map[string]string
test	Evaluate whether a check is healthy	Expression
display	Expression to change the formatting of the display	Expression
transform	Transform data from a check into multiple individual checks	Expression
metrics	Metrics to export from	[]Metrics

Duration

Durations are strings with an optional fraction and unit e.g. 300ms, 1.5h or 2h45m. Valid time units are ms, s, m, h.

Size

Sizes are string with a unit suffix e.g. 100 / 100b, 10mb, Valid size units are kb, mb, gb, tb

FolderFilter

Field	Description	Scheme	Required
maxAge	MaxAge the latest object should be younger than defined age	Duration
maxSize	MaxSize of the files inside the searchPath	Size
minAge	MinAge the latest object should be older than defined age	Duration
minSize	MinSize of the files inside the searchPath	Size
regex	Filter files based on regular expression	regex

e.g. to verify that database backups are being performed

postgres-backup-check.yaml

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: folder-check
spec:
  interval: 30
  folder:
    - path: /data/backups
      filter:
        regex: "pg-backups-.*.zip"
      maxAge: 1d # require a daily backup
      minSize: 10mb # the backup should be at least 10mb

Result Variables

The following fields are available in test, display and transform expressions

Field	Scheme
Oldest	os.FileInfo
Newest	os.FileInfo
MinSize	os.FileInfo
MaxSize	os.FileInfo
SupportsTotalSize (Only true for SMB folders)	bool
SupportsAvailableSize (Only true for SMB folders)	bool
TotalSize	int64
AvailableSize	int64
Files	[]os.FileInfo

FolderFilter

Field	Description	Scheme	Required
minAge		Duration
maxAge		Duration
minSize		Size
maxSize		Size
regex		string

Connection Types

SFTP

sftp-check.yaml

# ...
kind: Canary
spec:
  folder:
  - path: folder
    sftpConnection:
      host: 192.168.1.9
      # ...

Field	Description	Scheme
connection	Path of existing connection e.g. connection://sftp/instance Mutually exclusive with username	Connection
username	utually exclusive with connection	EnvVar
password	Mutually exclusive with connection	EnvVar
host	Custom AWS Cloudwatch endpoint	string
port	Default to 22	int

S3

s3-check.yaml

        # ...
kind: Canary
spec:
  folder:
    - path: s3://bucket/folder
      awsConnection:
        # ...

SMB

smb-check.yaml

# ...
kind: Canary
spec:
  folder:
    - path: smb:\\192.168.1.9\Some Public Folder\somedir
      smbConnection:
           #...

Field	Description	Scheme
connection	Path of existing connection e.g. connection://windows/svc-account Mutually exclusive withusername and password	Connections
username	Mutually exclusive with connection	EnvVar
password	Mutually exclusive with connection	EnvVar
domain	Windows domain name	string
port	Default to 445	int

GCS

gcs-check.yaml

# ...
kind: Canary
spec:
  folder:
  - path: gcs://bucket/folder
    gcpConnection:
      # ...