mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-11-21 14:44:00 +00:00
added packer build for DigitalOcean Droplets (#1917)
* added packer build for DigitalOcean Droplets * fixed typo * added packer RELEASE_GUIDE.md, Makefile * Apply suggestions from code review Co-authored-by: Roman Khavronenko <hagen1778@gmail.com> * added corrections amd improvements * added packer link & templating for sed version * fixed typo Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
This commit is contained in:
Denys Holius
GitHub
parent
ee17516afd
commit
d44cc14c6b
36 changed files with 2245 additions and 0 deletions
|
|
@ -4,3 +4,4 @@ gocache-for-docker
|
|||
victoria-metrics-data
|
||||
vmstorage-data
|
||||
vmselect-cache
|
||||
.vscode
|
||||
|
|
|
|||
1
.gitignore
vendored
1
.gitignore
vendored
|
|
@ -4,6 +4,7 @@
|
|||
*.pprof
|
||||
/bin
|
||||
.idea
|
||||
.vscode
|
||||
*.test
|
||||
*.swp
|
||||
/gocache-for-docker
|
||||
|
|
|
|||
2
deployment/marketplace/digitialocean/one-click-droplet-default/.gitignore
vendored
Normal file
2
deployment/marketplace/digitialocean/one-click-droplet-default/.gitignore
vendored
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
.vscode/*
|
||||
.vscode
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
RELEASE_NAME := vm-oneclick-droplet
|
||||
VM_VERSION ?= $(shell git describe --abbrev=0 --tags)
|
||||
|
||||
.PHONY: $(MAKECMDGOALS)
|
||||
|
||||
release-victoria-metrics-digitalocean-oneclick-droplet:
|
||||
sed -i -e "s/VM_VERSION/${VM_VERSION}/g" ./files/etc/update-motd.d/99-one-click
|
||||
packer build template.pkr.hcl
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
## Application summary
|
||||
|
||||
VictoriaMetrics is a fast and scalable open source time series database and monitoring solution.
|
||||
|
||||
## Description
|
||||
|
||||
VictoriaMetrics is a free [open source time series database](https://en.wikipedia.org/wiki/Time_series_database) (TSDB) and monitoring solution, designed to collect, store and process real-time metrics.
|
||||
|
||||
It supports the [Prometheus](https://en.wikipedia.org/wiki/Prometheus_(software)) pull model and various push protocols ([Graphite](https://en.wikipedia.org/wiki/Graphite_(software)), [InfluxDB](https://en.wikipedia.org/wiki/InfluxDB), OpenTSDB) for data ingestion. It is optimized for storage with high-latency IO, low IOPS and time series with [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate).
|
||||
|
||||
For reading the data and evaluating alerting rules, VictoriaMetrics supports the PromQL, [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html) and Graphite query languages. VictoriaMetrics Single is fully autonomous and can be used as a long-term storage for time series.
|
||||
|
||||
[VictoriaMetrics Single](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html) = Hassle-free monitoring solution. Easily handles 10M+ of active time series on a single instance. Perfect for small and medium environments.
|
||||
|
||||
## Getting started after deploying VictoriaMetrics Single
|
||||
|
||||
### Config
|
||||
|
||||
VictoriaMetrics configuration is located at `/etc/victoriametrics/single/scrape.yml` on the droplet.
|
||||
This One Click app uses 8428, 2003, 4242 and 8089 ports to accept metrics from different protocols. It's recommended to disable ports for protocols which are not needed. [Ubuntu firewall](https://help.ubuntu.com/community/UFW) can be used to easily disable access for specific ports.
|
||||
|
||||
### Scraping metrics
|
||||
|
||||
VictoriaMetrics supports metrics scraping in the same way as Prometheus does. Check the configuration file to edit scraping targets. See more details about scraping at [How to scrape Prometheus exporters](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-scrape-prometheus-exporters-such-as-node-exporter).
|
||||
|
||||
### Sending metrics
|
||||
|
||||
Besides scraping, VictoriaMetrics accepts write requests for various ingestion protocols. This One Click app supports the following protocols:
|
||||
- [Datadog](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-send-data-from-datadog-agent), [Influx (telegraph)](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-send-data-from-influxdb-compatible-agents-such-as-telegraf), [JSON](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-import-data-in-json-line-format), [CSV](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-import-csv-data), [Prometheus](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-import-data-in-prometheus-exposition-format) on port :8428
|
||||
- [Graphite (statsd)](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-send-data-from-graphite-compatible-agents-such-as-statsd) on port :2003 tcp/udp
|
||||
- [OpenTSDB](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-send-data-from-opentsdb-compatible-agents) on port :4242
|
||||
- Influx (telegraph) on port :8089 tcp/udp
|
||||
|
||||
See more details and examples in [official documentation](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html).
|
||||
|
||||
### UI
|
||||
|
||||
VictoriaMetrics provides a [User Interface (UI)](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui) for query troubleshooting and exploration. The UI is available at `http://your_droplet_public_ipv4:8428/vmui`. It lets users explore query results via graphs and tables.
|
||||
|
||||
To check it, open the following in your browser `http://your_droplet_public_ipv4:8428/vmui` and then enter `vm_app_uptime_seconds` to the Query Field to Execute the Query.
|
||||
|
||||
Run the following command to query and retrieve a result from VictoriaMetrics Single with `curl`:
|
||||
|
||||
```bash
|
||||
curl -sg http://your_droplet_public_ipv4:8428/api/v1/query_range?query=vm_app_uptime_seconds | jq
|
||||
```
|
||||
|
||||
### Accessing
|
||||
|
||||
Once the Droplet is created, you can use DigitalOcean's web console to start a session or SSH directly to the server as root:
|
||||
|
||||
```bash
|
||||
ssh root@your_droplet_public_ipv4
|
||||
```
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
## Release guide for DigitalOcean 1-ClickApp Droplet
|
||||
|
||||
### Build image
|
||||
|
||||
To build the snapshot in DigitalOcean account you will need API Token.
|
||||
API Token can be generated on [https://cloud.digitalocean.com/account/api/tokens](https://cloud.digitalocean.com/account/api/tokens) or use already generated from OnePassword.
|
||||
|
||||
Set variable `DIGITALOCEAN_API_TOKEN` for environment:
|
||||
|
||||
```bash
|
||||
export DIGITALOCEAN_API_TOKEN="your_token_here"
|
||||
```
|
||||
or set it by with make:
|
||||
|
||||
```bash
|
||||
make release-victoria-metrics-digitalocean-oneclick-droplet DIGITALOCEAN_API_TOKEN="your_token_here"
|
||||
```
|
||||
|
||||
### Update information on Vendor Portal
|
||||
|
||||
After packer build finished you need to update a product page.
|
||||
|
||||
1. Go to [https://cloud.digitalocean.com/vendorportal](https://cloud.digitalocean.com/vendorportal).
|
||||
2. Choose a product that you need to update.
|
||||
3. Enter newer information for this release and choose a droplet's snapshot which was builded recently.
|
||||
4. Submit updates for approve on DigitalOcean Marketplace.
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
[Unit]
|
||||
Description=VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
|
||||
# https://docs.victoriametrics.com
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=victoriametrics
|
||||
Group=victoriametrics
|
||||
WorkingDirectory=/var/lib/victoria-metrics-data
|
||||
StartLimitBurst=5
|
||||
StartLimitInterval=0
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
EnvironmentFile=-/etc/victoriametrics/single/victoriametrics.conf
|
||||
ExecStart=/usr/bin/victoria-metrics-prod $ARGS
|
||||
ExecStop=/bin/kill -s SIGTERM $MAINPID
|
||||
ExecReload=/bin/kill -HUP $MAINPID
|
||||
# See docs https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#tuning
|
||||
ProtectSystem=full
|
||||
LimitNOFILE=1048576
|
||||
LimitNPROC=1048576
|
||||
LimitCORE=infinity
|
||||
StandardOutput=syslog
|
||||
StandardError=syslog
|
||||
SyslogIdentifier=vmsingle
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
#!/bin/sh
|
||||
#
|
||||
# Configured as part of the DigitalOcean 1-Click Image build process
|
||||
|
||||
myip=$(hostname -I | awk '{print$1}')
|
||||
cat <<EOF
|
||||
********************************************************************************
|
||||
|
||||
Welcome to DigitalOcean's 1-Click VictoriaMetrics Droplet.
|
||||
To keep this Droplet secure, the UFW firewall is enabled.
|
||||
All ports are BLOCKED except 22 (SSH), 80 (HTTP), and 443 (HTTPS), 8428 (VictoriaMetrics HTTP), 8089 (VictoriaMetrics Influx),
|
||||
4242 (VictoriaMetrics OpenTSDB), 2003 (VictoriaMetrics Graphite)
|
||||
|
||||
In a web browser, you can view:
|
||||
* The VictoriaMetrics 1-Click Quickstart guide: https://kutt.it/1click-quickstart
|
||||
|
||||
On the server:
|
||||
* The default VictoriaMetrics root is located at /var/lib/victoria-metrics-data
|
||||
* VictoriaMetrics is running on ports: 8428, 8089, 4242, 2003 and they are bound to the local interface.
|
||||
|
||||
********************************************************************************
|
||||
# This image includes version VM_VERSION of VictoriaMetrics.
|
||||
# See Release notes https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/VM_VERSION
|
||||
|
||||
# Welcome to VictoriaMetrics droplet!
|
||||
|
||||
# Website: https://victoriametrics.com
|
||||
# Documentation: https://docs.victoriametrics.com
|
||||
# VictoriaMetrics Github : https://github.com/VictoriaMetrics/VictoriaMetrics
|
||||
# VictoriaMetrics Slack Community: https://slack.victoriametrics.com
|
||||
# VictoriaMetrics Telegram Community: https://t.me/VictoriaMetrics_en
|
||||
|
||||
# VictoriaMetrics config: /etc/victoriametrics/single/victoriametrics.conf
|
||||
|
||||
********************************************************************************
|
||||
EOF
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
# Scrape config example
|
||||
#
|
||||
scrape_configs:
|
||||
- job_name: self_scrape
|
||||
scrape_interval: 10s
|
||||
static_configs:
|
||||
- targets: ['127.0.0.1:8428']
|
||||
|
|
@ -0,0 +1 @@
|
|||
ARGS="-promscrape.config=/etc/victoriametrics/single/scrape.yml -storageDataPath=/var/lib/victoria-metrics-data -retentionPeriod=12 -httpListenAddr=:8428 -graphiteListenAddr=:2003 -opentsdbListenAddr=:4242 -influxListenAddr=:8089 -enableTCP6"
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
#!/bin/sh
|
||||
|
||||
systemctl start vmsingle.service
|
||||
|
||||
# Remove the ssh force logout command
|
||||
sed -e '/Match User root/d' \
|
||||
-e '/.*ForceCommand.*droplet.*/d' \
|
||||
-i /etc/ssh/sshd_config
|
||||
|
||||
systemctl restart ssh
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Create victoriametrics user
|
||||
groupadd -r victoriametrics
|
||||
useradd -g victoriametrics -d /var/lib/victoria-metrics-data -s /sbin/nologin --system victoriametrics
|
||||
|
||||
mkdir -p /var/lib/victoria-metrics-data
|
||||
chown -R victoriametrics:victoriametrics /var/lib/victoria-metrics-data
|
||||
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
apt update
|
||||
DEBIAN_FRONTEND=noninteractive apt -y full-upgrade
|
||||
DEBIAN_FRONTEND=noninteractive apt -y install curl git wget software-properties-common
|
||||
rm -rf /var/log/kern.log
|
||||
rm -rf /var/log/ufw.log
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
#!/bin/sh
|
||||
|
||||
sed -e 's|DEFAULT_FORWARD_POLICY=.*|DEFAULT_FORWARD_POLICY="ACCEPT"|g' \
|
||||
-i /etc/default/ufw
|
||||
|
||||
ufw allow ssh comment "SSH port"
|
||||
ufw allow http comment "HTTP port"
|
||||
ufw allow https comment "HTTPS port"
|
||||
ufw allow 8428 comment "VictoriaMetrics Single HTTP port"
|
||||
ufw allow 8089/tcp comment "TCP Influx Listen port for VictoriaMetrics"
|
||||
ufw allow 8089/udp comment "UDP Influx Listen port for VictoriaMetrics"
|
||||
ufw allow 2003/tcp comment "TCP Graphite Listen port for VictoriaMetrics"
|
||||
ufw allow 2003/udp comment "UDP Graphite Listen port for VictoriaMetrics"
|
||||
ufw allow 4242 comment "OpenTSDB Listen port for VictoriaMetrics"
|
||||
|
||||
ufw --force enable
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Wait for cloud-init
|
||||
cloud-init status --wait
|
||||
|
||||
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/${VM_VER}/victoria-metrics-amd64-${VM_VER}.tar.gz -O /tmp/victoria-metrics.tar.gz
|
||||
tar xvf /tmp/victoria-metrics.tar.gz -C /usr/bin
|
||||
chmod +x /usr/bin/victoria-metrics-prod
|
||||
chown root:root /usr/bin/victoria-metrics-prod
|
||||
|
||||
|
||||
# Enable VictoriaMetrics on boot
|
||||
systemctl enable vmsingle.service
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
#!/bin/bash
|
||||
#
|
||||
# Remove log files generated during instance creation and
|
||||
# configuration.
|
||||
|
||||
rm -rf /var/log/kern.log
|
||||
rm -rf /var/log/ufw.log
|
||||
|
|
@ -0,0 +1,44 @@
|
|||
#!/bin/bash
|
||||
|
||||
# Ensure /tmp exists and has the proper permissions before
|
||||
# checking for security updates
|
||||
# https://github.com/digitalocean/marketplace-partners/issues/94
|
||||
if [[ ! -d /tmp ]]; then
|
||||
mkdir /tmp
|
||||
fi
|
||||
chmod 1777 /tmp
|
||||
|
||||
apt-get -y update
|
||||
apt-get -y upgrade
|
||||
rm -rf /tmp/* /var/tmp/*
|
||||
history -c
|
||||
cat /dev/null > /root/.bash_history
|
||||
unset HISTFILE
|
||||
apt-get -y autoremove
|
||||
apt-get -y autoclean
|
||||
find /var/log -mtime -1 -type f -exec truncate -s 0 {} \;
|
||||
rm -rf /var/log/*.gz /var/log/*.[0-9] /var/log/*-????????
|
||||
rm -rf /var/lib/cloud/instances/*
|
||||
rm -f /root/.ssh/authorized_keys /etc/ssh/*key*
|
||||
touch /etc/ssh/revoked_keys
|
||||
chmod 600 /etc/ssh/revoked_keys
|
||||
|
||||
# Securely erase the unused portion of the filesystem
|
||||
GREEN='\033[0;32m'
|
||||
NC='\033[0m'
|
||||
printf "\n${GREEN}Writing zeros to the remaining disk space to securely
|
||||
erase the unused portion of the file system.
|
||||
Depending on your disk size this may take several minutes.
|
||||
The secure erase will complete successfully when you see:${NC}
|
||||
dd: writing to '/zerofile': No space left on device\n
|
||||
Beginning secure erase now\n"
|
||||
|
||||
dd if=/dev/zero of=/zerofile &
|
||||
PID=$!
|
||||
while [ -d /proc/$PID ]
|
||||
do
|
||||
printf "."
|
||||
sleep 5
|
||||
done
|
||||
sync; rm /zerofile; sync
|
||||
cat /dev/null > /var/log/lastlog; cat /dev/null > /var/log/wtmp
|
||||
|
|
@ -0,0 +1,682 @@
|
|||
#!/bin/bash
|
||||
|
||||
# DigitalOcean Marketplace Image Validation Tool
|
||||
# © 2021 DigitalOcean LLC.
|
||||
# This code is licensed under Apache 2.0 license (see LICENSE.md for details)
|
||||
|
||||
VERSION="v. 1.6"
|
||||
RUNDATE=$( date )
|
||||
|
||||
# Script should be run with SUDO
|
||||
if [ "$EUID" -ne 0 ]
|
||||
then echo "[Error] - This script must be run with sudo or as the root user."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
STATUS=0
|
||||
PASS=0
|
||||
WARN=0
|
||||
FAIL=0
|
||||
|
||||
# $1 == command to check for
|
||||
# returns: 0 == true, 1 == false
|
||||
cmdExists() {
|
||||
if command -v "$1" > /dev/null 2>&1; then
|
||||
return 0
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
function getDistro {
|
||||
if [ -f /etc/os-release ]; then
|
||||
# freedesktop.org and systemd
|
||||
. /etc/os-release
|
||||
OS=$NAME
|
||||
VER=$VERSION_ID
|
||||
elif type lsb_release >/dev/null 2>&1; then
|
||||
# linuxbase.org
|
||||
OS=$(lsb_release -si)
|
||||
VER=$(lsb_release -sr)
|
||||
elif [ -f /etc/lsb-release ]; then
|
||||
# For some versions of Debian/Ubuntu without lsb_release command
|
||||
. /etc/lsb-release
|
||||
OS=$DISTRIB_ID
|
||||
VER=$DISTRIB_RELEASE
|
||||
elif [ -f /etc/debian_version ]; then
|
||||
# Older Debian/Ubuntu/etc.
|
||||
OS=Debian
|
||||
VER=$(cat /etc/debian_version)
|
||||
elif [ -f /etc/SuSe-release ]; then
|
||||
# Older SuSE/etc.
|
||||
:
|
||||
elif [ -f /etc/redhat-release ]; then
|
||||
# Older Red Hat, CentOS, etc.
|
||||
VER=$( cat /etc/redhat-release | cut -d" " -f3 | cut -d "." -f1)
|
||||
d=$( cat /etc/redhat-release | cut -d" " -f1 | cut -d "." -f1)
|
||||
if [[ $d == "CentOS" ]]; then
|
||||
OS="CentOS Linux"
|
||||
fi
|
||||
else
|
||||
# Fall back to uname, e.g. "Linux <version>", also works for BSD, etc.
|
||||
OS=$(uname -s)
|
||||
VER=$(uname -r)
|
||||
fi
|
||||
}
|
||||
function loadPasswords {
|
||||
SHADOW=$(cat /etc/shadow)
|
||||
}
|
||||
|
||||
function checkAgent {
|
||||
# Check for the presence of the do-agent in the filesystem
|
||||
if [ -d /var/opt/digitalocean/do-agent ];then
|
||||
echo -en "\e[41m[FAIL]\e[0m DigitalOcean Monitoring Agent detected.\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
if [[ $OS == "CentOS Linux" ]]; then
|
||||
echo "The agent can be removed with 'sudo yum remove do-agent' "
|
||||
elif [[ $OS == "Ubuntu" ]]; then
|
||||
echo "The agent can be removed with 'sudo apt-get purge do-agent' "
|
||||
fi
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m DigitalOcean Monitoring agent was not found\n"
|
||||
((PASS++))
|
||||
fi
|
||||
}
|
||||
|
||||
function checkLogs {
|
||||
cp_ignore="/var/log/cpanel-install.log"
|
||||
echo -en "\nChecking for log files in /var/log\n\n"
|
||||
# Check if there are log archives or log files that have not been recently cleared.
|
||||
for f in /var/log/*-????????; do
|
||||
[[ -e $f ]] || break
|
||||
if [ $f != $cp_ignore ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m Log archive ${f} found\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
done
|
||||
for f in /var/log/*.[0-9];do
|
||||
[[ -e $f ]] || break
|
||||
echo -en "\e[93m[WARN]\e[0m Log archive ${f} found\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
done
|
||||
for f in /var/log/*.log; do
|
||||
[[ -e $f ]] || break
|
||||
if [[ "${f}" = '/var/log/lfd.log' && "$( cat "${f}" | egrep -v '/var/log/messages has been reset| Watching /var/log/messages' | wc -c)" -gt 50 ]]; then
|
||||
if [ $f != $cp_ignore ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m un-cleared log file, ${f} found\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
elif [[ "${f}" != '/var/log/lfd.log' && "$( cat "${f}" | wc -c)" -gt 50 ]]; then
|
||||
if [ $f != $cp_ignore ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m un-cleared log file, ${f} found\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
}
|
||||
function checkTMP {
|
||||
# Check the /tmp directory to ensure it is empty. Warn on any files found.
|
||||
return 1
|
||||
}
|
||||
function checkRoot {
|
||||
user="root"
|
||||
uhome="/root"
|
||||
for usr in $SHADOW
|
||||
do
|
||||
IFS=':' read -r -a u <<< "$usr"
|
||||
if [[ "${u[0]}" == "${user}" ]]; then
|
||||
if [[ ${u[1]} == "!" ]] || [[ ${u[1]} == "!!" ]] || [[ ${u[1]} == "*" ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m User ${user} has no password set.\n"
|
||||
((PASS++))
|
||||
else
|
||||
echo -en "\e[41m[FAIL]\e[0m User ${user} has a password set on their account.\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
fi
|
||||
done
|
||||
if [ -d ${uhome}/ ]; then
|
||||
if [ -d ${uhome}/.ssh/ ]; then
|
||||
if ls ${uhome}/.ssh/*> /dev/null 2>&1; then
|
||||
for key in ${uhome}/.ssh/*
|
||||
do
|
||||
if [ "${key}" == "${uhome}/.ssh/authorized_keys" ]; then
|
||||
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 50 ]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m User \e[1m${user}\e[0m has a populated authorized_keys file in \e[93m${key}\e[0m\n"
|
||||
akey=$(cat ${key})
|
||||
echo "File Contents:"
|
||||
echo $akey
|
||||
echo "--------------"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
elif [ "${key}" == "${uhome}/.ssh/id_rsa" ]; then
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 0 ]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m User \e[1m${user}\e[0m has a private key file in \e[93m${key}\e[0m\n"
|
||||
akey=$(cat ${key})
|
||||
echo "File Contents:"
|
||||
echo $akey
|
||||
echo "--------------"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has empty private key file in \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
elif [ "${key}" != "${uhome}/.ssh/known_hosts" ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has a file in their .ssh directory at \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
else
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 50 ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has a populated known_hosts file in \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m has no SSH keys present\n"
|
||||
fi
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m does not have an .ssh directory\n"
|
||||
fi
|
||||
if [ -f /root/.bash_history ];then
|
||||
|
||||
BH_S=$( cat /root/.bash_history | wc -c)
|
||||
|
||||
if [[ $BH_S -lt 200 ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m ${user}'s Bash History appears to have been cleared\n"
|
||||
((PASS++))
|
||||
else
|
||||
echo -en "\e[41m[FAIL]\e[0m ${user}'s Bash History should be cleared to prevent sensitive information from leaking\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
|
||||
return 1;
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m The Root User's Bash History is not present\n"
|
||||
((PASS++))
|
||||
fi
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m does not have a directory in /home\n"
|
||||
fi
|
||||
echo -en "\n\n"
|
||||
return 1
|
||||
}
|
||||
|
||||
function checkUsers {
|
||||
# Check each user-created account
|
||||
for user in $(awk -F: '$3 >= 1000 && $1 != "nobody" {print $1}' /etc/passwd;)
|
||||
do
|
||||
# Skip some other non-user system accounts
|
||||
if [[ $user == "centos" ]]; then
|
||||
:
|
||||
elif [[ $user == "nfsnobody" ]]; then
|
||||
:
|
||||
else
|
||||
echo -en "\nChecking user: ${user}...\n"
|
||||
for usr in $SHADOW
|
||||
do
|
||||
IFS=':' read -r -a u <<< "$usr"
|
||||
if [[ "${u[0]}" == "${user}" ]]; then
|
||||
if [[ ${u[1]} == "!" ]] || [[ ${u[1]} == "!!" ]] || [[ ${u[1]} == "*" ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m User ${user} has no password set.\n"
|
||||
((PASS++))
|
||||
else
|
||||
echo -en "\e[41m[FAIL]\e[0m User ${user} has a password set on their account. Only system users are allowed on the image.\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
fi
|
||||
done
|
||||
#echo "User Found: ${user}"
|
||||
uhome="/home/${user}"
|
||||
if [ -d "${uhome}/" ]; then
|
||||
if [ -d "${uhome}/.ssh/" ]; then
|
||||
if ls "${uhome}/.ssh/*"> /dev/null 2>&1; then
|
||||
for key in ${uhome}/.ssh/*
|
||||
do
|
||||
if [ "${key}" == "${uhome}/.ssh/authorized_keys" ]; then
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 50 ]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m User \e[1m${user}\e[0m has a populated authorized_keys file in \e[93m${key}\e[0m\n"
|
||||
akey=$(cat ${key})
|
||||
echo "File Contents:"
|
||||
echo $akey
|
||||
echo "--------------"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
elif [ "${key}" == "${uhome}/.ssh/id_rsa" ]; then
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 0 ]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m User \e[1m${user}\e[0m has a private key file in \e[93m${key}\e[0m\n"
|
||||
akey=$(cat ${key})
|
||||
echo "File Contents:"
|
||||
echo $akey
|
||||
echo "--------------"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has empty private key file in \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
elif [ "${key}" != "${uhome}/.ssh/known_hosts" ]; then
|
||||
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has a file in their .ssh directory named \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
|
||||
else
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 50 ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has a known_hosts file in \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
done
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m has no SSH keys present\n"
|
||||
fi
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m does not have an .ssh directory\n"
|
||||
fi
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m does not have a directory in /home\n"
|
||||
fi
|
||||
|
||||
# Check for an uncleared .bash_history for this user
|
||||
if [ -f "${uhome}/.bash_history" ]; then
|
||||
BH_S=$( cat "${uhome}/.bash_history" | wc -c )
|
||||
|
||||
if [[ $BH_S -lt 200 ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m ${user}'s Bash History appears to have been cleared\n"
|
||||
((PASS++))
|
||||
else
|
||||
echo -en "\e[41m[FAIL]\e[0m ${user}'s Bash History should be cleared to prevent sensitive information from leaking\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
|
||||
fi
|
||||
echo -en "\n\n"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
}
|
||||
function checkFirewall {
|
||||
|
||||
if [[ $OS == "Ubuntu" ]]; then
|
||||
fw="ufw"
|
||||
ufwa=$(ufw status |head -1| sed -e "s/^Status:\ //")
|
||||
if [[ $ufwa == "active" ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
elif [[ $OS == "CentOS Linux" ]]; then
|
||||
if [ -f /usr/lib/systemd/system/csf.service ]; then
|
||||
fw="csf"
|
||||
if [[ $(systemctl status $fw >/dev/null 2>&1) ]]; then
|
||||
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
elif cmdExists "firewall-cmd"; then
|
||||
if [[ $(systemctl is-active firewalld >/dev/null 2>&1 && echo 1 || echo 0) ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
else
|
||||
fw="firewalld"
|
||||
if [[ $(systemctl is-active firewalld >/dev/null 2>&1 && echo 1 || echo 0) ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
fi
|
||||
elif [[ "$OS" =~ Debian.* ]]; then
|
||||
# user could be using a number of different services for managing their firewall
|
||||
# we will check some of the most common
|
||||
if cmdExists 'ufw'; then
|
||||
fw="ufw"
|
||||
ufwa=$(ufw status |head -1| sed -e "s/^Status:\ //")
|
||||
if [[ $ufwa == "active" ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
elif cmdExists "firewall-cmd"; then
|
||||
fw="firewalld"
|
||||
if [[ $(systemctl is-active --quiet $fw) ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
else
|
||||
# user could be using vanilla iptables, check if kernel module is loaded
|
||||
fw="iptables"
|
||||
if [[ $(lsmod | grep -q '^ip_tables' 2>/dev/null) ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
}
|
||||
function checkUpdates {
|
||||
if [[ $OS == "Ubuntu" ]] || [[ "$OS" =~ Debian.* ]]; then
|
||||
# Ensure /tmp exists and has the proper permissions before
|
||||
# checking for security updates
|
||||
# https://github.com/digitalocean/marketplace-partners/issues/94
|
||||
if [[ ! -d /tmp ]]; then
|
||||
mkdir /tmp
|
||||
fi
|
||||
chmod 1777 /tmp
|
||||
|
||||
echo -en "\nUpdating apt package database to check for security updates, this may take a minute...\n\n"
|
||||
apt-get -y update > /dev/null
|
||||
|
||||
uc=$(apt-get --just-print upgrade | grep -i "security" | wc -l)
|
||||
if [[ $uc -gt 0 ]]; then
|
||||
update_count=$(( ${uc} / 2 ))
|
||||
else
|
||||
update_count=0
|
||||
fi
|
||||
|
||||
if [[ $update_count -gt 0 ]]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m There are ${update_count} security updates available for this image that have not been installed.\n"
|
||||
echo -en
|
||||
echo -en "Here is a list of the security updates that are not installed:\n"
|
||||
sleep 2
|
||||
apt-get --just-print upgrade | grep -i security | awk '{print $2}' | awk '!seen[$0]++'
|
||||
echo -en
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m There are no pending security updates for this image.\n\n"
|
||||
fi
|
||||
elif [[ $OS == "CentOS Linux" ]]; then
|
||||
echo -en "\nChecking for available security updates, this may take a minute...\n\n"
|
||||
|
||||
update_count=$(yum check-update --security --quiet | wc -l)
|
||||
if [[ $update_count -gt 0 ]]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m There are ${update_count} security updates available for this image that have not been installed.\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m There are no pending security updates for this image.\n"
|
||||
((PASS++))
|
||||
fi
|
||||
else
|
||||
echo "Error encountered"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
return 1;
|
||||
}
|
||||
function checkCloudInit {
|
||||
|
||||
if hash cloud-init 2>/dev/null; then
|
||||
CI="\e[32m[PASS]\e[0m Cloud-init is installed.\n"
|
||||
((PASS++))
|
||||
else
|
||||
CI="\e[41m[FAIL]\e[0m No valid verison of cloud-init was found.\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
function checkMongoDB {
|
||||
# Check if MongoDB is installed
|
||||
# If it is, verify the version is allowed (non-SSPL)
|
||||
|
||||
if [[ $OS == "Ubuntu" ]] || [[ "$OS" =~ Debian.* ]]; then
|
||||
|
||||
if [[ -f "/usr/bin/mongod" ]]; then
|
||||
version=$(/usr/bin/mongod --version --quiet | grep "db version" | sed -e "s/^db\ version\ v//")
|
||||
|
||||
if version_gt $version 4.0.0; then
|
||||
if version_gt $version 4.0.3; then
|
||||
echo -en "\e[41m[FAIL]\e[0m An SSPL version of MongoDB is present, ${version}"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m The version of MongoDB installed, ${version} is not under the SSPL"
|
||||
((PASS++))
|
||||
fi
|
||||
else
|
||||
if version_gt $version 3.6.8; then
|
||||
echo -en "\e[41m[FAIL]\e[0m An SSPL version of MongoDB is present, ${version}"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m The version of MongoDB installed, ${version} is not under the SSPL"
|
||||
((PASS++))
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m MongoDB is not installed"
|
||||
((PASS++))
|
||||
fi
|
||||
|
||||
elif [[ $OS == "CentOS Linux" ]]; then
|
||||
|
||||
if [[ -f "/usr/bin/mongod" ]]; then
|
||||
version=$(/usr/bin/mongod --version --quiet | grep "db version" | sed -e "s/^db\ version\ v//")
|
||||
|
||||
|
||||
if version_gt $version 4.0.0; then
|
||||
if version_gt $version 4.0.3; then
|
||||
echo -en "\e[41m[FAIL]\e[0m An SSPL version of MongoDB is present"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m The version of MongoDB installed is not under the SSPL"
|
||||
((PASS++))
|
||||
fi
|
||||
else
|
||||
if version_gt $version 3.6.8; then
|
||||
echo -en "\e[41m[FAIL]\e[0m An SSPL version of MongoDB is present"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m The version of MongoDB installed is not under the SSPL"
|
||||
((PASS++))
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m MongoDB is not installed"
|
||||
((PASS++))
|
||||
fi
|
||||
|
||||
else
|
||||
echo "ERROR: Unable to identify distribution"
|
||||
((FAIL++))
|
||||
STATUS 2
|
||||
return 1
|
||||
fi
|
||||
|
||||
|
||||
}
|
||||
|
||||
function version_gt() { test "$(printf '%s\n' "$@" | sort -V | head -n 1)" != "$1"; }
|
||||
|
||||
|
||||
clear
|
||||
echo "DigitalOcean Marketplace Image Validation Tool ${VERSION}"
|
||||
echo "Executed on: ${RUNDATE}"
|
||||
echo "Checking local system for Marketplace compatibility..."
|
||||
|
||||
getDistro
|
||||
|
||||
echo -en "\n\e[1mDistribution:\e[0m ${OS}\n"
|
||||
echo -en "\e[1mVersion:\e[0m ${VER}\n\n"
|
||||
|
||||
ost=0
|
||||
osv=0
|
||||
|
||||
if [[ $OS == "Ubuntu" ]]; then
|
||||
ost=1
|
||||
if [[ $VER == "20.04" ]]; then
|
||||
osv=1
|
||||
elif [[ $VER == "18.04" ]]; then
|
||||
osv=1
|
||||
elif [[ $VER == "16.04" ]]; then
|
||||
osv=1
|
||||
else
|
||||
osv=0
|
||||
fi
|
||||
|
||||
elif [[ "$OS" =~ Debian.* ]]; then
|
||||
ost=1
|
||||
case "$VER" in
|
||||
9)
|
||||
osv=1
|
||||
;;
|
||||
10)
|
||||
osv=1
|
||||
;;
|
||||
*)
|
||||
osv=2
|
||||
;;
|
||||
esac
|
||||
|
||||
elif [[ $OS == "CentOS Linux" ]]; then
|
||||
ost=1
|
||||
if [[ $VER == "8" ]]; then
|
||||
osv=1
|
||||
elif [[ $VER == "7" ]]; then
|
||||
osv=1
|
||||
elif [[ $VER == "6" ]]; then
|
||||
osv=1
|
||||
else
|
||||
osv=2
|
||||
fi
|
||||
else
|
||||
ost=0
|
||||
fi
|
||||
|
||||
if [[ $ost == 1 ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m Supported Operating System Detected: ${OS}\n"
|
||||
((PASS++))
|
||||
else
|
||||
echo -en "\e[41m[FAIL]\e[0m ${OS} is not a supported Operating System\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
|
||||
if [[ $osv == 1 ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m Supported Release Detected: ${VER}\n"
|
||||
((PASS++))
|
||||
elif [[ $ost == 1 ]]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m ${OS} ${VER} is not a supported Operating System Version\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo "Exiting..."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
checkCloudInit
|
||||
|
||||
echo -en "${CI}"
|
||||
|
||||
checkFirewall
|
||||
|
||||
echo -en "${FW_VER}"
|
||||
|
||||
checkUpdates
|
||||
|
||||
loadPasswords
|
||||
|
||||
checkLogs
|
||||
|
||||
echo -en "\n\nChecking all user-created accounts...\n"
|
||||
checkUsers
|
||||
|
||||
echo -en "\n\nChecking the root account...\n"
|
||||
checkRoot
|
||||
|
||||
checkAgent
|
||||
|
||||
checkMongoDB
|
||||
|
||||
|
||||
# Summary
|
||||
echo -en "\n\n---------------------------------------------------------------------------------------------------\n"
|
||||
|
||||
if [[ $STATUS == 0 ]]; then
|
||||
echo -en "Scan Complete.\n\e[32mAll Tests Passed!\e[0m\n"
|
||||
elif [[ $STATUS == 1 ]]; then
|
||||
echo -en "Scan Complete. \n\e[93mSome non-critical tests failed. Please review these items.\e[0m\e[0m\n"
|
||||
else
|
||||
echo -en "Scan Complete. \n\e[41mOne or more tests failed. Please review these items and re-test.\e[0m\n"
|
||||
fi
|
||||
echo "---------------------------------------------------------------------------------------------------"
|
||||
echo -en "\e[1m${PASS} Tests PASSED\e[0m\n"
|
||||
echo -en "\e[1m${WARN} WARNINGS\e[0m\n"
|
||||
echo -en "\e[1m${FAIL} Tests FAILED\e[0m\n"
|
||||
echo -en "---------------------------------------------------------------------------------------------------\n"
|
||||
|
||||
if [[ $STATUS == 0 ]]; then
|
||||
echo -en "We did not detect any issues with this image. Please be sure to manually ensure that all software installed on the base system is functional, secure and properly configured (or facilities for configuration on first-boot have been created).\n\n"
|
||||
exit 0
|
||||
elif [[ $STATUS == 1 ]]; then
|
||||
echo -en "Please review all [WARN] items above and ensure they are intended or resolved. If you do not have a specific requirement, we recommend resolving these items before image submission\n\n"
|
||||
exit 0
|
||||
else
|
||||
echo -en "Some critical tests failed. These items must be resolved and this scan re-run before you submit your image to the DigitalOcean Marketplace.\n\n"
|
||||
exit 1
|
||||
fi
|
||||
|
|
@ -0,0 +1,73 @@
|
|||
|
||||
{
|
||||
"variables": {
|
||||
"do_api_token": "{{env `DIGITALOCEAN_API_TOKEN`}}",
|
||||
"image_name": "vm-single-20-04-snapshot-{{timestamp}}",
|
||||
"apt_packages": "curl git wget software-properties-common net-tools",
|
||||
"application_name": "vm-single",
|
||||
"application_version": "{{ env `VM_VERSION` }}"
|
||||
},
|
||||
"sensitive-variables": ["do_api_token"],
|
||||
"builders": [
|
||||
{
|
||||
"type": "digitalocean",
|
||||
"api_token": "{{user `do_api_token`}}",
|
||||
"image": "ubuntu-20-04-x64",
|
||||
"region": "nyc3",
|
||||
"size": "s-1vcpu-1gb",
|
||||
"ssh_username": "root",
|
||||
"snapshot_name": "{{user `image_name`}}"
|
||||
}
|
||||
],
|
||||
"provisioners": [
|
||||
{
|
||||
"type": "shell",
|
||||
"inline": [
|
||||
"cloud-init status --wait"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "file",
|
||||
"source": "files/etc/",
|
||||
"destination": "/etc/"
|
||||
},
|
||||
{
|
||||
"type": "file",
|
||||
"source": "files/var/",
|
||||
"destination": "/var/"
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"environment_vars": [
|
||||
"DEBIAN_FRONTEND=noninteractive",
|
||||
"LC_ALL=C",
|
||||
"LANG=en_US.UTF-8",
|
||||
"LC_CTYPE=en_US.UTF-8"
|
||||
],
|
||||
"inline": [
|
||||
"apt -qqy update",
|
||||
"apt -qqy -o Dpkg::Options::='--force-confdef' -o Dpkg::Options::='--force-confold' install {{user `apt_packages`}}",
|
||||
"apt-get -qqy clean"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"environment_vars": [
|
||||
"application_name={{user `application_name`}}",
|
||||
"application_version={{user `application_version`}}",
|
||||
"DEBIAN_FRONTEND=noninteractive",
|
||||
"LC_ALL=C",
|
||||
"LANG=en_US.UTF-8",
|
||||
"LC_CTYPE=en_US.UTF-8"
|
||||
],
|
||||
"scripts": [
|
||||
"scripts/01-setup.sh",
|
||||
"scripts/02-firewall.sh",
|
||||
"scripts/04-install-victoriametrics.sh",
|
||||
"scripts/89-cleanup-logs.sh",
|
||||
"scripts/90-cleanup.sh",
|
||||
"scripts/99-img-check.sh"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,80 @@
|
|||
variable "token" {
|
||||
type = string
|
||||
default = "${env("DIGITALOCEAN_API_TOKEN")}"
|
||||
description = "DigitalOcean API token used to create droplets."
|
||||
}
|
||||
|
||||
variable "image_id" {
|
||||
type = string
|
||||
default = "ubuntu-20-04-x64"
|
||||
description = "DigitalOcean linux image ID."
|
||||
}
|
||||
|
||||
variable "victoriametrics_version" {
|
||||
type = string
|
||||
default = "${env("VM_VERSION")}"
|
||||
description = "Version number of the desired VictoriaMetrics binary."
|
||||
}
|
||||
|
||||
variable "image_name" {
|
||||
type = string
|
||||
default = "victoriametrics-snapshot-{{timestamp}}"
|
||||
description = "Name of the snapshot created on DigitalOcean."
|
||||
}
|
||||
|
||||
source "digitalocean" "default" {
|
||||
api_token = "${var.token}"
|
||||
image = "${var.image_id}"
|
||||
region = "nyc3"
|
||||
size = "s-1vcpu-1gb"
|
||||
snapshot_name = "${var.image_name}"
|
||||
ssh_username = "root"
|
||||
}
|
||||
|
||||
build {
|
||||
sources = ["source.digitalocean.default"]
|
||||
|
||||
provisioner "file" {
|
||||
destination = "/etc/"
|
||||
source = "files/etc/"
|
||||
}
|
||||
|
||||
provisioner "file" {
|
||||
destination = "/var/"
|
||||
source = "files/var/"
|
||||
}
|
||||
|
||||
# Setup instance configuration
|
||||
provisioner "shell" {
|
||||
environment_vars = [
|
||||
"DEBIAN_FRONTEND=noninteractive"
|
||||
]
|
||||
scripts = [
|
||||
"scripts/01-setup.sh",
|
||||
"scripts/02-firewall.sh",
|
||||
]
|
||||
}
|
||||
|
||||
# Install VictoriaMetrics
|
||||
provisioner "shell" {
|
||||
environment_vars = [
|
||||
"VM_VER=${var.victoriametrics_version}",
|
||||
"DEBIAN_FRONTEND=noninteractive"
|
||||
]
|
||||
scripts = [
|
||||
"scripts/04-install-victoriametrics.sh",
|
||||
]
|
||||
}
|
||||
|
||||
# Cleanup and validate instance
|
||||
provisioner "shell" {
|
||||
environment_vars = [
|
||||
"DEBIAN_FRONTEND=noninteractive"
|
||||
]
|
||||
scripts = [
|
||||
"scripts/89-cleanup-logs.sh",
|
||||
"scripts/90-cleanup.sh",
|
||||
"scripts/99-img-check.sh"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
RELEASE_NAME := vm-oneclick-droplet
|
||||
VM_VERSION ?= $(shell git describe --abbrev=0 --tags)
|
||||
|
||||
.PHONY: $(MAKECMDGOALS)
|
||||
|
||||
release-victoria-metrics-digitalocean-oneclick-droplet:
|
||||
cp ./files/etc/update-motd.d/99-one-click.tpl ./files/etc/update-motd.d/99-one-click
|
||||
sed -i -e "s/VM_VERSION/${VM_VERSION}/g" ./files/etc/update-motd.d/99-one-click
|
||||
packer build template.pkr.hcl
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
## Application summary
|
||||
|
||||
VictoriaMetrics is a fast and scalable open source time series database and monitoring solution.
|
||||
|
||||
## Description
|
||||
|
||||
VictoriaMetrics is a free [open source time series database](https://en.wikipedia.org/wiki/Time_series_database) (TSDB) and monitoring solution, designed to collect, store and process real-time metrics.
|
||||
|
||||
It supports the [Prometheus](https://en.wikipedia.org/wiki/Prometheus_(software)) pull model and various push protocols ([Graphite](https://en.wikipedia.org/wiki/Graphite_(software)), [InfluxDB](https://en.wikipedia.org/wiki/InfluxDB), OpenTSDB) for data ingestion. It is optimized for storage with high-latency IO, low IOPS and time series with [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate).
|
||||
|
||||
For reading the data and evaluating alerting rules, VictoriaMetrics supports the PromQL, [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html) and Graphite query languages. VictoriaMetrics Single is fully autonomous and can be used as a long-term storage for time series.
|
||||
|
||||
[VictoriaMetrics Single](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html) = Hassle-free monitoring solution. Easily handles 10M+ of active time series on a single instance. Perfect for small and medium environments.
|
||||
|
||||
## Getting started after deploying VictoriaMetrics Single
|
||||
|
||||
### Config
|
||||
|
||||
VictoriaMetrics configuration is located at `/etc/victoriametrics/single/scrape.yml` on the droplet.
|
||||
This One Click app uses 8428, 2003, 4242 and 8089 ports to accept metrics from different protocols. It's recommended to disable ports for protocols which are not needed. [Ubuntu firewall](https://help.ubuntu.com/community/UFW) can be used to easily disable access for specific ports.
|
||||
|
||||
### Scraping metrics
|
||||
|
||||
VictoriaMetrics supports metrics scraping in the same way as Prometheus does. Check the configuration file to edit scraping targets. See more details about scraping at [How to scrape Prometheus exporters](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-scrape-prometheus-exporters-such-as-node-exporter).
|
||||
|
||||
### Sending metrics
|
||||
|
||||
Besides scraping, VictoriaMetrics accepts write requests for various ingestion protocols. This One Click app supports the following protocols:
|
||||
- [Datadog](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-send-data-from-datadog-agent), [Influx (telegraph)](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-send-data-from-influxdb-compatible-agents-such-as-telegraf), [JSON](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-import-data-in-json-line-format), [CSV](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-import-csv-data), [Prometheus](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-import-data-in-prometheus-exposition-format) on port :8428
|
||||
- [Graphite (statsd)](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-send-data-from-graphite-compatible-agents-such-as-statsd) on port :2003 tcp/udp
|
||||
- [OpenTSDB](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-send-data-from-opentsdb-compatible-agents) on port :4242
|
||||
- Influx (telegraph) on port :8089 tcp/udp
|
||||
|
||||
See more details and examples in [official documentation](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html).
|
||||
|
||||
### UI
|
||||
|
||||
VictoriaMetrics provides a [User Interface (UI)](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui) for query troubleshooting and exploration. The UI is available at `http://your_droplet_public_ipv4:8428/vmui`. It lets users explore query results via graphs and tables.
|
||||
|
||||
To check it, open the following in your browser `http://your_droplet_public_ipv4:8428/vmui` and then enter `vm_app_uptime_seconds` to the Query Field to Execute the Query.
|
||||
|
||||
Run the following command to query and retrieve a result from VictoriaMetrics Single with `curl`:
|
||||
|
||||
```bash
|
||||
curl -sg http://your_droplet_public_ipv4:8428/api/v1/query_range?query=vm_app_uptime_seconds | jq
|
||||
```
|
||||
|
||||
### Accessing
|
||||
|
||||
Once the Droplet is created, you can use DigitalOcean's web console to start a session or SSH directly to the server as root:
|
||||
|
||||
```bash
|
||||
ssh root@your_droplet_public_ipv4
|
||||
```
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
## Release guide for DigitalOcean 1-ClickApp Droplet
|
||||
|
||||
### Build image
|
||||
|
||||
To build the snapshot in DigitalOcean account you will need API Token and [packer](https://learn.hashicorp.com/tutorials/packer/get-started-install-cli).
|
||||
|
||||
API Token can be generated on [https://cloud.digitalocean.com/account/api/tokens](https://cloud.digitalocean.com/account/api/tokens) or use already generated from OnePassword.
|
||||
|
||||
Set variable `DIGITALOCEAN_API_TOKEN` for environment:
|
||||
|
||||
```bash
|
||||
export DIGITALOCEAN_API_TOKEN="your_token_here"
|
||||
```
|
||||
|
||||
or set it by with make:
|
||||
|
||||
```bash
|
||||
make release-victoria-metrics-digitalocean-oneclick-droplet DIGITALOCEAN_API_TOKEN="your_token_here"
|
||||
```
|
||||
|
||||
### Update information on Vendor Portal
|
||||
|
||||
After packer build finished you need to update a product page.
|
||||
|
||||
1. Go to [https://cloud.digitalocean.com/vendorportal](https://cloud.digitalocean.com/vendorportal).
|
||||
2. Choose a product that you need to update.
|
||||
3. Enter newer information for this release and choose a droplet's snapshot which was builded recently.
|
||||
4. Submit updates for approve on DigitalOcean Marketplace.
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
[Unit]
|
||||
Description=VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
|
||||
# https://docs.victoriametrics.com
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=victoriametrics
|
||||
Group=victoriametrics
|
||||
WorkingDirectory=/var/lib/victoria-metrics-data
|
||||
StartLimitBurst=5
|
||||
StartLimitInterval=0
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
EnvironmentFile=-/etc/victoriametrics/single/victoriametrics.conf
|
||||
ExecStart=/usr/bin/victoria-metrics-prod $ARGS
|
||||
ExecStop=/bin/kill -s SIGTERM $MAINPID
|
||||
ExecReload=/bin/kill -HUP $MAINPID
|
||||
# See docs https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#tuning
|
||||
ProtectSystem=full
|
||||
LimitNOFILE=1048576
|
||||
LimitNPROC=1048576
|
||||
LimitCORE=infinity
|
||||
StandardOutput=syslog
|
||||
StandardError=syslog
|
||||
SyslogIdentifier=vmsingle
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
#!/bin/sh
|
||||
#
|
||||
# Configured as part of the DigitalOcean 1-Click Image build process
|
||||
|
||||
myip=$(hostname -I | awk '{print$1}')
|
||||
cat <<EOF
|
||||
********************************************************************************
|
||||
|
||||
Welcome to DigitalOcean's 1-Click VictoriaMetrics Droplet.
|
||||
To keep this Droplet secure, the UFW firewall is enabled.
|
||||
All ports are BLOCKED except 22 (SSH), 80 (HTTP), and 443 (HTTPS), 8428 (VictoriaMetrics HTTP), 8089 (VictoriaMetrics Influx),
|
||||
4242 (VictoriaMetrics OpenTSDB), 2003 (VictoriaMetrics Graphite)
|
||||
|
||||
In a web browser, you can view:
|
||||
* The VictoriaMetrics 1-Click Quickstart guide: https://kutt.it/1click-quickstart
|
||||
|
||||
On the server:
|
||||
* The default VictoriaMetrics root is located at /var/lib/victoria-metrics-data
|
||||
* VictoriaMetrics is running on ports: 8428, 8089, 4242, 2003 and they are bound to the local interface.
|
||||
|
||||
********************************************************************************
|
||||
# This image includes version VM_VERSION of VictoriaMetrics.
|
||||
# See Release notes https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/VM_VERSION
|
||||
|
||||
# Welcome to VictoriaMetrics droplet!
|
||||
|
||||
# Website: https://victoriametrics.com
|
||||
# Documentation: https://docs.victoriametrics.com
|
||||
# VictoriaMetrics Github : https://github.com/VictoriaMetrics/VictoriaMetrics
|
||||
# VictoriaMetrics Slack Community: https://slack.victoriametrics.com
|
||||
# VictoriaMetrics Telegram Community: https://t.me/VictoriaMetrics_en
|
||||
|
||||
# VictoriaMetrics config: /etc/victoriametrics/single/victoriametrics.conf
|
||||
|
||||
********************************************************************************
|
||||
EOF
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
#!/bin/sh
|
||||
#
|
||||
# Configured as part of the DigitalOcean 1-Click Image build process
|
||||
|
||||
myip=$(hostname -I | awk '{print$1}')
|
||||
cat <<EOF
|
||||
********************************************************************************
|
||||
|
||||
Welcome to DigitalOcean's 1-Click VictoriaMetrics Droplet.
|
||||
To keep this Droplet secure, the UFW firewall is enabled.
|
||||
All ports are BLOCKED except 22 (SSH), 80 (HTTP), and 443 (HTTPS), 8428 (VictoriaMetrics HTTP), 8089 (VictoriaMetrics Influx),
|
||||
4242 (VictoriaMetrics OpenTSDB), 2003 (VictoriaMetrics Graphite)
|
||||
|
||||
In a web browser, you can view:
|
||||
* The VictoriaMetrics 1-Click Quickstart guide: https://kutt.it/1click-quickstart
|
||||
|
||||
On the server:
|
||||
* The default VictoriaMetrics root is located at /var/lib/victoria-metrics-data
|
||||
* VictoriaMetrics is running on ports: 8428, 8089, 4242, 2003 and they are bound to the local interface.
|
||||
|
||||
********************************************************************************
|
||||
# This image includes version VM_VERSION of VictoriaMetrics.
|
||||
# See Release notes https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/VM_VERSION
|
||||
|
||||
# Welcome to VictoriaMetrics droplet!
|
||||
|
||||
# Website: https://victoriametrics.com
|
||||
# Documentation: https://docs.victoriametrics.com
|
||||
# VictoriaMetrics Github : https://github.com/VictoriaMetrics/VictoriaMetrics
|
||||
# VictoriaMetrics Slack Community: https://slack.victoriametrics.com
|
||||
# VictoriaMetrics Telegram Community: https://t.me/VictoriaMetrics_en
|
||||
|
||||
# VictoriaMetrics config: /etc/victoriametrics/single/victoriametrics.conf
|
||||
|
||||
********************************************************************************
|
||||
EOF
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
# Scrape config example
|
||||
#
|
||||
scrape_configs:
|
||||
- job_name: self_scrape
|
||||
scrape_interval: 10s
|
||||
static_configs:
|
||||
- targets: ['127.0.0.1:8428']
|
||||
|
|
@ -0,0 +1 @@
|
|||
ARGS="-promscrape.config=/etc/victoriametrics/single/scrape.yml -storageDataPath=/var/lib/victoria-metrics-data -retentionPeriod=12 -httpListenAddr=:8428 -graphiteListenAddr=:2003 -opentsdbListenAddr=:4242 -influxListenAddr=:8089 -enableTCP6"
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
#!/bin/sh
|
||||
|
||||
systemctl start vmsingle.service
|
||||
|
||||
# Remove the ssh force logout command
|
||||
sed -e '/Match User root/d' \
|
||||
-e '/.*ForceCommand.*droplet.*/d' \
|
||||
-i /etc/ssh/sshd_config
|
||||
|
||||
systemctl restart ssh
|
||||
15
deployment/marketplace/digitialocean/one-click-droplet/scripts/01-setup.sh
Executable file
15
deployment/marketplace/digitialocean/one-click-droplet/scripts/01-setup.sh
Executable file
|
|
@ -0,0 +1,15 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Create victoriametrics user
|
||||
groupadd -r victoriametrics
|
||||
useradd -g victoriametrics -d /var/lib/victoria-metrics-data -s /sbin/nologin --system victoriametrics
|
||||
|
||||
mkdir -p /var/lib/victoria-metrics-data
|
||||
chown -R victoriametrics:victoriametrics /var/lib/victoria-metrics-data
|
||||
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
apt update
|
||||
DEBIAN_FRONTEND=noninteractive apt -y full-upgrade
|
||||
DEBIAN_FRONTEND=noninteractive apt -y install curl git wget software-properties-common
|
||||
rm -rf /var/log/kern.log
|
||||
rm -rf /var/log/ufw.log
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
#!/bin/sh
|
||||
|
||||
sed -e 's|DEFAULT_FORWARD_POLICY=.*|DEFAULT_FORWARD_POLICY="ACCEPT"|g' \
|
||||
-i /etc/default/ufw
|
||||
|
||||
ufw allow ssh comment "SSH port"
|
||||
ufw allow http comment "HTTP port"
|
||||
ufw allow https comment "HTTPS port"
|
||||
ufw allow 8428 comment "VictoriaMetrics Single HTTP port"
|
||||
ufw allow 8089/tcp comment "TCP Influx Listen port for VictoriaMetrics"
|
||||
ufw allow 8089/udp comment "UDP Influx Listen port for VictoriaMetrics"
|
||||
ufw allow 2003/tcp comment "TCP Graphite Listen port for VictoriaMetrics"
|
||||
ufw allow 2003/udp comment "UDP Graphite Listen port for VictoriaMetrics"
|
||||
ufw allow 4242 comment "OpenTSDB Listen port for VictoriaMetrics"
|
||||
|
||||
ufw --force enable
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Wait for cloud-init
|
||||
cloud-init status --wait
|
||||
|
||||
wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/${VM_VER}/victoria-metrics-amd64-${VM_VER}.tar.gz -O /tmp/victoria-metrics.tar.gz
|
||||
tar xvf /tmp/victoria-metrics.tar.gz -C /usr/bin
|
||||
chmod +x /usr/bin/victoria-metrics-prod
|
||||
chown root:root /usr/bin/victoria-metrics-prod
|
||||
|
||||
|
||||
# Enable VictoriaMetrics on boot
|
||||
systemctl enable vmsingle.service
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
#!/bin/bash
|
||||
#
|
||||
# Remove log files generated during instance creation and
|
||||
# configuration.
|
||||
|
||||
rm -rf /var/log/kern.log
|
||||
rm -rf /var/log/ufw.log
|
||||
44
deployment/marketplace/digitialocean/one-click-droplet/scripts/90-cleanup.sh
Executable file
44
deployment/marketplace/digitialocean/one-click-droplet/scripts/90-cleanup.sh
Executable file
|
|
@ -0,0 +1,44 @@
|
|||
#!/bin/bash
|
||||
|
||||
# Ensure /tmp exists and has the proper permissions before
|
||||
# checking for security updates
|
||||
# https://github.com/digitalocean/marketplace-partners/issues/94
|
||||
if [[ ! -d /tmp ]]; then
|
||||
mkdir /tmp
|
||||
fi
|
||||
chmod 1777 /tmp
|
||||
|
||||
apt-get -y update
|
||||
apt-get -y upgrade
|
||||
rm -rf /tmp/* /var/tmp/*
|
||||
history -c
|
||||
cat /dev/null > /root/.bash_history
|
||||
unset HISTFILE
|
||||
apt-get -y autoremove
|
||||
apt-get -y autoclean
|
||||
find /var/log -mtime -1 -type f -exec truncate -s 0 {} \;
|
||||
rm -rf /var/log/*.gz /var/log/*.[0-9] /var/log/*-????????
|
||||
rm -rf /var/lib/cloud/instances/*
|
||||
rm -f /root/.ssh/authorized_keys /etc/ssh/*key*
|
||||
touch /etc/ssh/revoked_keys
|
||||
chmod 600 /etc/ssh/revoked_keys
|
||||
|
||||
# Securely erase the unused portion of the filesystem
|
||||
GREEN='\033[0;32m'
|
||||
NC='\033[0m'
|
||||
printf "\n${GREEN}Writing zeros to the remaining disk space to securely
|
||||
erase the unused portion of the file system.
|
||||
Depending on your disk size this may take several minutes.
|
||||
The secure erase will complete successfully when you see:${NC}
|
||||
dd: writing to '/zerofile': No space left on device\n
|
||||
Beginning secure erase now\n"
|
||||
|
||||
dd if=/dev/zero of=/zerofile &
|
||||
PID=$!
|
||||
while [ -d /proc/$PID ]
|
||||
do
|
||||
printf "."
|
||||
sleep 5
|
||||
done
|
||||
sync; rm /zerofile; sync
|
||||
cat /dev/null > /var/log/lastlog; cat /dev/null > /var/log/wtmp
|
||||
682
deployment/marketplace/digitialocean/one-click-droplet/scripts/99-img-check.sh
Executable file
682
deployment/marketplace/digitialocean/one-click-droplet/scripts/99-img-check.sh
Executable file
|
|
@ -0,0 +1,682 @@
|
|||
#!/bin/bash
|
||||
|
||||
# DigitalOcean Marketplace Image Validation Tool
|
||||
# © 2021 DigitalOcean LLC.
|
||||
# This code is licensed under Apache 2.0 license (see LICENSE.md for details)
|
||||
|
||||
VERSION="v. 1.6"
|
||||
RUNDATE=$( date )
|
||||
|
||||
# Script should be run with SUDO
|
||||
if [ "$EUID" -ne 0 ]
|
||||
then echo "[Error] - This script must be run with sudo or as the root user."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
STATUS=0
|
||||
PASS=0
|
||||
WARN=0
|
||||
FAIL=0
|
||||
|
||||
# $1 == command to check for
|
||||
# returns: 0 == true, 1 == false
|
||||
cmdExists() {
|
||||
if command -v "$1" > /dev/null 2>&1; then
|
||||
return 0
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
function getDistro {
|
||||
if [ -f /etc/os-release ]; then
|
||||
# freedesktop.org and systemd
|
||||
. /etc/os-release
|
||||
OS=$NAME
|
||||
VER=$VERSION_ID
|
||||
elif type lsb_release >/dev/null 2>&1; then
|
||||
# linuxbase.org
|
||||
OS=$(lsb_release -si)
|
||||
VER=$(lsb_release -sr)
|
||||
elif [ -f /etc/lsb-release ]; then
|
||||
# For some versions of Debian/Ubuntu without lsb_release command
|
||||
. /etc/lsb-release
|
||||
OS=$DISTRIB_ID
|
||||
VER=$DISTRIB_RELEASE
|
||||
elif [ -f /etc/debian_version ]; then
|
||||
# Older Debian/Ubuntu/etc.
|
||||
OS=Debian
|
||||
VER=$(cat /etc/debian_version)
|
||||
elif [ -f /etc/SuSe-release ]; then
|
||||
# Older SuSE/etc.
|
||||
:
|
||||
elif [ -f /etc/redhat-release ]; then
|
||||
# Older Red Hat, CentOS, etc.
|
||||
VER=$( cat /etc/redhat-release | cut -d" " -f3 | cut -d "." -f1)
|
||||
d=$( cat /etc/redhat-release | cut -d" " -f1 | cut -d "." -f1)
|
||||
if [[ $d == "CentOS" ]]; then
|
||||
OS="CentOS Linux"
|
||||
fi
|
||||
else
|
||||
# Fall back to uname, e.g. "Linux <version>", also works for BSD, etc.
|
||||
OS=$(uname -s)
|
||||
VER=$(uname -r)
|
||||
fi
|
||||
}
|
||||
function loadPasswords {
|
||||
SHADOW=$(cat /etc/shadow)
|
||||
}
|
||||
|
||||
function checkAgent {
|
||||
# Check for the presence of the do-agent in the filesystem
|
||||
if [ -d /var/opt/digitalocean/do-agent ];then
|
||||
echo -en "\e[41m[FAIL]\e[0m DigitalOcean Monitoring Agent detected.\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
if [[ $OS == "CentOS Linux" ]]; then
|
||||
echo "The agent can be removed with 'sudo yum remove do-agent' "
|
||||
elif [[ $OS == "Ubuntu" ]]; then
|
||||
echo "The agent can be removed with 'sudo apt-get purge do-agent' "
|
||||
fi
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m DigitalOcean Monitoring agent was not found\n"
|
||||
((PASS++))
|
||||
fi
|
||||
}
|
||||
|
||||
function checkLogs {
|
||||
cp_ignore="/var/log/cpanel-install.log"
|
||||
echo -en "\nChecking for log files in /var/log\n\n"
|
||||
# Check if there are log archives or log files that have not been recently cleared.
|
||||
for f in /var/log/*-????????; do
|
||||
[[ -e $f ]] || break
|
||||
if [ $f != $cp_ignore ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m Log archive ${f} found\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
done
|
||||
for f in /var/log/*.[0-9];do
|
||||
[[ -e $f ]] || break
|
||||
echo -en "\e[93m[WARN]\e[0m Log archive ${f} found\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
done
|
||||
for f in /var/log/*.log; do
|
||||
[[ -e $f ]] || break
|
||||
if [[ "${f}" = '/var/log/lfd.log' && "$( cat "${f}" | egrep -v '/var/log/messages has been reset| Watching /var/log/messages' | wc -c)" -gt 50 ]]; then
|
||||
if [ $f != $cp_ignore ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m un-cleared log file, ${f} found\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
elif [[ "${f}" != '/var/log/lfd.log' && "$( cat "${f}" | wc -c)" -gt 50 ]]; then
|
||||
if [ $f != $cp_ignore ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m un-cleared log file, ${f} found\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
}
|
||||
function checkTMP {
|
||||
# Check the /tmp directory to ensure it is empty. Warn on any files found.
|
||||
return 1
|
||||
}
|
||||
function checkRoot {
|
||||
user="root"
|
||||
uhome="/root"
|
||||
for usr in $SHADOW
|
||||
do
|
||||
IFS=':' read -r -a u <<< "$usr"
|
||||
if [[ "${u[0]}" == "${user}" ]]; then
|
||||
if [[ ${u[1]} == "!" ]] || [[ ${u[1]} == "!!" ]] || [[ ${u[1]} == "*" ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m User ${user} has no password set.\n"
|
||||
((PASS++))
|
||||
else
|
||||
echo -en "\e[41m[FAIL]\e[0m User ${user} has a password set on their account.\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
fi
|
||||
done
|
||||
if [ -d ${uhome}/ ]; then
|
||||
if [ -d ${uhome}/.ssh/ ]; then
|
||||
if ls ${uhome}/.ssh/*> /dev/null 2>&1; then
|
||||
for key in ${uhome}/.ssh/*
|
||||
do
|
||||
if [ "${key}" == "${uhome}/.ssh/authorized_keys" ]; then
|
||||
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 50 ]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m User \e[1m${user}\e[0m has a populated authorized_keys file in \e[93m${key}\e[0m\n"
|
||||
akey=$(cat ${key})
|
||||
echo "File Contents:"
|
||||
echo $akey
|
||||
echo "--------------"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
elif [ "${key}" == "${uhome}/.ssh/id_rsa" ]; then
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 0 ]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m User \e[1m${user}\e[0m has a private key file in \e[93m${key}\e[0m\n"
|
||||
akey=$(cat ${key})
|
||||
echo "File Contents:"
|
||||
echo $akey
|
||||
echo "--------------"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has empty private key file in \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
elif [ "${key}" != "${uhome}/.ssh/known_hosts" ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has a file in their .ssh directory at \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
else
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 50 ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has a populated known_hosts file in \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m has no SSH keys present\n"
|
||||
fi
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m does not have an .ssh directory\n"
|
||||
fi
|
||||
if [ -f /root/.bash_history ];then
|
||||
|
||||
BH_S=$( cat /root/.bash_history | wc -c)
|
||||
|
||||
if [[ $BH_S -lt 200 ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m ${user}'s Bash History appears to have been cleared\n"
|
||||
((PASS++))
|
||||
else
|
||||
echo -en "\e[41m[FAIL]\e[0m ${user}'s Bash History should be cleared to prevent sensitive information from leaking\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
|
||||
return 1;
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m The Root User's Bash History is not present\n"
|
||||
((PASS++))
|
||||
fi
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m does not have a directory in /home\n"
|
||||
fi
|
||||
echo -en "\n\n"
|
||||
return 1
|
||||
}
|
||||
|
||||
function checkUsers {
|
||||
# Check each user-created account
|
||||
for user in $(awk -F: '$3 >= 1000 && $1 != "nobody" {print $1}' /etc/passwd;)
|
||||
do
|
||||
# Skip some other non-user system accounts
|
||||
if [[ $user == "centos" ]]; then
|
||||
:
|
||||
elif [[ $user == "nfsnobody" ]]; then
|
||||
:
|
||||
else
|
||||
echo -en "\nChecking user: ${user}...\n"
|
||||
for usr in $SHADOW
|
||||
do
|
||||
IFS=':' read -r -a u <<< "$usr"
|
||||
if [[ "${u[0]}" == "${user}" ]]; then
|
||||
if [[ ${u[1]} == "!" ]] || [[ ${u[1]} == "!!" ]] || [[ ${u[1]} == "*" ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m User ${user} has no password set.\n"
|
||||
((PASS++))
|
||||
else
|
||||
echo -en "\e[41m[FAIL]\e[0m User ${user} has a password set on their account. Only system users are allowed on the image.\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
fi
|
||||
done
|
||||
#echo "User Found: ${user}"
|
||||
uhome="/home/${user}"
|
||||
if [ -d "${uhome}/" ]; then
|
||||
if [ -d "${uhome}/.ssh/" ]; then
|
||||
if ls "${uhome}/.ssh/*"> /dev/null 2>&1; then
|
||||
for key in ${uhome}/.ssh/*
|
||||
do
|
||||
if [ "${key}" == "${uhome}/.ssh/authorized_keys" ]; then
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 50 ]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m User \e[1m${user}\e[0m has a populated authorized_keys file in \e[93m${key}\e[0m\n"
|
||||
akey=$(cat ${key})
|
||||
echo "File Contents:"
|
||||
echo $akey
|
||||
echo "--------------"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
elif [ "${key}" == "${uhome}/.ssh/id_rsa" ]; then
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 0 ]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m User \e[1m${user}\e[0m has a private key file in \e[93m${key}\e[0m\n"
|
||||
akey=$(cat ${key})
|
||||
echo "File Contents:"
|
||||
echo $akey
|
||||
echo "--------------"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has empty private key file in \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
elif [ "${key}" != "${uhome}/.ssh/known_hosts" ]; then
|
||||
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has a file in their .ssh directory named \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
|
||||
else
|
||||
if [ "$( cat "${key}" | wc -c)" -gt 50 ]; then
|
||||
echo -en "\e[93m[WARN]\e[0m User \e[1m${user}\e[0m has a known_hosts file in \e[93m${key}\e[0m\n"
|
||||
((WARN++))
|
||||
if [[ $STATUS != 2 ]]; then
|
||||
STATUS=1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
done
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m has no SSH keys present\n"
|
||||
fi
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m does not have an .ssh directory\n"
|
||||
fi
|
||||
else
|
||||
echo -en "\e[32m[ OK ]\e[0m User \e[1m${user}\e[0m does not have a directory in /home\n"
|
||||
fi
|
||||
|
||||
# Check for an uncleared .bash_history for this user
|
||||
if [ -f "${uhome}/.bash_history" ]; then
|
||||
BH_S=$( cat "${uhome}/.bash_history" | wc -c )
|
||||
|
||||
if [[ $BH_S -lt 200 ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m ${user}'s Bash History appears to have been cleared\n"
|
||||
((PASS++))
|
||||
else
|
||||
echo -en "\e[41m[FAIL]\e[0m ${user}'s Bash History should be cleared to prevent sensitive information from leaking\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
|
||||
fi
|
||||
echo -en "\n\n"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
}
|
||||
function checkFirewall {
|
||||
|
||||
if [[ $OS == "Ubuntu" ]]; then
|
||||
fw="ufw"
|
||||
ufwa=$(ufw status |head -1| sed -e "s/^Status:\ //")
|
||||
if [[ $ufwa == "active" ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
elif [[ $OS == "CentOS Linux" ]]; then
|
||||
if [ -f /usr/lib/systemd/system/csf.service ]; then
|
||||
fw="csf"
|
||||
if [[ $(systemctl status $fw >/dev/null 2>&1) ]]; then
|
||||
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
elif cmdExists "firewall-cmd"; then
|
||||
if [[ $(systemctl is-active firewalld >/dev/null 2>&1 && echo 1 || echo 0) ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
else
|
||||
fw="firewalld"
|
||||
if [[ $(systemctl is-active firewalld >/dev/null 2>&1 && echo 1 || echo 0) ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
fi
|
||||
elif [[ "$OS" =~ Debian.* ]]; then
|
||||
# user could be using a number of different services for managing their firewall
|
||||
# we will check some of the most common
|
||||
if cmdExists 'ufw'; then
|
||||
fw="ufw"
|
||||
ufwa=$(ufw status |head -1| sed -e "s/^Status:\ //")
|
||||
if [[ $ufwa == "active" ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
elif cmdExists "firewall-cmd"; then
|
||||
fw="firewalld"
|
||||
if [[ $(systemctl is-active --quiet $fw) ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
else
|
||||
# user could be using vanilla iptables, check if kernel module is loaded
|
||||
fw="iptables"
|
||||
if [[ $(lsmod | grep -q '^ip_tables' 2>/dev/null) ]]; then
|
||||
FW_VER="\e[32m[PASS]\e[0m Firewall service (${fw}) is active\n"
|
||||
((PASS++))
|
||||
else
|
||||
FW_VER="\e[93m[WARN]\e[0m No firewall is configured. Ensure ${fw} is installed and configured\n"
|
||||
((WARN++))
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
}
|
||||
function checkUpdates {
|
||||
if [[ $OS == "Ubuntu" ]] || [[ "$OS" =~ Debian.* ]]; then
|
||||
# Ensure /tmp exists and has the proper permissions before
|
||||
# checking for security updates
|
||||
# https://github.com/digitalocean/marketplace-partners/issues/94
|
||||
if [[ ! -d /tmp ]]; then
|
||||
mkdir /tmp
|
||||
fi
|
||||
chmod 1777 /tmp
|
||||
|
||||
echo -en "\nUpdating apt package database to check for security updates, this may take a minute...\n\n"
|
||||
apt-get -y update > /dev/null
|
||||
|
||||
uc=$(apt-get --just-print upgrade | grep -i "security" | wc -l)
|
||||
if [[ $uc -gt 0 ]]; then
|
||||
update_count=$(( ${uc} / 2 ))
|
||||
else
|
||||
update_count=0
|
||||
fi
|
||||
|
||||
if [[ $update_count -gt 0 ]]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m There are ${update_count} security updates available for this image that have not been installed.\n"
|
||||
echo -en
|
||||
echo -en "Here is a list of the security updates that are not installed:\n"
|
||||
sleep 2
|
||||
apt-get --just-print upgrade | grep -i security | awk '{print $2}' | awk '!seen[$0]++'
|
||||
echo -en
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m There are no pending security updates for this image.\n\n"
|
||||
fi
|
||||
elif [[ $OS == "CentOS Linux" ]]; then
|
||||
echo -en "\nChecking for available security updates, this may take a minute...\n\n"
|
||||
|
||||
update_count=$(yum check-update --security --quiet | wc -l)
|
||||
if [[ $update_count -gt 0 ]]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m There are ${update_count} security updates available for this image that have not been installed.\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m There are no pending security updates for this image.\n"
|
||||
((PASS++))
|
||||
fi
|
||||
else
|
||||
echo "Error encountered"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
return 1;
|
||||
}
|
||||
function checkCloudInit {
|
||||
|
||||
if hash cloud-init 2>/dev/null; then
|
||||
CI="\e[32m[PASS]\e[0m Cloud-init is installed.\n"
|
||||
((PASS++))
|
||||
else
|
||||
CI="\e[41m[FAIL]\e[0m No valid verison of cloud-init was found.\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
function checkMongoDB {
|
||||
# Check if MongoDB is installed
|
||||
# If it is, verify the version is allowed (non-SSPL)
|
||||
|
||||
if [[ $OS == "Ubuntu" ]] || [[ "$OS" =~ Debian.* ]]; then
|
||||
|
||||
if [[ -f "/usr/bin/mongod" ]]; then
|
||||
version=$(/usr/bin/mongod --version --quiet | grep "db version" | sed -e "s/^db\ version\ v//")
|
||||
|
||||
if version_gt $version 4.0.0; then
|
||||
if version_gt $version 4.0.3; then
|
||||
echo -en "\e[41m[FAIL]\e[0m An SSPL version of MongoDB is present, ${version}"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m The version of MongoDB installed, ${version} is not under the SSPL"
|
||||
((PASS++))
|
||||
fi
|
||||
else
|
||||
if version_gt $version 3.6.8; then
|
||||
echo -en "\e[41m[FAIL]\e[0m An SSPL version of MongoDB is present, ${version}"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m The version of MongoDB installed, ${version} is not under the SSPL"
|
||||
((PASS++))
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m MongoDB is not installed"
|
||||
((PASS++))
|
||||
fi
|
||||
|
||||
elif [[ $OS == "CentOS Linux" ]]; then
|
||||
|
||||
if [[ -f "/usr/bin/mongod" ]]; then
|
||||
version=$(/usr/bin/mongod --version --quiet | grep "db version" | sed -e "s/^db\ version\ v//")
|
||||
|
||||
|
||||
if version_gt $version 4.0.0; then
|
||||
if version_gt $version 4.0.3; then
|
||||
echo -en "\e[41m[FAIL]\e[0m An SSPL version of MongoDB is present"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m The version of MongoDB installed is not under the SSPL"
|
||||
((PASS++))
|
||||
fi
|
||||
else
|
||||
if version_gt $version 3.6.8; then
|
||||
echo -en "\e[41m[FAIL]\e[0m An SSPL version of MongoDB is present"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m The version of MongoDB installed is not under the SSPL"
|
||||
((PASS++))
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
|
||||
else
|
||||
echo -en "\e[32m[PASS]\e[0m MongoDB is not installed"
|
||||
((PASS++))
|
||||
fi
|
||||
|
||||
else
|
||||
echo "ERROR: Unable to identify distribution"
|
||||
((FAIL++))
|
||||
STATUS 2
|
||||
return 1
|
||||
fi
|
||||
|
||||
|
||||
}
|
||||
|
||||
function version_gt() { test "$(printf '%s\n' "$@" | sort -V | head -n 1)" != "$1"; }
|
||||
|
||||
|
||||
clear
|
||||
echo "DigitalOcean Marketplace Image Validation Tool ${VERSION}"
|
||||
echo "Executed on: ${RUNDATE}"
|
||||
echo "Checking local system for Marketplace compatibility..."
|
||||
|
||||
getDistro
|
||||
|
||||
echo -en "\n\e[1mDistribution:\e[0m ${OS}\n"
|
||||
echo -en "\e[1mVersion:\e[0m ${VER}\n\n"
|
||||
|
||||
ost=0
|
||||
osv=0
|
||||
|
||||
if [[ $OS == "Ubuntu" ]]; then
|
||||
ost=1
|
||||
if [[ $VER == "20.04" ]]; then
|
||||
osv=1
|
||||
elif [[ $VER == "18.04" ]]; then
|
||||
osv=1
|
||||
elif [[ $VER == "16.04" ]]; then
|
||||
osv=1
|
||||
else
|
||||
osv=0
|
||||
fi
|
||||
|
||||
elif [[ "$OS" =~ Debian.* ]]; then
|
||||
ost=1
|
||||
case "$VER" in
|
||||
9)
|
||||
osv=1
|
||||
;;
|
||||
10)
|
||||
osv=1
|
||||
;;
|
||||
*)
|
||||
osv=2
|
||||
;;
|
||||
esac
|
||||
|
||||
elif [[ $OS == "CentOS Linux" ]]; then
|
||||
ost=1
|
||||
if [[ $VER == "8" ]]; then
|
||||
osv=1
|
||||
elif [[ $VER == "7" ]]; then
|
||||
osv=1
|
||||
elif [[ $VER == "6" ]]; then
|
||||
osv=1
|
||||
else
|
||||
osv=2
|
||||
fi
|
||||
else
|
||||
ost=0
|
||||
fi
|
||||
|
||||
if [[ $ost == 1 ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m Supported Operating System Detected: ${OS}\n"
|
||||
((PASS++))
|
||||
else
|
||||
echo -en "\e[41m[FAIL]\e[0m ${OS} is not a supported Operating System\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
fi
|
||||
|
||||
if [[ $osv == 1 ]]; then
|
||||
echo -en "\e[32m[PASS]\e[0m Supported Release Detected: ${VER}\n"
|
||||
((PASS++))
|
||||
elif [[ $ost == 1 ]]; then
|
||||
echo -en "\e[41m[FAIL]\e[0m ${OS} ${VER} is not a supported Operating System Version\n"
|
||||
((FAIL++))
|
||||
STATUS=2
|
||||
else
|
||||
echo "Exiting..."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
checkCloudInit
|
||||
|
||||
echo -en "${CI}"
|
||||
|
||||
checkFirewall
|
||||
|
||||
echo -en "${FW_VER}"
|
||||
|
||||
checkUpdates
|
||||
|
||||
loadPasswords
|
||||
|
||||
checkLogs
|
||||
|
||||
echo -en "\n\nChecking all user-created accounts...\n"
|
||||
checkUsers
|
||||
|
||||
echo -en "\n\nChecking the root account...\n"
|
||||
checkRoot
|
||||
|
||||
checkAgent
|
||||
|
||||
checkMongoDB
|
||||
|
||||
|
||||
# Summary
|
||||
echo -en "\n\n---------------------------------------------------------------------------------------------------\n"
|
||||
|
||||
if [[ $STATUS == 0 ]]; then
|
||||
echo -en "Scan Complete.\n\e[32mAll Tests Passed!\e[0m\n"
|
||||
elif [[ $STATUS == 1 ]]; then
|
||||
echo -en "Scan Complete. \n\e[93mSome non-critical tests failed. Please review these items.\e[0m\e[0m\n"
|
||||
else
|
||||
echo -en "Scan Complete. \n\e[41mOne or more tests failed. Please review these items and re-test.\e[0m\n"
|
||||
fi
|
||||
echo "---------------------------------------------------------------------------------------------------"
|
||||
echo -en "\e[1m${PASS} Tests PASSED\e[0m\n"
|
||||
echo -en "\e[1m${WARN} WARNINGS\e[0m\n"
|
||||
echo -en "\e[1m${FAIL} Tests FAILED\e[0m\n"
|
||||
echo -en "---------------------------------------------------------------------------------------------------\n"
|
||||
|
||||
if [[ $STATUS == 0 ]]; then
|
||||
echo -en "We did not detect any issues with this image. Please be sure to manually ensure that all software installed on the base system is functional, secure and properly configured (or facilities for configuration on first-boot have been created).\n\n"
|
||||
exit 0
|
||||
elif [[ $STATUS == 1 ]]; then
|
||||
echo -en "Please review all [WARN] items above and ensure they are intended or resolved. If you do not have a specific requirement, we recommend resolving these items before image submission\n\n"
|
||||
exit 0
|
||||
else
|
||||
echo -en "Some critical tests failed. These items must be resolved and this scan re-run before you submit your image to the DigitalOcean Marketplace.\n\n"
|
||||
exit 1
|
||||
fi
|
||||
|
|
@ -0,0 +1,73 @@
|
|||
|
||||
{
|
||||
"variables": {
|
||||
"do_api_token": "{{env `DIGITALOCEAN_API_TOKEN`}}",
|
||||
"image_name": "vm-single-20-04-snapshot-{{timestamp}}",
|
||||
"apt_packages": "curl git wget software-properties-common net-tools",
|
||||
"application_name": "vm-single",
|
||||
"application_version": "{{ env `VM_VERSION` }}"
|
||||
},
|
||||
"sensitive-variables": ["do_api_token"],
|
||||
"builders": [
|
||||
{
|
||||
"type": "digitalocean",
|
||||
"api_token": "{{user `do_api_token`}}",
|
||||
"image": "ubuntu-20-04-x64",
|
||||
"region": "nyc3",
|
||||
"size": "s-1vcpu-1gb",
|
||||
"ssh_username": "root",
|
||||
"snapshot_name": "{{user `image_name`}}"
|
||||
}
|
||||
],
|
||||
"provisioners": [
|
||||
{
|
||||
"type": "shell",
|
||||
"inline": [
|
||||
"cloud-init status --wait"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "file",
|
||||
"source": "files/etc/",
|
||||
"destination": "/etc/"
|
||||
},
|
||||
{
|
||||
"type": "file",
|
||||
"source": "files/var/",
|
||||
"destination": "/var/"
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"environment_vars": [
|
||||
"DEBIAN_FRONTEND=noninteractive",
|
||||
"LC_ALL=C",
|
||||
"LANG=en_US.UTF-8",
|
||||
"LC_CTYPE=en_US.UTF-8"
|
||||
],
|
||||
"inline": [
|
||||
"apt -qqy update",
|
||||
"apt -qqy -o Dpkg::Options::='--force-confdef' -o Dpkg::Options::='--force-confold' install {{user `apt_packages`}}",
|
||||
"apt-get -qqy clean"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "shell",
|
||||
"environment_vars": [
|
||||
"application_name={{user `application_name`}}",
|
||||
"application_version={{user `application_version`}}",
|
||||
"DEBIAN_FRONTEND=noninteractive",
|
||||
"LC_ALL=C",
|
||||
"LANG=en_US.UTF-8",
|
||||
"LC_CTYPE=en_US.UTF-8"
|
||||
],
|
||||
"scripts": [
|
||||
"scripts/01-setup.sh",
|
||||
"scripts/02-firewall.sh",
|
||||
"scripts/04-install-victoriametrics.sh",
|
||||
"scripts/89-cleanup-logs.sh",
|
||||
"scripts/90-cleanup.sh",
|
||||
"scripts/99-img-check.sh"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,80 @@
|
|||
variable "token" {
|
||||
type = string
|
||||
default = "${env("DIGITALOCEAN_API_TOKEN")}"
|
||||
description = "DigitalOcean API token used to create droplets."
|
||||
}
|
||||
|
||||
variable "image_id" {
|
||||
type = string
|
||||
default = "ubuntu-20-04-x64"
|
||||
description = "DigitalOcean linux image ID."
|
||||
}
|
||||
|
||||
variable "victoriametrics_version" {
|
||||
type = string
|
||||
default = "${env("VM_VERSION")}"
|
||||
description = "Version number of the desired VictoriaMetrics binary."
|
||||
}
|
||||
|
||||
variable "image_name" {
|
||||
type = string
|
||||
default = "victoriametrics-snapshot-{{timestamp}}"
|
||||
description = "Name of the snapshot created on DigitalOcean."
|
||||
}
|
||||
|
||||
source "digitalocean" "default" {
|
||||
api_token = "${var.token}"
|
||||
image = "${var.image_id}"
|
||||
region = "nyc3"
|
||||
size = "s-1vcpu-1gb"
|
||||
snapshot_name = "${var.image_name}"
|
||||
ssh_username = "root"
|
||||
}
|
||||
|
||||
build {
|
||||
sources = ["source.digitalocean.default"]
|
||||
|
||||
provisioner "file" {
|
||||
destination = "/etc/"
|
||||
source = "files/etc/"
|
||||
}
|
||||
|
||||
provisioner "file" {
|
||||
destination = "/var/"
|
||||
source = "files/var/"
|
||||
}
|
||||
|
||||
# Setup instance configuration
|
||||
provisioner "shell" {
|
||||
environment_vars = [
|
||||
"DEBIAN_FRONTEND=noninteractive"
|
||||
]
|
||||
scripts = [
|
||||
"scripts/01-setup.sh",
|
||||
"scripts/02-firewall.sh",
|
||||
]
|
||||
}
|
||||
|
||||
# Install VictoriaMetrics
|
||||
provisioner "shell" {
|
||||
environment_vars = [
|
||||
"VM_VER=${var.victoriametrics_version}",
|
||||
"DEBIAN_FRONTEND=noninteractive"
|
||||
]
|
||||
scripts = [
|
||||
"scripts/04-install-victoriametrics.sh",
|
||||
]
|
||||
}
|
||||
|
||||
# Cleanup and validate instance
|
||||
provisioner "shell" {
|
||||
environment_vars = [
|
||||
"DEBIAN_FRONTEND=noninteractive"
|
||||
]
|
||||
scripts = [
|
||||
"scripts/89-cleanup-logs.sh",
|
||||
"scripts/90-cleanup.sh",
|
||||
"scripts/99-img-check.sh"
|
||||
]
|
||||
}
|
||||
}
|
||||
Loading…
Reference in a new issue