Author Topic: Backup CWP to an Amazon S3 bucket  (Read 694 times)

0 Members and 1 Guest are viewing this topic.

Offline
*
Backup CWP to an Amazon S3 bucket
« on: September 07, 2021, 06:23:45 PM »
Hey folks, sharing this here if anyone needs it!

I host CWP on Amazon EC2, needed a better backup solution because:
  • standard EC2 volumes are expensive.
  • cheaper (per GB) volumes for backups are minimum 500GB, so end up expensive anyway.
  • CWP native backups suck.
  • CWP backups to FTP have extreme versioning issues, the daly backup simply overwrites the previous day.

To do this, you'll need to create Amazon security credentials here: https://console.aws.amazon.com/iam/home?#/security_credentials
Go to the section titled "Access keys (access key ID and secret access key)".
Create yourself an 'access key'.

Install Amazon client on your CWP server:
Code: [Select]
# sudo yum install -y python-pip
Code: [Select]
# sudo pip install awscli
Code: [Select]
# sudo pip install futures
Code: [Select]
# pip uninstall rsa
Code: [Select]
# pip install -v rsa==4.0

Set up the Amazon connection to S3 and create a bucket
Code: [Select]
# aws configure
In here, add your access keys and AWS region like so:
Code: [Select]
AWS Access Key ID [None]: ACCESSKEYID
AWS Secret Access Key [None]: ACCESSKEY
Default region name [None]: us-west-2
Default output format [None]: json

Once set up, create your S3 bucket (replace 'servername_backups' with the name of your desired S3 bucket):
Code: [Select]
# aws s3 mb s3://servername_backups
Create a backup script (if you don't have nano, use vi instead or install nano with 'yum install nano'):
Code: [Select]
# nano backups-s3.sh
Add the backup script content, sample of mine here:
https://gist.github.com/FreshLondon/0e2bd776ba69ab542b1afef0ecdd0db9
Don't forget to add your S3 bucket name at the top of your file, and choose your /home/ directory path if it isn't standard..

Save the file, then make that script executable:
Code: [Select]
# chmod +x backups-s3.sh
Run the script:
Code: [Select]
# ./backups-s3.sh
Add a cron job for the script to run every night:
Open chrontab:
Code: [Select]
#crontab -eAdd a new line at the end fo the file:
Code: [Select]
0 3 * * * /path/to/backups-s3.sh > /dev/null 2>&1The above runs your backup script every day at 3am (according to the serverís clock).


Hope it helps!
Thanks to cinique for the original script format, modified for Amazon S3 use (and reduced a lot of the process to preserve disk space on the server doing the backup.

Offline
*
Re: Backup CWP to an Amazon S3 bucket
« Reply #1 on: November 20, 2021, 07:13:03 PM »
Hey, thank you for the great post. Please can you help in which folder/server should I put the backups-s3.sh file? The rest of commands should be executed on the server too ?