Is there any way to make remote backup to google drive ?

Started by net.136@rediffmail.com, May 01, 2018, 01:43:27 PM

Previous topic - Next topic

net.136@rediffmail.com

Is there any way to make remote backup to google drive ?

Jamshed Datori

Quote from: net.136@rediffmail.com on May 01, 2018, 01:43:27 PM
Is there any way to make remote backup to google drive ?

According to CWP support, they are releasing the backup module very soon which will support external locations too.

belrpr


daz123

This is based on a script I use for another Control Panel.

What this script does is backup everything in "/backup" folder (admin backups), and upload it to google drive in tar.gz format, this script will run after the CWP admin backup script has ran.

To use; get the gdrive binary for your OS (https://github.com/gdrive-org/gdrive#downloads).

Centos 6/7 64bit:
wget https://drive.google.com/uc?id=1Ej8VgsW5RgK66Btb9p74tSdHMH3p4UNb&export=download

Centos 6 32-bit
wget https://drive.google.com/uc?id=1eo9hMXz0WyuBwRxPM0LrTtQmhTgOLUlg&export=download

Make it executable:
chmod +x gdrive

Move it to /usr/sbin
mv gdrive /usr/sbin/

Setup:
gdrive about

Follow the link and get the verification code by authenticating with the google account for the drive you want access to.

Add the script into /root directory named: GDbackup.sh:
nano /root/GDbackup.sh

Copy  and paste the below script into the new file using (CTRL C / CTRL V).

And then save the file:
CTRL + X + Y

Script:
#!/bin/bash
IFS=$'\n'

# Copy the folder you want to store your file in at Google Drive (eg. https://drive.google.com/.../folders/...).
DRIVEFOLDERID=

# Days in which to delete the old backups from google drive (set 0 to disable).
DELETEDAYS=180


  echo "#############################################"
  echo "   ---Google Drive Backup---          "
  echo "#############################################"
  gdrive about
  echo "#############################################"

  DATE=$(date '+%d-%m-%Y')
  TS=$(date +'%H%M%S')
  tar -czvf /tmp/$HOSTNAME.$DATE.$TS.backup.tar.gz -C / backup > /dev/null
  BAKFILE=/tmp/$HOSTNAME.$DATE.$TS.backup.tar.gz

  if [ -e "$BAKFILE" ] ; then
   if [ "$DELETEDAYS" > 0 ]; then
   upTime=$(date --date="$DELETEDAYS days ago" '+%Y-%m-%dT%T')
   gdrive list -q "name contains 'backup.tar.gz' and name contains '$HOSTNAME' and modifiedTime < '$upTime'" >> drive.txt
   filename='drive.txt'
   # Read the list file
   while read -r line
  do
   theId=$(cut -c-2 <<< "$line")
   #basically skip the first line...
   if [ "$theId" != "Id" ]; then
        echo "Deleting old backup file: "
gdrive delete $(cut -c-33 <<< "$line")
   fi
done < $filename
# Remove temp list file
rm -rf drive.txt
fi


  gdrive upload -p "$DRIVEFOLDERID" --delete "$BAKFILE"
  # ensures removal of backup file from /tmp
  rm -rf "$BAKFILE"
 
  echo "Backup to Google Drive is complete..."
  echo "#############################################"
fi

exit 0



Run this command to activate the automated cron job:
echo 'sh /root/GDbackup.sh' >> /etc/cron.daily/cwp

You will see the backup information in the email that CWP sends when the backup cron has ran, you can also run the script manually:
sh /root/GDbackup.sh

and enjoy your free space...! Hope this helps others..

menigma2

Quote from: daz123 on May 09, 2019, 09:55:48 AM
This is based on a script I use for another Control Panel.

What this script does is backup everything in "/backup" folder (admin backups), and upload it to google drive in tar.gz format, this script will run after the CWP admin backup script has ran.

To use; get the gdrive binary for your OS (https://github.com/gdrive-org/gdrive#downloads).

Centos 6/7 64bit:
wget https://drive.google.com/uc?id=1Ej8VgsW5RgK66Btb9p74tSdHMH3p4UNb&export=download

Centos 6 32-bit
wget https://drive.google.com/uc?id=1eo9hMXz0WyuBwRxPM0LrTtQmhTgOLUlg&export=download

Make it executable:
chmod +x gdrive

Move it to /usr/sbin
mv gdrive /usr/sbin/

Setup:
gdrive about

Follow the link and get the verification code by authenticating with the google account for the drive you want access to.

Add the script into /root directory named: GDbackup.sh:
nano /root/GDbackup.sh

Copy  and paste the below script into the new file using (CTRL C / CTRL V).

And then save the file:
CTRL + X + Y

Script:
#!/bin/bash
IFS=$'\n'

# Copy the folder you want to store your file in at Google Drive (eg. https://drive.google.com/.../folders/...).
DRIVEFOLDERID=

# Days in which to delete the old backups from google drive (set 0 to disable).
DELETEDAYS=180


  echo "#############################################"
  echo "   ---Google Drive Backup---          "
  echo "#############################################"
  gdrive about
  echo "#############################################"

  DATE=$(date '+%d-%m-%Y')
  TS=$(date +'%H%M%S')
  tar -czvf /tmp/$HOSTNAME.$DATE.$TS.backup.tar.gz -C / backup > /dev/null
  BAKFILE=/tmp/$HOSTNAME.$DATE.$TS.backup.tar.gz

  if [ -e "$BAKFILE" ] ; then
   if [ "$DELETEDAYS" > 0 ]; then
   upTime=$(date --date="$DELETEDAYS days ago" '+%Y-%m-%dT%T')
   gdrive list -q "name contains 'backup.tar.gz' and name contains '$HOSTNAME' and modifiedTime < '$upTime'" >> drive.txt
   filename='drive.txt'
   # Read the list file
   while read -r line
  do
   theId=$(cut -c-2 <<< "$line")
   #basically skip the first line...
   if [ "$theId" != "Id" ]; then
        echo "Deleting old backup file: "
gdrive delete $(cut -c-33 <<< "$line")
   fi
done < $filename
# Remove temp list file
rm -rf drive.txt
fi


  gdrive upload -p "$DRIVEFOLDERID" --delete "$BAKFILE"
  # ensures removal of backup file from /tmp
  rm -rf "$BAKFILE"
 
  echo "Backup to Google Drive is complete..."
  echo "#############################################"
fi

exit 0



Run this command to activate the automated cron job:
echo 'sh /root/GDbackup.sh' >> /etc/cron.daily/cwp

You will see the backup information in the email that CWP sends when the backup cron has ran, you can also run the script manually:
sh /root/GDbackup.sh

and enjoy your free space...! Hope this helps others..


Hello,
Thank you for your nomination.
Here when you try to run the script this error is reported.

Failed to get file: googleapi: Error 400: Invalid field selection appProperties, invalidParameter

with someone else, did the same thing happen?

daz123

Hi,

Currently I do not have en copy of CWPanel installed, although I am running a control panel which is based on above script and should work with CWP aswell.

Did you install the gdrive module?

#!/bin/bash
#
# The function uploads the backup directory
# to Google Drive in tar.gz format

# Copy the Drive folder ID where you'll upload your files (the after the https://drive.google.com/.../folders/...).
DRIVEFOLDERID=''

# Days in which to delete the old backups from google drive (set 0 to disable).
DELETEDAYS=30

# PASSWORD FOR ENCRYPTION (leave blank 2 disable)
PASSWORD=''

  echo "================================"
  gdrive about

  DATE=$(date '+%d-%m-%Y')
  TS=$(date +'%H%M%S')

  BAKFILE=/tmp/$HOSTNAME.$DATE.$TS.backup.tar.gz
  ENC_FILE=/tmp/$HOSTNAME.$DATE.$TS.backup.tar.gz.gpg
  tar -czvf "$BAKFILE" -C / backup > /dev/null

  if [ "$password" != '' ] ; then
    gpg -c --batch --yes --passphrase "$PASSWORD" "$BAKFILE"
  fi
 
  if [ -e "$BAKFILE" ] ; then
   if [ "$DELETEDAYS" > 0 ]; then
   upTime=$(date --date="$DELETEDAYS days ago" '+%Y-%m-%dT%T')
   gdrive list -q "name contains 'backup.tar.gz' and name contains '$HOSTNAME' and modifiedTime < '$upTime'" >> drive.txt
   filename='drive.txt'
   # Read the list file
   while read -r line
  do
   theId=$(cut -c-2 <<< "$line")
   #basically skip the first line...
   if [ "$theId" != "Id" ]; then
        echo "Deleting old backup file: "
gdrive delete $(cut -d ' ' -f1 <<< "$line")
   fi
   done < $filename
   # Remove temp list file
    rm -rf drive.txt
   fi

  if [ "$password" != '' ] ; then
    gdrive upload -p "$DRIVEFOLDERID" --delete "$ENC_FILE"
  else
    gdrive upload -p "$DRIVEFOLDERID" --delete "$BAKFILE"
  fi
 
  # ensures removal of backup file from /tmp
  rm -rf "$BAKFILE"

  if [ "$password" != '' ] ; then
    rm -rf "$ENC_FILE"
  fi
 
  echo "Google Drive backup complete ..."
  echo "--------------------------------"
fi

exit 0


Daz

ring_c

Quote from: daz123 on July 02, 2019, 03:39:43 AM
Hi,

Currently I do not have en copy of CWPanel installed, although I am running a control panel which is based on above script and should work with CWP aswell.
Just out of curiosity, which control panel are you now using instead of CWP?


daz123

I use VestaCP on Ubuntu 18.04, the VestaCP requires more 'manual' setup than CWP, I wouldn't say it's for beginners but it's 100% open source. Personally, I will not run code that I cannot see, hence I moved from CWP.

ring_c

Quote from: daz123 on July 02, 2019, 02:14:31 PM
I use VestaCP on Ubuntu 18.04, the VestaCP requires more 'manual' setup than CWP, I wouldn't say it's for beginners but it's 100% open source. Personally, I will not run code that I cannot see, hence I moved from CWP.
I've checked it once. I remember it to be pretty simple.
Actually I have another server running ISPConfig for a single site and it's a no hassle control panel. But for my reseller setup, I find CWP to be the best one with most features.

menigma2

Quote from: daz123 on July 02, 2019, 03:39:43 AM
Hi,

Currently I do not have en copy of CWPanel installed, although I am running a control panel which is based on above script and should work with CWP aswell.

Did you install the gdrive module?

#!/bin/bash
#
# The function uploads the backup directory
# to Google Drive in tar.gz format

# Copy the Drive folder ID where you'll upload your files (the after the https://drive.google.com/.../folders/...).
DRIVEFOLDERID=''

# Days in which to delete the old backups from google drive (set 0 to disable).
DELETEDAYS=30

# PASSWORD FOR ENCRYPTION (leave blank 2 disable)
PASSWORD=''

  echo "================================"
  gdrive about

  DATE=$(date '+%d-%m-%Y')
  TS=$(date +'%H%M%S')

  BAKFILE=/tmp/$HOSTNAME.$DATE.$TS.backup.tar.gz
  ENC_FILE=/tmp/$HOSTNAME.$DATE.$TS.backup.tar.gz.gpg
  tar -czvf "$BAKFILE" -C / backup > /dev/null

  if [ "$password" != '' ] ; then
    gpg -c --batch --yes --passphrase "$PASSWORD" "$BAKFILE"
  fi
 
  if [ -e "$BAKFILE" ] ; then
   if [ "$DELETEDAYS" > 0 ]; then
   upTime=$(date --date="$DELETEDAYS days ago" '+%Y-%m-%dT%T')
   gdrive list -q "name contains 'backup.tar.gz' and name contains '$HOSTNAME' and modifiedTime < '$upTime'" >> drive.txt
   filename='drive.txt'
   # Read the list file
   while read -r line
  do
   theId=$(cut -c-2 <<< "$line")
   #basically skip the first line...
   if [ "$theId" != "Id" ]; then
        echo "Deleting old backup file: "
gdrive delete $(cut -d ' ' -f1 <<< "$line")
   fi
   done < $filename
   # Remove temp list file
    rm -rf drive.txt
   fi

  if [ "$password" != '' ] ; then
    gdrive upload -p "$DRIVEFOLDERID" --delete "$ENC_FILE"
  else
    gdrive upload -p "$DRIVEFOLDERID" --delete "$BAKFILE"
  fi
 
  # ensures removal of backup file from /tmp
  rm -rf "$BAKFILE"

  if [ "$password" != '' ] ; then
    rm -rf "$ENC_FILE"
  fi
 
  echo "Google Drive backup complete ..."
  echo "--------------------------------"
fi

exit 0


Daz

Hello people.
Thanks for all the help and attention.
I found my error was that I did not fill the field DRIVEFOLDERID =
After filling properly it worked perfect.
Thank you for sharing your scritp.

timimi

Quote from: daz123 on May 09, 2019, 09:55:48 AM
This is based on a script I use for another Control Panel.

What this script does is backup everything in "/backup" folder (admin backups), and upload it to google drive in tar.gz format, this script will run after the CWP admin backup script has ran.

To use; get the gdrive binary for your OS (https://github.com/gdrive-org/gdrive#downloads).

Centos 6/7 64bit:
wget https://drive.google.com/uc?id=1Ej8VgsW5RgK66Btb9p74tSdHMH3p4UNb&export=download

Centos 6 32-bit
wget https://drive.google.com/uc?id=1eo9hMXz0WyuBwRxPM0LrTtQmhTgOLUlg&export=download

Make it executable:
chmod +x gdrive

Move it to /usr/sbin
mv gdrive /usr/sbin/

Setup:
gdrive about

Follow the link and get the verification code by authenticating with the google account for the drive you want access to.

Add the script into /root directory named: GDbackup.sh:
nano /root/GDbackup.sh

Copy  and paste the below script into the new file using (CTRL C / CTRL V).

And then save the file:
CTRL + X + Y

Script:
#!/bin/bash
IFS=$'\n'

# Copy the folder you want to store your file in at Google Drive (eg. https://drive.google.com/.../folders/...).
DRIVEFOLDERID=

# Days in which to delete the old backups from google drive (set 0 to disable).
DELETEDAYS=180


  echo "#############################################"
  echo "   ---Google Drive Backup---          "
  echo "#############################################"
  gdrive about
  echo "#############################################"

  DATE=$(date '+%d-%m-%Y')
  TS=$(date +'%H%M%S')
  tar -czvf /tmp/$HOSTNAME.$DATE.$TS.backup.tar.gz -C / backup > /dev/null
  BAKFILE=/tmp/$HOSTNAME.$DATE.$TS.backup.tar.gz

  if [ -e "$BAKFILE" ] ; then
   if [ "$DELETEDAYS" > 0 ]; then
   upTime=$(date --date="$DELETEDAYS days ago" '+%Y-%m-%dT%T')
   gdrive list -q "name contains 'backup.tar.gz' and name contains '$HOSTNAME' and modifiedTime < '$upTime'" >> drive.txt
   filename='drive.txt'
   # Read the list file
   while read -r line
  do
   theId=$(cut -c-2 <<< "$line")
   #basically skip the first line...
   if [ "$theId" != "Id" ]; then
        echo "Deleting old backup file: "
gdrive delete $(cut -c-33 <<< "$line")
   fi
done < $filename
# Remove temp list file
rm -rf drive.txt
fi


  gdrive upload -p "$DRIVEFOLDERID" --delete "$BAKFILE"
  # ensures removal of backup file from /tmp
  rm -rf "$BAKFILE"
 
  echo "Backup to Google Drive is complete..."
  echo "#############################################"
fi

exit 0



Run this command to activate the automated cron job:
echo 'sh /root/GDbackup.sh' >> /etc/cron.daily/cwp

You will see the backup information in the email that CWP sends when the backup cron has ran, you can also run the script manually:
sh /root/GDbackup.sh

and enjoy your free space...! Hope this helps others..

Thanks for this tips.

But I see this not make cron each day.  how to see tach cron? Another thing i see the backup don't take the email account. How to add email account in the backup. 

Thanks