Merge pull request #939 from einervonvielen/Debian9

Debian9 - Changes for Debian 9 and several fixes

3 files changed, 119 insertions, 99 deletions

diff --git a/.homeinstall/README.md b/.homeinstall/README.md
index c0de914a1..e78f78fc6 100644
--- a/.homeinstall/README.md
+++ b/.homeinstall/README.md
@@ -2,10 +2,19 @@
 
 Run hubzilla-setup.sh for an unattended installation of hubzilla.
 
-The script is known to work with Debian 8.3 stable (Jessie)
+The script is known to work without adjustments with
 
-+ Home-PC (Debian-8.3.0-amd64)
-+ DigitalOcean droplet (Debian 8.3 x64 / 512 MB Memory / 20 GB Disk / NYC3)
++ Hardware
+  - Mini-PC with Debian-9.2-amd64, or
+  - Rapberry 3 with Raspbian, Debian-9.3
++ DynDNS
+  - selfHOST.de
+  - freedns.afraid.org
+
+## Disclaimers
+
+- This script does work with Debian 9 only.
+- This script has to be used on a fresh debian install only (it does not take account for a possibly already installed and configured webserver or sql implementation).
 
 # Step-by-Step Overwiew
 
@@ -14,25 +23,28 @@ The script is known to work with Debian 8.3 stable (Jessie)
 Hardware
 
 + Internet connection and router at home
-+ Mini-pc connected to your router
++ Mini-pc connected to your router (a Raspberry 3 will do for very small Hubs)
 + USB drive for backups
 
 Software
 
-+ Fresh installation of Debian on your mini-pc
++ Fresh installation of Debian 9 (Stretch) on your mini-pc
 + Router with open ports 80 and 443 for your Debian
 
 ## The basic steps (quick overview)
 
 + Register your own domain (for example at selfHOST) or a free subdomain (for example at freeDNS)
-+ Log on to your new debian (server)
++ Install Debian 9
++ On your router: Open the ports 80 and 443
++ Log on to your fresh Debian
   - apt-get install git
   - mkdir -p /var/www
   - cd /var/www
   - git clone https://github.com/redmatrix/hubzilla.git html
   - cp .homeinstall/hubzilla-config.txt.template .homeinstall/hubzilla-config.txt
   - nano .homeinstall/hubzilla-config.txt
-    - Enter your values there: db pass, domain, values for dyn DNS
+    - Read the comments carefully
+    - Enter your values: db pass, domain, values for dyn DNS
   - hubzilla-setup.sh as root
     - ... wait, wait, wait until the script is finised
   - reboot
@@ -51,18 +63,36 @@ The installation will create a daily backup.
 If the backup process does not find an external device than the backup goes to
 the internal disk.
 
-The USB drive must be compatible with an encrpyted filesystem LUKS + ext4.
+The USB drive must be compatible with the filesystems
+
+- ext4 (if you do not want to encrypt the USB) 
+- LUKS + ext4 (if you want to encrypt the USB) 
 
 ## Preparations Software
 
 ### Install Debian Linux on the Mini-PC
 
-Download the stable Debian at https://www.debian.org/
+Download the stable Debian 9 at https://www.debian.org/  
+(Debian 8 is no longer supported.)
+
+Create bootable USB drive with Debian on it. You could use
+
+- unetbootin, https://en.wikipedia.org/wiki/UNetbootin
+- or simply the linux command "dd"
+
+Example for command dd...
 
-Create bootable USB drive with Debian on it. You could use the programm
-unetbootin, https://en.wikipedia.org/wiki/UNetbootin
+    su -
+    dd if=2017-11-29-raspbian-stretch.img of=/dev/mmcblk0
 
-Switch of your mini pc, plug in your USB drive and start the mini pc from the
+Do not forget to unmount the SD card before and check if unmounted like in this example...
+
+    su -
+    umount /dev/mmcblk0*
+    df -h
+
+
+Switch off your mini pc, plug in your USB drive and start the mini pc from the
 stick. Install Debian. Follow the instructions of the installation.
 
 ### Configure your Router
@@ -79,32 +109,27 @@ You can use subdomains as well
 
     my.cooldomain.org
 
-There are two way to get a domain
+There are two ways to get a domain
 
-- buy a domain (recommended) or
+- buy a domain, or
 - register a free subdomain
 
-### Method 1: Get yourself an own Domain (recommended)
+### Method 1: Buy an own Domain 
 
-...for example at selfHOST.de
+...for example buy at selfHOST.de  
 
-### Method 2 Register a (free) Subdomain
+The cost are around 10,- € once and 1,50 € per month (2017).
 
-Register a free subdomain for example at
+### Method 2 Register a (free) Subdomain
 
-- freeDNS
-- selfHOST
+...for example register at freeDNS
 
-WATCH THIS: A free subdomain is not the prefered way to get a domain name. Why?
+Follow the instructions in .homeinstall/hubzilla-config.txt.  
 
-Let's encrpyt issues a limited number of certificates each
-day. Possibly other users of this domain will try to issue a certificate
-at the same day as you do. So make sure you choose a domain with as less subdomains as
-possible.
 
 ## Install Hubzilla on your Debian
 
-Login to your debian
+Login to your Debian
 (Provided your username is "you" and the name of the mini pc is "debian". You
 could take the IP address instead of "debian")
 
@@ -135,7 +160,7 @@ Copy the template file
     
     cp hubzilla-config.txt.template hubzilla-config.txt
 
-Change the file "hubzilla-config.txt". Read the instructions there and enter your values.
+Modify the file "hubzilla-config.txt". Read the instructions there carefully and enter your values.
 
     nano hubzilla-config.txt
 
@@ -146,7 +171,7 @@ Run the script
 Wait... The script should not finish with an error message.
 
 In a webbrowser open your domain.
-Expected: A test page of hubzilla is shown. All checks there shoulg be
+Expected: A test page of hubzilla is shown. All checks there should be
 successfull. Go on...
 Expected: A page for the Hubzilla server configuration shows up.
 
@@ -162,3 +187,23 @@ Leave db type "MySQL" untouched.
 
 Follow the instructions in the next pages.
 
+## Note for the Rasperry 
+
+The script was tested with a Raspberry 3 under Raspian (Debian 9.3, 2017-11-29-raspbian-stretch.img).
+
+Be patient when a page is loaded by your Raspi-Hub for the very first time. Especially the config pages after the install will load very slowly.
+
+It is recommended to deinstall these programms to avoid endless updates. Use...
+
+    sudo apt-get purge wolfram-engine sonic-pi
+    sudo apt-get autoremove
+
+It is recommended to run the Raspi without graphical frontend (X-Server). Use...
+
+    sudo raspi-config
+
+There choose "3 Boot Options" > "31 Desktop / CLI" > "B1 Console". Reboot.
+
+**DO NOT FORGET TO CHANGE THE DEFAULT PASSWORD FOR USER PI!**
+
+
diff --git a/.homeinstall/hubzilla-config.txt.template b/.homeinstall/hubzilla-config.txt.template
index 040b0f2f4..089917c68 100644
--- a/.homeinstall/hubzilla-config.txt.template
+++ b/.homeinstall/hubzilla-config.txt.template
@@ -13,10 +13,10 @@ db_pass=
 # This script automates installation of an SSL certificate from
 # Let's Encrypt (https://letsencrypt.org)
 #
-# Please give the domain name of your hub
+# Please give the domain name of your hub. Examples:
 #
-#     Example: my.cooldomain.org
-#     Example: cooldomain.org
+#     le_domain=my.cooldomain.org
+#     le_domain=cooldomain.org
 #
 # Email is optional
 #
@@ -25,7 +25,7 @@ le_domain=
 le_email=
 
 ###############################################
-### OPTIONAL - selfHOST - dynamic IP address ##
+### OPTIONAL - dynamic IP address - selfHOST ##
 #
 # 1. Register a domain at selfhost.de
 #    - choose offer "DOMAIN dynamisch" 1,50€/mon at 08.01.2016
@@ -42,7 +42,7 @@ selfhost_user=
 selfhost_pass=
 
 ###############################################
-### OPTIONAL - FreeDNS - dynamic IP address ###
+### OPTIONAL - dynamic IP address - FreeDNS ###
 #
 # Please give the alpha-numeric-key of freedns
 #
@@ -70,15 +70,17 @@ selfhost_pass=
 #       freedns_key=U1Z6aGt2R0NzMFNPNWRjbWxxZGpsd093OjE1Mzg5NDE5
 #
 #
-#freedns_key=
+freedns_key=
 
 
 ###############################################
 ### OPTIONAL - Backup to external device ######
 #
 # The script can use an external device for the daily backup.
-# The file system of the device (USB stick for example) must be compatible
-# with encrypted LUKS + ext4
+# The file system of the device (USB stick for example) must be compatible with
+#
+# - encrypted LUKS + ext4, or
+# - ext4
 #
 # You should test to mount the device befor you run the script
 # (hubzilla-setup.sh).
@@ -113,28 +115,22 @@ selfhost_pass=
 #     lsof /media/hubzilla_backup
 #
 # If you leave the following parameters
+#
 # - "backup_device_name" and
 # - "backup_device_pass"
+#
 # empty the script will create daily backups on the internal disk (which could
 # save you as well).
 #
 #   Example: backup_device_name=/dev/sdc1
 #
+# Leave "backup_device_pass=" empty if the external device is not encrypted.
+#
 backup_device_name=
 backup_device_pass=
 
 
 ###############################################
-### OPTIONAL - Owncloud - deprecated ##########
-#
-# To install owncloud: owncloud=y
-# Leave empty if you don't want to install owncloud
-#
-#owncloud=
-
-
-
-###############################################
 ### OPTIONAL - do not mess with things below ##
 #              (...if you are not certain)
 #
@@ -160,18 +156,3 @@ mysqlpass=$db_pass
 #   Example: phpmyadminpass="aber hallo has blanks in it"
 phpmyadminpass=$db_pass
 
-# TODO Prepare hubzilla for programmers
-# - install eclipse and plugins
-# - install xdebug to debug the php with eclipse
-# - weaken permissions on /var/www/html
-# - manual steps after this script
-#   * in eclipse: install plugins for php git hub
-#   * in eclipse: configure firefox (chrome,...) as browser to run with the php debuger
-#   * in eclipse: switch php debugger from zend to xdebug
-#   * in eclipse: add local hubzilla github repository
-#
-# Which user will use eclipse?
-# Leave this empty if you do not want to prepare hubzilla for debugging
-#
-#developer_name=
-
diff --git a/.homeinstall/hubzilla-setup.sh b/.homeinstall/hubzilla-setup.sh
index c83fb8de9..023b9eed1 100755
--- a/.homeinstall/hubzilla-setup.sh
+++ b/.homeinstall/hubzilla-setup.sh
@@ -114,7 +114,11 @@ function check_sanity {
     fi
     if [ ! -f /etc/debian_version ]
     then
-        die "Ubuntu is not supported"
+        die "Debian is supported only"
+    fi
+    if ! grep -q 'Linux 9' /etc/issue
+    then
+        die "Linux 9 (stretch) is supported only"x
     fi
 }
 
@@ -253,11 +257,11 @@ function install_sendmail {
 }
 
 function install_php {
-    # openssl and mbstring are included in libapache2-mod-php5
-    # to_to:  php5-suhosin
+    # openssl and mbstring are included in libapache2-mod-php
     print_info "installing php..."
-    nocheck_install "libapache2-mod-php5 php5 php-pear php5-xcache php5-curl php5-mcrypt php5-gd"
-    php5enmod mcrypt
+    nocheck_install "libapache2-mod-php php php-pear php-curl php-mcrypt php-gd"
+    sed -i "s/^upload_max_filesize =.*/upload_max_filesize = 100M/g" /etc/php/7.0/apache2/php.ini
+    sed -i "s/^post_max_size =.*/post_max_size = 100M/g" /etc/php/7.0/apache2/php.ini
 }
 
 function install_mysql {
@@ -277,18 +281,17 @@ function install_mysql {
     # want to be prompted for it then this can be arranged by preseeding the
     # DebConf database with the required information.
     #
-    #     echo mysql-server-5.5 mysql-server/root_password password xyzzy | debconf-set-selections
-    #     echo mysql-server-5.5 mysql-server/root_password_again password xyzzy | debconf-set-selections
+    #     echo mysql-server mysql-server/root_password password xyzzy | debconf-set-selections
+    #     echo mysql-server mysql-server/root_password_again password xyzzy | debconf-set-selections
     #
     print_info "installing mysql..."
     if [ -z "$mysqlpass" ]
     then
         die "mysqlpass not set in $configfile"
     fi
-    echo mysql-server-5.5 mysql-server/root_password password $mysqlpass | debconf-set-selections
-    echo mysql-server-5.5 mysql-server/root_password_again password $mysqlpass | debconf-set-selections
-    nocheck_install "php5-mysql mysql-server mysql-client"
-    php5enmod mcrypt
+    echo mysql-server mysql-server/root_password password $mysqlpass | debconf-set-selections
+    echo mysql-server mysql-server/root_password_again password $mysqlpass | debconf-set-selections
+    nocheck_install "php-mysql mysql-server mysql-client"
 }
 
 function install_phpmyadmin {
@@ -327,6 +330,7 @@ function install_phpmyadmin {
         echo "Include /etc/phpmyadmin/apache.conf" >> /etc/apache2/apache2.conf
     fi
     service apache2 restart
+    /etc/init.d/mysql start
 }
 
 function create_hubzilla_db {
@@ -511,6 +515,8 @@ END
     then
         die "Failed to load $url_http"
     fi
+    # accept terms of service of letsencrypt
+    ./dehydrated --register --accept-terms
     # run script dehydrated
     # 
     ./dehydrated --cron --config $le_dir/config.sh
@@ -574,7 +580,10 @@ function install_hubzilla {
     chmod -R 777 store
     touch .htconfig.php
     chmod ou+w .htconfig.php
-    install_hubzilla_plugins
+    # uncomment the last function call "install_hubzilla_plugins" 
+    # - if you want to install addons and themes that are not officially supported
+    # - and read the comments in function "install_hubzilla_plugins" how do do it
+    # install_hubzilla_plugins
     cd /var/www/
     chown -R www-data:www-data html
 	chown root:www-data /var/www/html/
@@ -607,7 +616,6 @@ function install_hubzilla_plugins {
         echo "#   cd /var/www/html/.homeinstall" >> $plugin_install
         echo "#   ./hubzilla-setup.sh" >> $plugin_install
         echo "https://gitlab.com/zot/ownmapp.git ownMapp" >> $plugin_install
-        echo "https://gitlab.com/zot/hubzilla-chess.git chess" >> $plugin_install
     fi
     # install plugins
     while read -r line; do
@@ -675,25 +683,19 @@ function rewrite_to_https {
 function install_rsnapshot {
     print_info "installing rsnapshot..."
     nocheck_install "rsnapshot"
-	# internal disk
-    cp -f /etc/rsnapshot.conf $snapshotconfig   
-    sed -i "/hourly/s/retain/#retain/" $snapshotconfig 
-    sed -i "/monthly/s/#retain/retain/" $snapshotconfig 
+    # internal disk
+    cp -f /etc/rsnapshot.conf $snapshotconfig
     sed -i "s/^cmd_cp/#cmd_cp/" $snapshotconfig
     sed -i "s/^backup/#backup/" $snapshotconfig
-    if [ -z "`grep 'letsencrypt' $snapshotconfig`" ]
-    then
-		echo "backup	/var/lib/mysql/	localhost/" >> $snapshotconfig
-		echo "backup	/var/www/html/	localhost/" >> $snapshotconfig
-		echo "backup	/var/www/letsencrypt/	localhost/" >> $snapshotconfig
-    fi
+	echo "backup	/var/lib/mysql/	localhost/" >> $snapshotconfig
+	echo "backup	/var/www/html/	localhost/" >> $snapshotconfig
+	echo "backup	/var/www/letsencrypt/	localhost/" >> $snapshotconfig
 	# external disk
-	if [ -n "$backup_device_name" ] && [ -n "$backup_device_pass" ]
+	if [ -n "$backup_device_name" ]
 	then
 		cp -f /etc/rsnapshot.conf $snapshotconfig_external_device   
 		sed -i "s#snapshot_root.*#snapshot_root	$backup_mount_point#" $snapshotconfig_external_device
-		sed -i "/hourly/s/retain/#retain/" $snapshotconfig_external_device 
-		sed -i "/monthly/s/#retain/retain/" $snapshotconfig_external_device 
+		sed -i "/alpha/s/6/30/" $snapshotconfig_external_device 
 		sed -i "s/^cmd_cp/#cmd_cp/" $snapshotconfig_external_device
 		sed -i "s/^backup/#backup/" $snapshotconfig_external_device
 		if [ -z "`grep 'letsencrypt' $snapshotconfig_external_device`" ]
@@ -767,9 +769,7 @@ echo "        if mount $backup_device_name $backup_mount_point" >> /var/www/$hub
 echo "        then" >> /var/www/$hubzilladaily
 echo "            device_mounted=1" >> /var/www/$hubzilladaily
 echo "            echo \"device $backup_device_name is now mounted. Starting backup...\"" >> /var/www/$hubzilladaily
-echo "			rsnapshot -c $snapshotconfig_external_device daily" >> /var/www/$hubzilladaily
-echo "			rsnapshot -c $snapshotconfig_external_device weekly" >> /var/www/$hubzilladaily
-echo "			rsnapshot -c $snapshotconfig_external_device monthly" >> /var/www/$hubzilladaily
+echo "			rsnapshot -c $snapshotconfig_external_device alpha" >> /var/www/$hubzilladaily
 echo "			echo \"\$(date) - disk sizes...\"" >> /var/www/$hubzilladaily
 echo "			df -h" >> /var/www/$hubzilladaily
 echo "			echo \"\$(date) - db size...\"" >> /var/www/$hubzilladaily
@@ -789,9 +789,7 @@ echo "fi" >> /var/www/$hubzilladaily
 echo "if [ \$device_mounted == 0 ]" >> /var/www/$hubzilladaily
 echo "then" >> /var/www/$hubzilladaily
 echo "    echo \"device could not be mounted $backup_device_name. Using internal disk for backup...\"" >> /var/www/$hubzilladaily
-echo "	rsnapshot -c $snapshotconfig daily" >> /var/www/$hubzilladaily
-echo "	rsnapshot -c $snapshotconfig weekly" >> /var/www/$hubzilladaily
-echo "	rsnapshot -c $snapshotconfig monthly" >> /var/www/$hubzilladaily
+echo "	rsnapshot -c $snapshotconfig alpha" >> /var/www/$hubzilladaily
 echo "fi" >> /var/www/$hubzilladaily
 echo "#" >> /var/www/$hubzilladaily
 echo "echo \"\$(date) - db size...\"" >> /var/www/$hubzilladaily
@@ -801,15 +799,12 @@ echo "# update" >> /var/www/$hubzilladaily
 echo "echo \"\$(date) - updating dehydrated...\"" >> /var/www/$hubzilladaily
 echo "git -C /var/www/letsencrypt/ pull" >> /var/www/$hubzilladaily
 echo "echo \"\$(date) - updating hubhilla core...\"" >> /var/www/$hubzilladaily
-echo "git -C /var/www/html/ pull" >> /var/www/$hubzilladaily
-echo "echo \"\$(date) - updating hubhilla addons...\"" >> /var/www/$hubzilladaily
-echo "git -C /var/www/html/addon/ pull" >> /var/www/$hubzilladaily
-echo "bash /var/www/html/$plugins_update" >> /var/www/$hubzilladaily
+echo "(cd /var/www/html/ ; util/udall)" >> /var/www/$hubzilladaily
 echo "chown -R www-data:www-data /var/www/html/ # make all accessable for the webserver" >> /var/www/$hubzilladaily
 echo "chown root:www-data /var/www/html/.htaccess" >> /var/www/$hubzilladaily
 echo "chmod 0644 /var/www/html/.htaccess # www-data can read but not write it" >> /var/www/$hubzilladaily
 echo "echo \"\$(date) - updating linux...\"" >> /var/www/$hubzilladaily
-echo "apt-get -q -y update && apt-get -q -y dist-upgrade # update linux and upgrade" >> /var/www/$hubzilladaily
+echo "apt-get -q -y update && apt-get -q -y dist-upgrade && apt-get -q -y autoremove # update linux and upgrade" >> /var/www/$hubzilladaily
 echo "echo \"\$(date) - Backup hubzilla and update linux finished. Rebooting...\"" >> /var/www/$hubzilladaily
 echo "#" >> /var/www/$hubzilladaily
 echo "reboot" >> /var/www/$hubzilladaily
@@ -894,7 +889,6 @@ install_run_selfhost
 ping_domain
 configure_cron_freedns
 configure_cron_selfhost
-install_git
 install_letsencrypt
 configure_apache_for_https
 check_https