https://lifeandshell.com/tags/web/ Recent content in Web on Life and Shell Hugo -- gohugo.io en-us Mattias Hemmingssion mattias@lifeandshell.com Thu, 01 Dec 2022 13:17:35 +0000 https://lifeandshell.com/posts/migrate-elasticsearch-helm-to-elasticsearch-operator/ Thu, 01 Dec 2022 13:17:35 +0000 https://lifeandshell.com/posts/migrate-elasticsearch-helm-to-elasticsearch-operator/ Migrate elasticsearch helm to elasticsearch operator and from version 7 to version 8. So in the start, I used the helm chart for elasticsearch, and everything worked fine. Then elasticsearch 8 comes and the Elasticsearch operator. This broke by helm chart and kind of left me in a stalled state. But now I have to migrate my current elasticsearch that uses a helm chart to start using the operator. https://lifeandshell.com/posts/openstreat-map-docker-och-docker-compose/ Thu, 17 Nov 2022 16:53:00 +0000 https://lifeandshell.com/posts/openstreat-map-docker-och-docker-compose/ Split up in separate containers ! Running openstreetmap map in docker was hard. And the docs all wanted to run it bounded with postgress and not in separate containers. I setup so we can run osm I different containers for you to scale https://github.com/mattiashem/osm Clone this GitHub repo and then start it with docker compose build then to start it, run docker compose up What is happening First we are building a custom Postgres docker image. https://lifeandshell.com/posts/kubernetes-update-1-22-1-23-helm-error/ Thu, 15 Sep 2022 16:02:06 +0000 https://lifeandshell.com/posts/kubernetes-update-1-22-1-23-helm-error/ I was in the progress of updating my cluster and in version 1.23 we have breaking changes. What I did not know was that helm saves the latest deployed version in secret. So I updated the cluster to version 1.23 and started getting helm errors. And it does not matter if I delete the resources in the cluster. The issue is that helm has saved the last deployment with a API version that with the new k8s version is no longer supported. https://lifeandshell.com/posts/recover-sql-innodb-database/ Sun, 04 Feb 2018 21:20:20 +0000 https://lifeandshell.com/posts/recover-sql-innodb-database/ How to recover an SQL innodb db with docker. When I moved this wordpress to it new hosting a did not have any good backup of the db. And i only got the mysql files from /var/lib/mysql.   So to get the site back without to much work I want to see if I could get the sql files mounted into a mysql docker and recverd to the export a . https://lifeandshell.com/posts/alexa-jenkins-docker/ Sat, 26 Nov 2016 22:05:28 +0000 https://lifeandshell.com/posts/alexa-jenkins-docker/ So I have an Alexa echo dot at home. Use it to control stuff but I wanted it to do more like release and deploy the stuff I build. This is how you can integrate Alexa voice service with Jenkins. First setup the server For receiving commands from Alexa and sending them to Jenkins we need a server and some code. First start with the server i use docker and a docker-compose to set it up. https://lifeandshell.com/posts/autodeploy-docker-images-aws-git-push-deploy/ Thu, 26 May 2016 12:56:19 +0000 https://lifeandshell.com/posts/autodeploy-docker-images-aws-git-push-deploy/ So I have a lot of small project and some large. To buil in quality into my code i need to run test in my code. And my code in a prod like env. I always uses docker so my dev env are verly like my prod. One key thing that i do is that when i push code to my master branch i do a release do server. This is so that i can verify that everything is working and i can run test on it. https://lifeandshell.com/posts/wordpress-multisite-wordpress-singelsite-easy-linux/ Fri, 15 Apr 2016 22:15:11 +0000 https://lifeandshell.com/posts/wordpress-multisite-wordpress-singelsite-easy-linux/ So i hade to slip up my wordpress multisite to singel sites and it was not that hard when i found out how. First start with setting up the new wordpress and then we migrate over the old wordpress site into the new. 1. Setup the new wordpress site Install and setup the new wordpress site. You can run the instalation we will clean out the instalaltion later. 2. In the old multisite find the site id. https://lifeandshell.com/posts/maxscale-sql-scaling-with-mariadb-cluster-in-docker/ Thu, 28 Jan 2016 22:31:46 +0000 https://lifeandshell.com/posts/maxscale-sql-scaling-with-mariadb-cluster-in-docker/ So scaling sql server has now bean easy with mariadb maxscale. Here i uses it to connect to my mariadb cluster and setup two new servers. One is a loadbalanser and onw is a read/write splitter 1.First prep your mariadb servers with som users for you maxscale CREATE user 'maxscale'@'%' identified by 'maxscaleW222'; GRANT SELECT ON mysql.user TO 'maxscale'@'%'; GRANT SELECT ON mysql.db TO 'maxscale'@'%'; GRANT SHOW DATABASES ON *.* TO 'maxscale'@'%';   https://lifeandshell.com/posts/mariadb-cluster-with-dynamic-nodes-in-docker/ Wed, 27 Jan 2016 13:45:55 +0000 https://lifeandshell.com/posts/mariadb-cluster-with-dynamic-nodes-in-docker/ So running sql in docker is a big qestion now. To make some test i have setup two mariadb cluster docker containers. The first one is the mariadb cluster master. This will setup a master mariadb sql node running. The second one is the MariaDB cluster slave. This docker will connect to the master and rsync the database over to the slave. Then en database is rsynced over it will start the sql and can process sql data. https://lifeandshell.com/posts/openldap-with-sql-backend-mariadb/ Thu, 21 Jan 2016 15:59:14 +0000 https://lifeandshell.com/posts/openldap-with-sql-backend-mariadb/ We use Ldap for handling our users and I have spent time setting up Openldap and tryng to configur it. But now i have given up my ldap skills and setup my openldap to use a sql backend and then i config my user with SQL that i like more. I have also build i Dockerfile for docker that you can use.     So what you need is one sql databserver to hold that database, One odbc connection from the ldap server to that sql server. https://lifeandshell.com/posts/mesos-cluster-with-marathon-running-docker/ Fri, 11 Dec 2015 21:47:19 +0000 https://lifeandshell.com/posts/mesos-cluster-with-marathon-running-docker/ Hi So for hosting docker in large scale i have tested mesos cluster. Here is a guide for setting up 3 nodes in mesos running Centos 7. And the adding Marathon to controll the dockers running. The network mesos-master 172.0.0.10 mesos-slave1 172.0.0.11 mesos-slave2 172.0.0.12   The node also have on nic connect to the network with internet access.   Security For this guide stop iptables and turn selinux off setenforce 0 systemect stop firewalld   https://lifeandshell.com/posts/elasticsearch-controller/ Fri, 06 Nov 2015 15:23:27 +0000 https://lifeandshell.com/posts/elasticsearch-controller/ So we uses alot of easticsearch. And here is i small script to get status and do some simple task with es server. You can get cluster status and cron for index deletions.   import urllib2 # # # Clean up elastich search index by removing old stuff. #The defult ip to es server dhost='10.101.1.31' #The index name you are using index_name='logstash-syslog' #Drop index back in time drop_index_back=90 def date_back_in_time(days_back): ''' Get the date back in time the days you send in ''' import datetime as DT today = DT. https://lifeandshell.com/posts/python-dos-protection-iptablesdos/ Fri, 06 Nov 2015 15:18:51 +0000 https://lifeandshell.com/posts/python-dos-protection-iptablesdos/ here are a small script I use to have some sort of dos protection on my webservers.   import subprocess whitelist=['192.168.1.2'] blockvalue=2 alertvalue=1 proc = subprocess.Popen("netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n", shell=True,stdout=subprocess.PIPE) running = proc.stdout.read() runing_sorted = running.split('\n') for r in runing_sorted: con =r.split() if len(con) ==2: #If ip has more conenctions then block value ip block if con[0] <= blockvalue: print " https://lifeandshell.com/posts/foreman-provision-to-bare-and-libvirtd-centos7-foreman-libvirtd-kvm/ Sun, 05 Jul 2015 21:26:46 +0000 https://lifeandshell.com/posts/foreman-provision-to-bare-and-libvirtd-centos7-foreman-libvirtd-kvm/ So I have started to play around with foreman and to get it to provision my diffrent servers. I started by starting up some local virtual servers on my laptop and played around with them. The flow is i started installing foreman as a virtual server. Then i provisin a new virtual server as bare matal (I created a virtual server in virsh) ater that virtual server is prevision i installed it as a virtual host(kvm on kvm) and connected it to foreman so foreman kan provision kvm host. https://lifeandshell.com/posts/installing-go-build-server-on-centos-7/ Wed, 11 Feb 2015 21:39:35 +0000 https://lifeandshell.com/posts/installing-go-build-server-on-centos-7/ Installing the go build server in centos 7 with some easy step     1. First head over to the go page and have a look around http://www.go.cd/ 2. Download go server to you centos box   wget http://download.go.cd/gocd-rpm/go-server-14.4.0-1356.noarch.rpm wget http://download.go.cd/gocd-rpm/go-agent-14.4.0-1356.noarch.rpm 3. Install it First start by adding the go user (something broken in install) useradd go Now run yum localinstall to install local packages yum install java-1.7.0-openjdk -y yum localinstall go-server-14. https://lifeandshell.com/posts/hosting-you-private-docker-repo/ Wed, 11 Feb 2015 16:41:19 +0000 https://lifeandshell.com/posts/hosting-you-private-docker-repo/ We are staring using docker in our developing process. and in that we need to have our own docker repo for hosting our private docker. The path is [public docker cloud(centos img)] –public docker image- > [jenkins build our code and docker img] —> our docker images —> [private docker repo]–our docker image->[Servers [int,qa,prod]   1. Setting up the docker imaged for the docker repo make a folder that will hold you data https://lifeandshell.com/posts/getting-django-docker-prod-ready-with-jenkins-part-1-the-build/ Sun, 18 Jan 2015 21:25:59 +0000 https://lifeandshell.com/posts/getting-django-docker-prod-ready-with-jenkins-part-1-the-build/ So i have some django webb projects and now its time to get my django apps prod ready with docker. My plan is to with jenkins build my django apps (soon start a docker of the app and run some test but that will be later) make a docker image and send that to the docker cloud. Then a can download the docker image on my prod server and start the app. https://lifeandshell.com/posts/installing-openstack-centos-7/ Fri, 03 Oct 2014 14:45:31 +0000 https://lifeandshell.com/posts/installing-openstack-centos-7/ Time to install Openstack on an Centos 7 server. This was my first meeting with Openstack and it took some time for gettings things up. This would be a beginners guide to get you first server up and running. I followed this page in my installation   https://openstack.redhat.com/Neutron_with_existing_external_network   My Openstack server has one NIC connect to my DMZ network and then routed out.   First install a Centos 7 minimal server and setup network   https://lifeandshell.com/posts/installing-jenkins-on-centos-7/ Fri, 03 Oct 2014 14:43:32 +0000 https://lifeandshell.com/posts/installing-jenkins-on-centos-7/ So guide how to get jenkins up and running on centos 7 1. First install it ! yum install -y wget sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key sudo yum install jenkins 2. Install java sudo yum install java-1.7.0-openjdk   3. Open firewall   firewall-cmd --zone=public --add-port=8080/tcp --permanent firewall-cmd --reload systemctl enable firewalld systemctl start firewalld systemctl status firewalld   4. start it ! sudo /etc/init.d/jenkins restart systemctl restart jenkins. https://lifeandshell.com/posts/dyndns-to-loopia-se-to-update-you-domain-dynamic/ Sun, 06 Jul 2014 21:37:17 +0000 https://lifeandshell.com/posts/dyndns-to-loopia-se-to-update-you-domain-dynamic/ So many of my dominas I have registered on loopia.se. And they have dyndns support so I can create a subdomain to my domian. And have it updated when my laptop ore home ip changes. This make the task of connecting back to my home server easy. First install the dyndns clinet on you host here I’m installing it on my Centos 6 server with EPEL REPO installed   yum install ddclient   https://lifeandshell.com/posts/install-pandora-fms-monitoring-system-on-centos/ Sat, 22 Mar 2014 13:10:54 +0000 https://lifeandshell.com/posts/install-pandora-fms-monitoring-system-on-centos/ So for many years i use nagios to monitor my server and now im would say i can handle nagios config files good. But I fund pandora fms monitoring and this i must try. From the pandora console its mutch easy to from the webbrowser setup new task and tweek task so you alarms realy are correct. Doing this in nagios then i had to change config files and restart nagios and nrpe. https://lifeandshell.com/posts/open-webbmail-rainloop-installation-and-setup/ Sat, 15 Mar 2014 11:00:59 +0000 https://lifeandshell.com/posts/open-webbmail-rainloop-installation-and-setup/ So I have testet so many differnt webbbased email programs. And have not been 100% happy with any of them. some are to big other look really bad. (Rainloop is open for non profit companies 🙂 ) But now i found one that I hope i can like some boor Rainloop http://rainloop.net/ It looks nice and are realy easy to install and setup. Here is how I installe if for my domain. https://lifeandshell.com/posts/protecting-you-web-with-modsecurity-on-centos/ Tue, 04 Mar 2014 22:00:40 +0000 https://lifeandshell.com/posts/protecting-you-web-with-modsecurity-on-centos/ So it you worry about you webb then modsecurity is rely nice to have on your webbserver. I have it installed on my apache server with the regular rules from OWAS and also some rules for my own sites. But here is also how to install it.   1. Download and build modsec on your server Add some packages yum install gcc make yum install libxml2 libxml2-devel httpd-devel pcre-devel curl-devel Go to http://www. https://lifeandshell.com/posts/setup-syco-on-you-centos-box/ Tue, 18 Feb 2014 15:27:04 +0000 https://lifeandshell.com/posts/setup-syco-on-you-centos-box/ So if you care about security and stability you must have syco installed on your server. Read more about syco on the github project https://github.com/systemconsole Im staring to use syco not only production but also on my “Own” server. So more of you should really start using it and here is i guide for you to start using syco 1. Installing and setting up centos yum install git   Gettings syco https://lifeandshell.com/posts/installing-asylguiden-on-centos-server/ Mon, 17 Feb 2014 21:21:20 +0000 https://lifeandshell.com/posts/installing-asylguiden-on-centos-server/ One of my own prodjects are Asylguiden. Its A python publish system build with django, Mysql and mongodb. You can find the code here on github https://github.com/mattiashem/asylguiden Asylguiden also works with wsgi for python and apache for displaying content here is my own how to for downloadning and setting up asylguiden on a production server. 1. Setting up server for hosing Centos yum install httpd mod_ssl git wget python-setuptools mod_wsgi   https://lifeandshell.com/posts/blocking-unwanted-traffic-ddosscrapers-apache-iptables/ Tue, 11 Feb 2014 23:16:22 +0000 https://lifeandshell.com/posts/blocking-unwanted-traffic-ddosscrapers-apache-iptables/ So spent last evning blocking ip comming from packetflip to our server. Looks in our Apache access log that there was some evil scraping going on so we started blocking. But its not that funny to block many ip manually so time for some scripts.   First some info to use Packetflip user agent was Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3. https://lifeandshell.com/posts/apache-strong-ssl-config/ Sun, 19 Jan 2014 22:46:53 +0000 https://lifeandshell.com/posts/apache-strong-ssl-config/ So only enable SSL on Apache is not good enough there are some config to add to apache to make it stronger.   This are the setting i use in my apache ssl configs. SSLEngine On SSLCertificateFile /etc/apache2/ssl/apache.pem SSLCertificateKeyFile /etc/apache2/ssl/apache.key Header add Strict-Transport-Security "max-age=15768000" SSLCompression off SSLUseStapling on SSLStaplingResponderTimeout 5 SSLStaplingReturnResponderErrors off SSLStaplingCache shmcb:/var/run/ocsp(128000) SSLProtocol All -SSLv2 -SSLv3 SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4 And for generating you cert I use openssl req -new -x509 -days 365 -nodes -out /etc/apache2/ssl/apache. https://lifeandshell.com/posts/install-and-setup-haystack-search-for-django/ Sun, 05 Jan 2014 22:28:01 +0000 https://lifeandshell.com/posts/install-and-setup-haystack-search-for-django/ So Mysql is crap at doing full text search. So in one of my projects i use Haystack so i can do full text searches. I have a running Django project up and this is how I setup haystack for my project.   Install and config sudo pip install django-haystack   in settings.py under INSTALLED_APPS add haystack 'haystack',   And also in settings.py file add some haystack settings import os HAYSTACK_CONNECTIONS = { 'default': { 'ENGINE': 'haystack. https://lifeandshell.com/posts/install-elgg-social-network-on-centos/ Thu, 26 Dec 2013 16:26:57 +0000 https://lifeandshell.com/posts/install-elgg-social-network-on-centos/ Elgg is a social network web application that could e nice as intranet for companies. Well Its a PHP application so its easy to install   First some yum packages yum install mysql mysql-server httpd php php-mysql php-gd php-imap php-ldap php-odbc php-pear php-xml php-xmlrpc php-mbstring wget unzip Setup an apache config <VirtualHost *:80> DocumentRoot /var/www/html/elinodrift.se ServerName domain.se ServerAlias www.yourdomain.se ServerAdmin webmaster@domain.se ErrorLog /var/log/httpd/elgg.log <Directory /var/www/html/elgg> Options FollowSymLinks AllowOverride All Order allow,deny Allow from all </Directory> </VirtualHost> Setup mysql https://lifeandshell.com/posts/apache-performance-config/ Mon, 16 Dec 2013 21:02:29 +0000 https://lifeandshell.com/posts/apache-performance-config/ Now on all my Apache i always load this Apache config. It enabled some apache standard performance config for Apache as a good standard. KeepAlive. Gzip all transfer and local disk cache   my /etc/httpd/cond.f/01.conf NameVirtualHost *:80 NameVirtualHost *:443 #Speedning upp webres Apache config # 2 HOURS <FilesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|js|css|swf)$"> Header set Cache-Control "max-age=7200, public" </FilesMatch> # 1 HOUR <FilesMatch "\.(xml|txt)$"> Header set Cache-Control "max-age=3600, public, must-revalidate" </FilesMatch> # 2 HOURS <FilesMatch " https://lifeandshell.com/posts/fail2ban-on-centos/ Mon, 16 Dec 2013 20:58:10 +0000 https://lifeandshell.com/posts/fail2ban-on-centos/ Fail2Ban is a small service to block unwanted traffic to you server. I use it to block ssh,and postfix loggins in to my virtual hosts. Fail2Ban scans the service loggfiles and if it find any strange traffik like ssh bruteforce. That ip will be blocket for some time. All settings are done in /etc/fail2ban/ folder. Install Have epel repo aktivated on server tha run yum install fail2ban Then do your local config in https://lifeandshell.com/posts/install-diaspora-one-centos-6-4-with-apache/ Sun, 24 Nov 2013 21:25:42 +0000 https://lifeandshell.com/posts/install-diaspora-one-centos-6-4-with-apache/ So Im going to test diaspora on one of my virtual server with run centos 6.4. Setup Centos Setup Repos wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm rpm -Uvh remi-release-6*.rpm epel-release-6*.rpm" Install packages yum install tar make automake gcc gcc-c++ git net-tools libcurl-devel libxml2-devel libffi-devel libxslt-devel tcl redis ImageMagick npm mysql-server mysql-devel httpd mod_ssl libyaml libyaml-devel patch readline-devel libtool bison Start services chkconfig --level 3 httpd on chkconfig --level 3 mysqld on chkconfig --level 3 redis on   https://lifeandshell.com/posts/custom-nagios-plugins-in-python/ Tue, 05 Nov 2013 15:20:51 +0000 https://lifeandshell.com/posts/custom-nagios-plugins-in-python/ For monitoring different service and function you may need to build some custom monitoring plugins. I have some build for nrpe and will work with both nagios and icinga. This script will do and mysql check and then send the data back and also start graphing the data back if you use pnp4nagios 🙂 Every plugin must have two things. 1. an exit code thet will say the state of the plugin (OK. https://lifeandshell.com/posts/owncloud-5-on-centos-6-4-apache-mysql/ Thu, 10 Oct 2013 13:33:35 +0000 https://lifeandshell.com/posts/owncloud-5-on-centos-6-4-apache-mysql/ How to install owncloud 5 on you centos 6.4 server with mysql and apache to serve it. First install packages and service needed. yum -y install mysql-server httpd php php-mysql unzip wget php-json php-xml php-mbstring php-zip php-gd curl php-curl php-pdo mod_ssl Set apache and mysql to start at boot chkconfig httpd on chkconfig mysqld on Start them up /etc/init.d/httpd start /etc/init.d/mysqld start Make a new file called /tmp/setup_owncloud.sql and put this in the file (Ore past it in the mysql shell) https://lifeandshell.com/posts/glassfish-monitoring-with-visualvm/ Wed, 25 Sep 2013 14:45:28 +0000 https://lifeandshell.com/posts/glassfish-monitoring-with-visualvm/ For monitoring Glassfish performance i use VisualVM. I have visual installed on my laptop and the connects using jmx to my glassfish servers to get server stats. This is only to get the current data and to se how mutch memory my apps are using and so on.   1. Download and start VisualVM Go here and download VisualVM http://visualvm.java.net/ Install visual on you local computer.   2. Set up Glassfish for reciving JMX connections from external ip On your glassfish you need to add som jvm values so in your server-config –> jvm-settings –> JVM options add the following. https://lifeandshell.com/posts/mysql-commands-to-remember/ Thu, 29 Aug 2013 08:45:06 +0000 https://lifeandshell.com/posts/mysql-commands-to-remember/ This is an reminder for mw some mysql commands that i use often and my mind not always bring with me.   Optimize table When i table that has many writes and delets get fregmant this will speed up the database. optimize table Sys   Creating users CREATE USER 'dbuser'@'localhost' IDENTIFIED BY 'some_pass'; GRANT ALL PRIVILEGES ON *.* TO 'dbuser'@'localhost' FLUSH PRIVILEGES; Adding Mysql Monitoring User GRANT SELECT, REPLICATION CLIENT, SHOW DATABASES, SUPER, PROCESS ON *. https://lifeandshell.com/posts/lvs-cluster-for-centos/ Thu, 01 Aug 2013 14:08:13 +0000 https://lifeandshell.com/posts/lvs-cluster-for-centos/ An other cluster solution for Linux is LVS. I im testing to use LVS cluster for some cloud server. My cloudserver has one external ip and i want all traffic to come to that ip and after that be redirected to my web nodes. Witch LVS i will redirect all traffic to that ip and load balance it between my nodes. When i set up HAProxy i only loadbalanse webb traffic. https://lifeandshell.com/posts/install-ha-proxy-for-load-balansing-on-centos/ Tue, 30 Jul 2013 20:25:36 +0000 https://lifeandshell.com/posts/install-ha-proxy-for-load-balansing-on-centos/ For load balasing my weebtraffic im setting up HA-proxy. The proxy recives reqest on one ip and then even loads the reqest between my web server nodes. First install and enable Epel repo yum install haproxy open the configfile /etc/haproxy/haproxt.cfg and ad to the buttom of the file listen http_web 192.168.44.20:80 mode http balance roundrobin # Load Balancing algorithm option httpchk option forwardfor server server1 192.168.44.21:80 weight 1 maxconn 512 check server server2 192.