CHEF HAVEn-Cookbook My SQL Install Guide V1.00 11112014 PDF

Preview

Summary

Interesting chef haven cookbook mysql install guide...

Description

CHEF HAVEn-Cookbook MySQL Install Guide

HP Restricted

Install Guide

DOCUMENT REVISION HISTORY Project Name: Amendment History – Document Status (e.g. Draft, Final, Release#): CR# (Optional)

Note:

Document Version#

Approval Date

0.1

10 Aug 2014

0.2

22 Aug 2014

0.3

5 Sep 2014

Modified By Witold Swierzy Slawomir Adamiec Witold Swierzy

Section, Pages(s) and Text Revised Initial version Recipe split Recipe Merge and Fixing bugs

Template Revision History can be found in Hidden Text.

Template Revision History (Hidden Text) Change Request#

Template Version

Approval Date

Modified By

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Section, Pages(s) and Text Revised

Page 2 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide

Contents 1

Introduction .................................................................................................................. 4 1.1 PURPOSE OF THIS DOCUMENT ................................................................. 4 1.2 PURPOSE OF MYSQL CHEF COOKBOOK .................................................. 4

2 3

General Architecture .................................................................................................... 5 MySQL Chef Cookbook v. 1.0 Requirements ............................................................. 6

4 5

MySQL Chef Cookbook v. 1.0 Content ....................................................................... 7 MySQL Chef Cookbook Setup ..................................................................................... 8

6

MySQL Installation Recipe execution ....................................................................... 11

7. Check master-master replication .................................................................................... 12 8. Ganglia Installation Recipe Execution. .......................................................................... 13

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 3 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide

1 1.1

Introduction PURPOSE OF THIS DOCUMENT

This document describes MySQL Chef Cookbook v 1.0, including its installation, setup, configuration and execution

1.2

PURPOSE OF MYSQL CHEF COOKBOOK

MySQL Chef Cookbook allows for automatic installation and configuration of two-node MySQL Database Symmetric (two-ways) Replication environment.

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 4 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide

2

General Architecture MySQL cookbook consists of a

   

singleoneChefrecipeforMySQLEnterpriseEditioninstallation:mysql_install.rb. SingleoneChefrecipeforGangliaserverandagentinstallation singleoneattributefilewithseveralattributestoset MySQLEnterpriseEditiontrialversionpackages

It also uses a single one Chef role: mysql. At the moment of writing of this manual, we assume, that MySQL installation will always use two nodes, which replicate a single one database. This database is replicated in both directions and both nodes are equal in terms of “importance” (in other words, at the end of succesfull installation we will obtain a fully symmetric, fault-tolerance two-node database system)

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 5 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide

3

MySQL Chef Cookbook v. 1.0 Requirements MySQLcookbookhasseveralrequirements,whichwewouldliketodiscussinthischapter

No

Description

1

Machines: required number of machines is 2, as mentioned in chapter 2 – “My SQL Cookbook General Architecture”

2

Hardware resources (memory, CPU, disk): requirements are the same, as for MySQL Enterprise Edition DBMS

3

Operating system: RedHat Enterprise Linux v. 6

4

Chef version: 11.12.8. Chef must be installed and configured. Machines assigned for MySQL installation must have Chef client installed and configured

5

MySQL Enterprise Edition v. 5.6. This cookbook comes with trial, 30-day version of MySQL server. To build a commercial system, there is need to provide licensed version of this system.

6

root access on all machines assigned for MySQL installation. MySQL cookbook performs several low-level steps, like OS preparation, installation of required packages, etc. which require root access.

7

Both the machines assigned for MySQL installation must be resolvable via DNS. Also, to ensure load balancing, there is need to create a DNS entry, which is resolvable to all the IP addresses of both MySQL machines. In this manual, we assume, that there are the following machines with the following IP addresses and DNS names:

    8

bdp‐1.bdpdomain.com,192.168.110.3–thisisaChefservermachine,andalsoa Kerberosservermachine bdp‐13.bdpdomain.com, 192.168.110.15 – this is the first host for MySQL installation bdp‐14.bdpdomain.com, 192.168.110.16 – this is the second host for MySQL installation mcluster.bdpdomain.com – it is resolvable to 192.168.110.15 and 192.168.110.16.Thisentryensuresloadbalancing.

Additional software:

a. LDAP server available on the local network. Our cookbook sets up MySQL databasetoauthenticateendusersviaLDAP.Please,note,thatLDAPserver isnotinstalledbyMySQLcookbook b. AccesstoayumrepositorywithstandardRHELv.6packages.Thecookbook performsinstallationandsetupofthreeadditionalpackages(iftheyarenot installedyet),whicharerequiredbyMySQLto authenticateusersviaLDAP: openldap‐clients,nss‐pam‐ldapdandpam_ldap. 9

SELINUX has to be disabled on both nodes. In other cases it is possible, that database creation will fail, and it will be impossible to start MySQL system service

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 6 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide

4

MySQL Chef Cookbook v. 1.0 Content

No 1

File(s) Attribute file mysql/attributes/default.rb Content: this file contains attributes, used by the cookbook to install MySQL EE v. 5.6 and create a database. We describe all the attributes in Chapter 5 – “Cookbook setup”

2

Recipes MySQL cookbook contains two recipes : mysql/recipes/mysql_install.rb This recipe performs all the tasks, including OS preparation, installation of required packages, building a replicated database and configuring LDAP authentication for the replicated database..

mysql/recipes/gang_install.rb. This recipe performs all the tasks, which are needed for Ganglia server and agent monitoring system installation. 3

Installation files mysql/files/default/*.rpm MySQL cookbook comes with the whole MySQL Enterprise Edition v. 5.6 set of packages. These packages are during installation process installed on both nodes of MySQL system.

4

PAM configuration file Mysql/files/default/mysql.pam.d To authenticate database users via LDAP, MySQL uses PAM (Pluggable Authentication Modules) system. This file is a main configuration file, which allows PAM to work with MySQL

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 7 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide

5

MySQL Chef Cookbook Setup MySQLcookbooksetupconsistsoffewsimplesteps,whichwewouldliketodiscuss inthis chapter. STEP1:unpack mysql‐cookbook.tarfilein chef‐repo/cookbooks directory on a machine with Chef server running. It can be done, using the following command (we assume, that the currentdirectoryischef‐repo/cookbooks):  $ cd /chef-repo/cookbooks  $ tar –xvf ./vertica-cookbook.tar STEP 2: being still in chef-repo/cookbooks directory, upload the unpacked Vertica cookbook into the Chef repository. It can be easily done, using command line tool for chef – knife:

$ knife cookbook upload mysql Uploading mysql [0.1.0] Uploaded 1 cookbook. STEP3:createrequiredchefroleforMySQLcookbook.Itcanbeeasilydone,usingcommand line tool for chef – knife. To create vertica‐main‐host role, there is need to execute the followingcommand $ knife role create mysql Knife tool creates above role and starts to edit it, using system editor (vi in most cases). During this editing, there is need to place name of vert_install recipe into the run_list element of this role:

{ "name": "mysql", "description": "", "json_class": "Chef::Role", "default_attributes": { }, "override_attributes": { }, "chef_type": "role", "run_list": [ "recipe[mysql::mysql_install]" ], "env_run_lists": { } } ~ STEP 4: assign MySQL cookbook role to particular machines, which are assigned for MySQL installation. It can be done also via knife:

$ knife node edit full-domain-node-name

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 8 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide For example:

$ knife node edit bdp-13.bdpdomain.com Knife starts to edit such node, using a system editor (vi in most cases). During this stage, there is need to add appropriate role to the run list of such node:

{ "name": "bdp-13.bdpdomain.com", "chef_environment": "_default", "normal": { "tags": [ ] }, "run_list": [ "recipe[mysql::mysql_install]", ] }~ and then exit the editor, saving this change. Repeat the step for the second node. STEP 5: MySQL Cookbook attributes settings. MySQL cookbook is strongly parametrized, which significantly increases its flexibility. Since then, to ensure succesfull installation, there is need to set all the attributes correctly. All the attruibutes are defined in chefrepo/cookbooks/mysql/attributes/default.rb file. You can use a system editor (vi in most cases) to edit this file:

$ cd /chef-repo/cookbooks/mysql/attributes $ vi default.rb

default['mysql']['nodes'] = ["bdp-13.bdpdomain.com","bdp-14.bdpdomain.com"] default['mysql']['MYSQL_DB_DIR'] = "/data/mysql" default['mysql']['MYSQL_INSTALL_DIR'] = "/nfshaven/mysql" default['mysql']['MYSQL_LOG_DIR'] = "/var/log/mysql" default['mysql']['MYSQL_DBA_PASSWORD'] = "admin" default['mysql']['MYSQL_DB_NAME'] = "test" default['mysql']['MYSQL_REPL_USER'] = "replicator" default['mysql']['MYSQL_CNF_FILE'] = "/usr/my.cnf" default['mysql']['MYSQL_LDAP_URI'] = "ldaps://bdp-1.bdpdomain.com" default['mysql']['MYSQL_LDAP_BASE'] = "cn=accounts,dc=bdpdomain,dc=com" default['mysql']['MYSQL_DBA_USER'] = "mysqladmin" default['mysql']['MYSQL_APP_USER'] = "mysqlapp" default['mysql']['MYSQL_RO_USER'] = "mysqlreadonly" default['mysql']['MYSQL_GANG_NODE'] = "bdp-13.bdpdomain.com" default['mysql']['MYSQL_GANG_GMETRICS_FREQ'] = 5

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 9 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide Below we descriobe all of these attributes:

          







default['mysql']['nodes'].Thisattributedefinesthelistoftwonodesassignedtoinstall MySQL default['mysql']['MYSQL_DB_DIR'].Thisattributedefinesthedirectory,whichwill containMySQLdatabasefiles default['mysql']['MYSQL_INSTALL_DIR'].Thisattributedefinesthedirectory,whichwill beusedtostoreinstallationfilesduringrecipeexecution default['mysql']['MYSQL_LOG_DIR']. This attribute definesthe directory, which willbe usedbyMySQLDBMSsystemtostoreitslogfiles default['mysql']['MYSQL_DBA_PASSWORD'].Thisattributedefinesthepasswordforthe databaseadministratoraccount,whichiscreatedduringMySQLrecipeexecution default['mysql']['MYSQL_DB_NAME'].Thisattribute definesthename of thedatabase, whichisbeingcreatedduringMySQLrecipeexecution default['mysql']['MYSQL_REPL_USER’]. This attribute defines the name of the MySQL databaseuser,whichwillbeusedtoreplicatethedatainreplicatedenvironment default['mysql']['MYSQL_CNF_FILE’].ThisattributedefinesnameandlocationofMySQL DBMSconfigurationfile default['mysql']['MYSQL_LDAP_URI]. This attribute defines the address of the LDAP server,whichisbeingusedintheenvironment default['mysql']['MYSQL_LDAP_BASE].ThisattributedefinesthebaseDN(distinguished name),whichisusedtostoreuserdataintheLDAPtree default['mysql']['MYSQL_DBA_USER'].This attribute sets theusername forpredefined (created automatically by MySQL cookbook) database administrator, authenticated by LDAP default['mysql']['MYSQL_APP_USER']. This attribute sets the name for predefined applicationuser,createdautomaticallybyMySQLcookbookandauthenticatedbyLDAP. Thisaccounthasprivllegesneeded tocreate,modifyanddropdatabaseobjectsinthe wholecluster.IthasnoDBAprivileges,sincethenitcannotbeusedtoadministerother databaseusersandstart/stopaMySQLdatabase default['mysql']['MYSQL_RO_USER'].Thisattributesetsthenameforpredefineduser, created automatically by MySQL cookbook and authenticated by LDAP with read only privilegesinthewholecluster default[‘mysq’][‘MYSQL_GANG_GMETRICS_FREQ’].Thisattributesetsthefrequency(in minutes)ofMySQLstatisticsgatheringforGangliamonitoringsystem

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 10 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide

6

MySQL Installation Recipe execution

This chapter describes the MySQL Cookbook recipe execution. To execute a recipe, you need to login into a machine where MySQL is to be installed and execute chef-client command with an argument specifying which recipe you want to execute. # chef-client . . . Compiling Cookbooks... [2014-08-03T23:38:08-07:00] WARN: Cloning resource attributes for service[iptables] from prior resource (CHEF-3694) [2014-08-03T23:38:08-07:00] WARN: Previous service[iptables]: /var/chef/cache/cookbooks/mysql/recipes/mysql.rb:134:in `from_file' [2014-08-03T23:38:08-07:00] WARN: Current service[iptables]: /var/chef/cache/cookbooks/mysql/recipes/mysql:150:in `from_file' Converging 21 resources Recipe: mysql::mysql

. . . Running handlers: Running handlers complete

Chef Client finished, 9/19 resources updated in 143.202744009 seconds #

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 11 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide

7. Check master-master replication Run some simple test to check a replication : create table on node1 and execute “show tables” on both databases to check it exists on both sides. Then drop a table from node2 and check it was dropped from both databases.

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 12 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide

8. Ganglia Installation Recipe Execution. If you decide to install Ganglia Monitoring System to monitor MySQL system, then you need to execute a separate, non-default recipe (gang_install) on all MySQL nodes. Below we present a sequence of required steps to perform a successful Ganglia installation. Please, note, that it is an optional component, and it has to be executed only, when you decide to monitor the MySQL Cluster via Ganglia. Additionally to that, please, note, that recipe described in this chapter installs only gmond (i.e. Ganglia Montoring Daemon), and not a gmetad (i.e. software, which is responsible for gathering statistics/metrics from gmond processes, and generating a Ganglia Web GUI). We assume, that gmetad software is already installed and configured, using a separate cookbook – monitoring. Some basic terms, which we use in Ganglia monitoring system:





 

Grid:thewholeHAVENenvironment,whichconsistsofsetofserversrunningdifferent software(VerticaDBMS,MySQLDBMS,LDAP/Kerberos,etc).DuringinstallationofGanglia, usingitsinstallationrecipefrommonitoringcookbookwehavecreatedgridcalledHAVEN Grid Cluster:asetofmachines,whichrunthesamesoftware(forexample:Verticacluster,which coversallmachinesrunningVerticaDBMS,MySQLcluster,whichcoversallmachines runningMySQLDBMS,LDAPcluster,whichcoversallmachinesrunningLDAP/Kerberos server.InthischapterwedescribeprocessofcreationofMySQLcluster Node:Asinglemachineinsideofacluster Gangliaserver:aparticularmachine,whichrunsgmetaddaemon–thisdaemonisinstalled byaseparaterecipefrommonitoringcookbook.Inthischapterweassume,thatGanglia serverisalreadyupandrunning.

Below we present a sequence of steps to install Ganglia Monitoring daemon on all MySQL cluster nodes STEP 1: set default[‘mysq’][‘MYSQL_GANG_GMETRICS_FREQ’] to a frequency of statistics/metrics gathering, if its default value (5 minutes) is not good enough for your installation STEP 2: log to the machine, which will be used as a Ganglia server (its name should be already set as a value of MYSQL_GANG_NODE attribute – please, see STEP 1), and execute gang_install recipe: # chef-client –o mysql::ganglia_install … Recipe: vertica::gang_install * execute[Ganglia Yum Repository Installation] action run - execute wget http://public-repo1.hortonworks.com/HDP/centos6/2.x/GA/2.1-latest/hdp.repo -O /etc/yum.repos.d/hdp.repo

* execute[Ganglia Firewall] action run - execute

service iptables start

iptables -I INPUT -m state --state NEW -m tcp -p tcp --dport 8649 -j ACCEPT iptables-save|awk '!x[$0]++' > /etc/sysconfig/iptables service iptables restart … Running handlers:

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 13 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00

Install Guide Running handlers complete

Chef Client finished, 5/5 resources updated in 15.175163545 seconds #

To complete this installation, you need to repeat this recipe execution on the second MySQL node.

HP Restricted © Copyright 2014 Hewlett-Packard Development Company, L.P.

Page 14 of 14 CHEF HAVEn-Cookbook MySQL Install Guide V1.00...