现redis官网稳定最新版本为redis3.2.8,因此我就拿这个版本的来做redis集群部署,以及部署完成如何配合laravel5.1框架进行缓存配置使用!
首先我这边使用了两台虚拟机centos6.5 开启6个节点,每台虚拟机开一个3个redis实例
192.168.0.218
192.168.0.220
以下我就简称218 220.
一、配置redis集群
1.1 在220主机下载 redis3.2.8 tar包 redis 中文官网 http://redis.cn/download.html
wget http://download.redis.io/releases/redis-3.2.8.tar.gz tar -zxvf redis-3.2.8.tar.gz cd redis-3.2.8 mkdir /usr/local/redis make make PREFIX=/usr/local/redis install cp redis.conf /usr/local/redis # 创建配置文件 echo "PATH=$PATH:/usr/local/redis/bin" >> /etc/profile # 添加环境变量 source /etc/profile cp src/redis-trib.rb /usr/local/bin/ # redis cluster 管理工具
1.2 配置 220 redis节点
1.2.0 我的redis是安装在/usr/local/redis 我在此目录新建立了三个目录 对应着7000 7001 7002 端口
mkdir 7000 7001 7002 cp redis.conf 7000 cp redis.conf 7001 cp redis.conf 7002 # copy 完成后分别把三个redis.conf文件按照 1.2.1 - 1.2.8 步骤更改redis.conf文件
1.2.1 daemonize no 更改为 yes 让redis可以在后台运行(默认redis为前台运行模式)
1.2.2 port 7000 redis 端口
1.2.3 bind 192.168.0.220 本机ip 也可以是内网地址,注意这里只需要写一个,默认是127.0.0.1 我在127.0.01 加了192.168.0.220 最后在启动集群时 总提示 waiting for the cluster to join .....
1.2.4 pidfile /var/run/redis-7000.pid # pidfile文件对应7000,7001,7002
1.2.5 cluster-enabled yes # 开启集群模式把注释#去掉
1.2.6 cluster-config-file nodes-7000.conf # 集群的配置 配置文件首次启动自动生成 7000,7001,7002
1.2.7 cluster-node-timeout 15000 # 请求超时 默认15秒,可自行设置
1.2.8 cluster-require-full-coverage no # 只要有结点宕机导致16384个槽没全被覆盖,整个集群就全部停止服务,所以一定要改为no
1.3 配置 218 redis节点
1.4 重复1.1 1.2 所有步骤,注意对应的参数
1.4 启动redis cluster
1.4.1 220 服务器
redis-server 7000/redis.conf # 分别启动7000-7002端口 redis-server 7001/redis.conf redis-server 7002/redis.conf ps -ef | grep redis # 查看redis启动状态 root 2694 1 0 17:22 ? 00:00:00 redis-server 192.168.0.220:7001 [cluster] root 2698 1 0 17:22 ? 00:00:00 redis-server 192.168.0.220:7002 [cluster] root 2706 1 0 17:23 ? 00:00:00 redis-server 192.168.0.220:7000 [cluster]
1.4.2 218 服务器
redis-server 7003/redis.conf # 分别启动7003-7005端口 redis-server 7004/redis.conf redis-server 7005/redis.conf ps -ef | grep redis # 查看redis启动状态 root 2694 1 0 17:22 ? 00:00:00 redis-server 192.168.0.220:7003 [cluster] root 2698 1 0 17:22 ? 00:00:00 redis-server 192.168.0.220:7004 [cluster] root 2706 1 0 17:23 ? 00:00:00 redis-server 192.168.0.220:7005 [cluster]
1.5 220 服务器启动redis集群 (如果防火墙打开的话,就关闭下防火墙,或者开放对应的端口)
yum -y install ruby ruby-devel rubygems rpm-build # redis-trib.rb 它是用ruby写的一个程序,所以我们还得安装ruby. gem install redis redis-trib.rb create --replicas 1 192.168.0.220:7000 192.168.0.220:7001 192.168.0.220:7002 192.168.0.218:7003 192.168.0.218:7004 192.168.0.218:7005 # 在跳出的对话中输入yes即可(不出意外) >>> Creating cluster >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 192.168.0.220:7000 192.168.0.218:7003 192.168.0.220:7001 Adding replica 192.168.0.218:7004 to 192.168.0.220:7000 Adding replica 192.168.0.220:7002 to 192.168.0.218:7003 Adding replica 192.168.0.218:7005 to 192.168.0.220:7001 M: 621450b4d5313d4da23c7792a23fee3a0677d23b 192.168.0.220:7000 slots:0-5460 (5461 slots) master M: 3acc1e17a7b937df6f1656e221b092c32f615e4a 192.168.0.220:7001 slots:10923-16383 (5461 slots) master S: a9ea663e6c1390829be629eaccb58a14349fe857 192.168.0.220:7002 replicates 2a99afdcc227884d91d9173e6609655bfeb098d0 M: 2a99afdcc227884d91d9173e6609655bfeb098d0 192.168.0.218:7003 slots:5461-10922 (5462 slots) master S: 5e5706fd36b24a1b83a4cc5a917a808ee938fb64 192.168.0.218:7004 replicates 621450b4d5313d4da23c7792a23fee3a0677d23b S: a037e0f0080f91f5643da95b4db1d54a56bc9d8f 192.168.0.218:7005 replicates 3acc1e17a7b937df6f1656e221b092c32f615e4a Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join... >>> Performing Cluster Check (using node 192.168.0.220:7000) M: 621450b4d5313d4da23c7792a23fee3a0677d23b 192.168.0.220:7000 slots:0-5460 (5461 slots) master 1 additional replica(s) M: 3acc1e17a7b937df6f1656e221b092c32f615e4a 192.168.0.220:7001 slots:10923-16383 (5461 slots) master 1 additional replica(s) S: a037e0f0080f91f5643da95b4db1d54a56bc9d8f 192.168.0.218:7005 slots: (0 slots) slave replicates 3acc1e17a7b937df6f1656e221b092c32f615e4a S: 5e5706fd36b24a1b83a4cc5a917a808ee938fb64 192.168.0.218:7004 slots: (0 slots) slave replicates 621450b4d5313d4da23c7792a23fee3a0677d23b S: a9ea663e6c1390829be629eaccb58a14349fe857 192.168.0.220:7002 slots: (0 slots) slave replicates 2a99afdcc227884d91d9173e6609655bfeb098d0 M: 2a99afdcc227884d91d9173e6609655bfeb098d0 192.168.0.218:7003 slots:5461-10922 (5462 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
1.6 使用redis
redis-cli -c -h 192.168.0.220 -p 7000 # -c 指的进入redis集群状态 -h 代表要连接的服务器 - p 指的是端口 192.168.0.220:7000> set detail:1 1 -> Redirected to slot [12379] located at 192.168.0.220:7001 # 可以看到 detail:1 这个key 和value 被存储到了 220 机器下的7001 端口 原理参考 http://blog.csdn.net/cfl20121314/article/details/52838986 OK
二、laravel5.1 项目如何使用redis 集群
2.1 在composer.json "predis/predis": "1.0.*"
2.2 命令行执行 composer update
2.3 .env 文件 CACHE_DRIVER=redis
2.4 config/database.php 让你指定一套 Predis 客户端选项 (参考 https://github.com/nrk/predis/wiki/Client-Options 以及laravel手册redis项 )
'redis' => [ 'cluster' => true, 'default' => [ 'host' => '192.168.0.218', 'port' => 7005, ], 'options' => [ 'cluster' => 'redis', ], ],
三、redis cluster 大批量删除某类型keys
redis cluster 不支持scan 或者keys方式命中删除的模式,于是在linux命令行中无法对现有keys进行删除,只能另想它法,在google上搜索了很久,发现使用redis pipe 结合脚本可以极大速度的删除想要删除的key
下面贴出主要得代码
1、mysql中 创建表
CREATE TABLE `del_redis_keys` ( `id` int(11) NOT NULL AUTO_INCREMENT, `keyname` varchar(60) DEFAULT NULL, PRIMARY KEY (`id`), KEY `idx_kn` (`keyname`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8
2、编辑SQL文件
cat del_redis_keys.sql
select concat("*2\r\n",'$3\r\n','DEL\r\n','$',length(redis_key),'\r\n',redis_key,'\r') from (select concat('',keyname) as redis_key from del_redis_keys) as t;
3、sheel脚本
#!/bin/bash source /etc/profile key_pattern_name=$1 time=`date "+%m%d%H%M"` manage_db_user="root" manage_db_pass="x13966498715" manage_db_host="localhost" manage_db_port="3306" manage_config_db="redis" manage_config_tb="tb_redis_node" base_dir=$(cd `dirname $0`; pwd) del_redis_keys_sql="${base_dir}/del_redis_keys.sql" scan_keys_dir="${base_dir}/logs" error_log="/tmp/del_redis_keys.log" error_log2="/tmp/del_redis_keys.log2" redis_cli="/usr/local/redis/bin/redis-cli" mysql="/usr/local/mysql/bin/mysql" lv1="\033[42;30m" lv2="\033[0m" end="\033[0m" red="\033[45;37m" if [ $# -ne 1 ];then echo "Usage: $0 key_pattern" exit fi if [ ! -f "$del_redis_keys_sql" ];then echo -e "${red}${del_redis_keys_sql}文件不存在,请检查${end}" exit fi if [ ! -f "$redis_cli" ];then echo -e "${red}${redis_cli} redis启动文件不存在,请检查${end}" exit fi if [ ! -x "$mysql" ];then echo -e "${red}${mysql} mysql启动文件不存在,请检查${end}" exit fi if [ ! -d "$scan_keys_dir" ];then mkdir -p $scan_keys_dir fi echo "keys所在目录: $scan_keys_dir" sum=0 check_key=0 redis_node_list=`$mysql -u $manage_db_user -p$manage_db_pass -h $manage_db_host -P $manage_db_port $manage_config_db -N -e "select concat_ws('|',hostip,port) from $manage_config_tb where status=1;"` #线上将status修改为1 for redis_info in $redis_node_list do host=`echo $redis_info | awk -F"|" '{print $1}'` port=`echo $redis_info | awk -F"|" '{print $2}'` befer_del_cnt=`$redis_cli -c -h $host -p $port info Keyspace|grep -v Keyspace|awk -F, '{print $1}'|awk -F: '{print $2}'|awk -F= '{print $2}'` #redis导出要删除的keys $redis_cli -c -h $host -p $port --scan --pattern "*${key_pattern_name}*" > $scan_keys_dir/${host}_${port}.log key_num=`wc -l $scan_keys_dir/${host}_${port}.log|awk '{print $1}'` sum=$[$key_num+$sum] echo -e "${lv1}$host:$port${lv2}" #清空del_redis_keys表 $mysql -u $manage_db_user -p$manage_db_pass -h $manage_db_host -P $manage_db_port $manage_config_db -N -e "truncate table del_redis_keys;" #导入要删除的keys $mysql -u $manage_db_user -p$manage_db_pass -h $manage_db_host -P $manage_db_port $manage_config_db --local-infile=1 --show-warnings -N -e "load data local infile '$scan_keys_dir/${host}_${port}.log' into table del_redis_keys (keyname);" > $error_log #判断导出的keys和导入的keys数目是否相等 mysql_keys_num=`$mysql -u $manage_db_user -p$manage_db_pass -h $manage_db_host -P $manage_db_port $manage_config_db -N -e "select count(1) from del_redis_keys;"` if [ $key_num -ne $mysql_keys_num ];then echo -e "${red}导入数据库的keys数目不正确${end}" exit fi #检验导入时是否有warning,如果有退出并提示 if [ -s "$error_log" ];then warning_cnt=`grep -o Warning $error_log|wc -l|awk '{print $1}'` if [ $warning_cnt -ge 1 ];then echo -e "${red}导入数据报错,请看$error_log${end}" exit fi fi #在数据库中查询导入的要删除的key是否正确 检查keys是否正确,只在第一个节点检查,后面节点不再重复检查 if [ $check_key -eq 0 ];then echo "请检查数据库里要删除的keys是否正确" echo "$mysql -u $manage_db_user -p$manage_db_pass -h $manage_db_host -P $manage_db_port $manage_config_db -e 'select * from del_redis_keys limit 20;'" #echo "select * from del_redis_keys limit 50;" while true; do read -p "检查完keys之后选择继续或退出(yes|no): " yn case $yn in yes ) break;; no ) exit;; * ) echo "Please input yes or no";; esac done while true; do read -p "确认继续吗(yes|no): " yn case $yn in yes ) break;; no ) exit;; * ) echo "Please input yes or no";; esac done fi check_key=$[check_key+1] #删除keys $mysql --raw --skip-column-names --default-character-set utf8 -u $manage_db_user -p$manage_db_pass -h $manage_db_host -P $manage_db_port $manage_config_db -A < ${del_redis_keys_sql} | $redis_cli -h $host -p $port --pipe >> $error_log2 2>&1 #打印 after_del_cnt=`$redis_cli -c -h $host -p $port info Keyspace|grep -v Keyspace|awk -F, '{print $1}'|awk -F: '{print $2}'|awk -F= '{print $2}'` echo "删除之前keys数目是${befer_del_cnt}" echo "删除之后keys数目是${after_del_cnt}" echo "删除keys数目是${key_num}" done echo "删除的keys总数是$sum"
参考 :
http://blog.csdn.net/dc_726/article/details/48552531 全面剖析Redis Cluster原理和应用
http://www.cnblogs.com/wuxl360/p/5920330.html Redis集群搭建与简单使用
http://blog.csdn.net/mysqldba23/article/details/65637158 redis cluster超大批量删除keys
结束语
本文有任何错误,或有任何疑问,欢迎留言说明。
网友最新评论