redis數(shù)據(jù)遷移

1.需求

需要將一個(gè)redis實(shí)例中的部分keys，轉(zhuǎn)移到另一個(gè)redis實(shí)例

2.遷移方案

2.1 源實(shí)例與目標(biāo)實(shí)例版本相同

2.1.1 使用dump命令

#!/bin/bash

#redis 源ip
src_ip=127.0.0.1
#redis 源port
src_port=6392

#redis 目的ip
dest_ip=127.0.0.1
#redis 目的port
dest_port=6393

#要遷移的key前綴
key_prefix=test

i=1

redis-cli -h $src_ip -p $src_port keys "${key_prefix}*" | while read key
do
    redis-cli -h $dest_ip -p $dest_port del $key
    redis-cli -h $src_ip -p $src_port --raw dump $key | perl -pe 'chomp if eof' | redis-cli -h $dest_ip -p $dest_port -x restore $key 0
    echo "$i migrate key $key"
    ((i++))
done

2.1.2 使用migrate命令

migrate用法：

MIGRATE host port key destination-db timeout [COPY] [REPLACE] 

起始版本：2.6.0
時(shí)間復(fù)雜度：This command actually executes a DUMP+DEL in the source instance, and a RESTORE in the target instance. See the pages of these commands for time complexity. Also an O(N) data transfer between the two instances is performed.

遷移腳本

#!/bin/bash

#redis 源ip
src_ip=127.0.0.1
#redis 源port
src_port=6392

#redis 目的ip
dest_ip=127.0.0.1
#redis 目的port
dest_port=6393

#要遷移的key前綴
key_prefix=test

i=1

redis-cli -h $src_ip -p $src_port keys "${key_prefix}*" | while read key
do
    redis-cli -h $src_ip -p $src_port migrate $dest_ip $dest_port $key 0 1000 replace
    echo "$i migrate key $key"
    ((i++))
done

2.2 源實(shí)例與目標(biāo)實(shí)例版本不相同

2.2.1 不可行方案

• 如果源實(shí)例與目標(biāo)實(shí)例版本不相同，使用migrate進(jìn)行遷移的時(shí)候會(huì)有如下錯(cuò)誤：

  1935 migrate key esf_common_auth_code_18587656289
  (error) ERR Target instance replied with error: ERR DUMP payload version or checksum are wrong

• 如果源實(shí)例與目標(biāo)實(shí)例版本不相同，使用dump進(jìn)行遷移的時(shí)候會(huì)有如下錯(cuò)誤

  (error) ERR DUMP payload version or checksum are wrong

• 如果源實(shí)例與目標(biāo)實(shí)例版本不相同,直接復(fù)制rdbw文件搭建從庫為有如下報(bào)錯(cuò)

  5453:S 23 Nov 18:13:14.153 * MASTER <-> SLAVE sync: Flushing old data
  5453:S 23 Nov 18:13:14.153 * MASTER <-> SLAVE sync: Loading DB in memory
  5453:S 23 Nov 18:13:14.153 # Can't handle RDB format version 8
  5453:S 23 Nov 18:13:14.153 # Failed trying to load the MASTER synchronization DB from disk

2.2.2 可行方案

1. 開啟源實(shí)例aof持久化功能

   config set appendonly yes

2. 手動(dòng)進(jìn)行aof持久化

   bgrewriteaof

3. 新建一個(gè)redis實(shí)例，與目標(biāo)實(shí)例版本相同并啟動(dòng)

4. 將源實(shí)例的redis的aof文件導(dǎo)入新建實(shí)例

   redis-cli -h 127.0.0.1 -p 6395 -a password --pipe < appendonly.aof

5. 通過dump或者migrate的方式將新實(shí)例的key遷移到目標(biāo)實(shí)例