您好,登錄后才能下訂單哦!
如何使用Sharding-JDBC對(duì)數(shù)據(jù)進(jìn)行分片處理,很多新手對(duì)此不是很清楚,為了幫助大家解決這個(gè)難題,下面小編將為大家詳細(xì)講解,有這方面需求的人可以來(lái)學(xué)習(xí)下,希望你能有所收獲。
Sharding-JDBC是ShardingSphere的第一個(gè)產(chǎn)品,也是ShardingSphere的前身。
它定位為輕量級(jí)Java框架,在Java的JDBC層提供的額外服務(wù)。它使用客戶端直連數(shù)據(jù)庫(kù),以jar包形式提供服務(wù),無(wú)需額外部署和依賴,可理解為增強(qiáng)版的JDBC驅(qū)動(dòng),完全兼容JDBC和各種ORM框架。
適用于任何基于Java的ORM框架,如:JPA, Hibernate, Mybatis, Spring JDBC Template或直接使用JDBC。
基于任何第三方的數(shù)據(jù)庫(kù)連接池,如:DBCP, C3P0, BoneCP, Druid, HikariCP等支持任意實(shí)現(xiàn)JDBC規(guī)范的數(shù)據(jù)庫(kù)。
目前支持MySQL,Oracle,SQLServer和PostgreSQL。
Sharding-JDBC的使用需要我們對(duì)項(xiàng)目進(jìn)行一些調(diào)整:結(jié)構(gòu)如下
ShardingSphere文檔地址
這里使用的是springBoot項(xiàng)目改造
<!-- 這里使用了druid連接池 --> <dependency> <groupId>com.alibaba</groupId> <artifactId>druid</artifactId> <version>1.1.9</version> </dependency> <!-- sharding-jdbc 包 --> <dependency> <groupId>com.dangdang</groupId> <artifactId>sharding-jdbc-core</artifactId> <version>1.5.4</version> </dependency> <!-- 這里使用了雪花算法生成組建,這個(gè)算法的實(shí)現(xiàn)的自己寫的代碼,各位客關(guān)老爺可以修改為自己的id生成策略 --> <dependency> <groupId>org.kcsm.common</groupId> <artifactId>kcsm-idgenerator</artifactId> <version>3.0.1</version> </dependency>
#啟動(dòng)接口 server: port: 30009 spring: jpa: database: mysql show-sql: true hibernate: # 修改不自動(dòng)更新表 ddl-auto: none #數(shù)據(jù)源0定義,這里只是用了一個(gè)數(shù)據(jù)源,各位客官可以根據(jù)自己的需求定義多個(gè)數(shù)據(jù)源 database0: databaseName: database0 url: jdbc:mysql://kcsm-pre.mysql.rds.aliyuncs.com:3306/dstest?characterEncoding=utf8&useUnicode=true&useSSL=false&serverTimezone=Hongkong username: root password: kcsm@111 driverClassName: com.mysql.jdbc.Driver
package com.lzx.code.codedemo.config; import com.alibaba.druid.pool.DruidDataSource; import lombok.Data; import org.springframework.boot.context.properties.ConfigurationProperties; import org.springframework.stereotype.Component; import javax.sql.DataSource; /** * 描述:數(shù)據(jù)源0定義 * * @Auther: lzx * @Date: 2019/9/9 15:19 */ @Data @ConfigurationProperties(prefix = "database0") @Component public class Database0Config { private String url; private String username; private String password; private String driverClassName; private String databaseName; public DataSource createDataSource() { DruidDataSource result = new DruidDataSource(); result.setDriverClassName(getDriverClassName()); result.setUrl(getUrl()); result.setUsername(getUsername()); result.setPassword(getPassword()); return result; } }
package com.lzx.code.codedemo.config; import com.dangdang.ddframe.rdb.sharding.api.ShardingValue; import com.dangdang.ddframe.rdb.sharding.api.strategy.database.SingleKeyDatabaseShardingAlgorithm; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; import java.util.ArrayList; import java.util.Collection; import java.util.List; /** * 描述:數(shù)據(jù)源分配算法 * * 這里我們只用了一個(gè)數(shù)據(jù)源,所以所有的都只返回了數(shù)據(jù)源0 * * @Auther: lzx * @Date: 2019/9/9 15:27 */ @Component public class DatabaseShardingAlgorithm implements SingleKeyDatabaseShardingAlgorithm { @Autowired private Database0Config database0Config; /** * = 條件時(shí)候返回的數(shù)據(jù)源 * @param collection * @param shardingValue * @return */ @Override public String doEqualSharding(Collection collection, ShardingValue shardingValue) { return database0Config.getDatabaseName(); } /** * IN 條件返回的數(shù)據(jù)源 * @param collection * @param shardingValue * @return */ @Override public Collection<String> doInSharding(Collection collection, ShardingValue shardingValue) { List<String> result = new ArrayList<String>(); result.add(database0Config.getDatabaseName()); return result; } /** * BETWEEN 條件放回的數(shù)據(jù)源 * @param collection * @param shardingValue * @return */ @Override public Collection<String> doBetweenSharding(Collection collection, ShardingValue shardingValue) { List<String> result = new ArrayList<String>(); result.add(database0Config.getDatabaseName()); return result; } }
package com.lzx.code.codedemo.config; import com.dangdang.ddframe.rdb.sharding.api.ShardingValue; import com.dangdang.ddframe.rdb.sharding.api.strategy.table.SingleKeyTableShardingAlgorithm; import com.google.common.collect.Range; import org.springframework.stereotype.Component; import java.util.Collection; import java.util.LinkedHashSet; /** * 描述: 數(shù)據(jù)表分配算法的實(shí)現(xiàn) * * @Auther: lzx * @Date: 2019/9/9 16:19 */ @Component public class TableShardingAlgorithm implements SingleKeyTableShardingAlgorithm<Long> { /** * = 條件時(shí)候返回的數(shù)據(jù)源 * @param collection * @param shardingValue * @return */ @Override public String doEqualSharding(Collection<String> collection, ShardingValue<Long> shardingValue) { for (String eaach:collection) { Long value = shardingValue.getValue(); value = value >> 22; if(eaach.endsWith(value%10+"")){ return eaach; } } throw new IllegalArgumentException(); } /** * IN 條件返回的數(shù)據(jù)源 * @param tableNames * @param shardingValue * @return */ @Override public Collection<String> doInSharding(Collection<String> tableNames, ShardingValue<Long> shardingValue) { Collection<String> result = new LinkedHashSet<>(tableNames.size()); for (Long value : shardingValue.getValues()) { for (String tableName : tableNames) { value = value >> 22; if (tableName.endsWith(value % 10 + "")) { result.add(tableName); } } } return result; } /** * BETWEEN 條件放回的數(shù)據(jù)源 * @param tableNames * @param shardingValue * @return */ @Override public Collection<String> doBetweenSharding(Collection<String> tableNames, ShardingValue<Long> shardingValue) { Collection<String> result = new LinkedHashSet<>(tableNames.size()); Range<Long> range = shardingValue.getValueRange(); for (Long i = range.lowerEndpoint(); i <= range.upperEndpoint(); i++) { for (String each : tableNames) { Long value = i >> 22; if (each.endsWith(i % 10 + "")) { result.add(each); } } } return result; } }
package com.lzx.code.codedemo.config; import com.dangdang.ddframe.rdb.sharding.api.ShardingDataSourceFactory; import com.dangdang.ddframe.rdb.sharding.api.rule.DataSourceRule; import com.dangdang.ddframe.rdb.sharding.api.rule.ShardingRule; import com.dangdang.ddframe.rdb.sharding.api.rule.TableRule; import com.dangdang.ddframe.rdb.sharding.api.strategy.database.DatabaseShardingStrategy; import com.dangdang.ddframe.rdb.sharding.api.strategy.table.TableShardingStrategy; import com.dangdang.ddframe.rdb.sharding.keygen.DefaultKeyGenerator; import com.dangdang.ddframe.rdb.sharding.keygen.KeyGenerator; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import javax.sql.DataSource; import java.sql.SQLException; import java.util.Arrays; import java.util.HashMap; import java.util.Map; /** * 描述:數(shù)據(jù)源配置 * * @Auther: lzx * @Date: 2019/9/9 15:21 */ @Configuration public class DataSourceConfig { @Autowired private Database0Config database0Config; @Autowired private DatabaseShardingAlgorithm databaseShardingAlgorithm; @Autowired private TableShardingAlgorithm tableShardingAlgorithm; @Bean public DataSource getDataSource() throws SQLException { return buildDataSource(); } private DataSource buildDataSource() throws SQLException { //分庫(kù)設(shè)置 Map<String, DataSource> dataSourceMap = new HashMap<>(2); //添加兩個(gè)數(shù)據(jù)庫(kù)database0和database1 dataSourceMap.put(database0Config.getDatabaseName(), database0Config.createDataSource()); //設(shè)置默認(rèn)數(shù)據(jù)庫(kù) DataSourceRule dataSourceRule = new DataSourceRule(dataSourceMap, database0Config.getDatabaseName()); //分表設(shè)置,大致思想就是將查詢虛擬表Goods根據(jù)一定規(guī)則映射到真實(shí)表中去 TableRule orderTableRule = TableRule.builder("user") .actualTables(Arrays.asList("user_0", "user_1", "user_2", "user_3", "user_4", "user_5", "user_6", "user_7", "user_8", "user_9")) .dataSourceRule(dataSourceRule) .build(); //分庫(kù)分表策略 ShardingRule shardingRule = ShardingRule.builder() .dataSourceRule(dataSourceRule) .tableRules(Arrays.asList(orderTableRule)) .databaseShardingStrategy(new DatabaseShardingStrategy("ID", databaseShardingAlgorithm)) .tableShardingStrategy(new TableShardingStrategy("ID", tableShardingAlgorithm)).build(); DataSource dataSource = ShardingDataSourceFactory.createDataSource(shardingRule); return dataSource; } @Bean public KeyGenerator keyGenerator() { return new DefaultKeyGenerator(); } }
package com.lzx.code.codedemo.entity; import com.fasterxml.jackson.annotation.JsonIgnoreProperties; import com.fasterxml.jackson.databind.annotation.JsonSerialize; import com.fasterxml.jackson.databind.ser.std.ToStringSerializer; import lombok.*; import org.hibernate.annotations.GenericGenerator; import javax.persistence.*; /** * 描述: 用戶 * * @Auther: lzx * @Date: 2019/7/11 15:39 */ @Entity(name = "USER") @Getter @Setter @ToString @JsonIgnoreProperties(ignoreUnknown = true) @AllArgsConstructor @NoArgsConstructor public class User { /** * 主鍵 */ @Id @GeneratedValue(generator = "idUserConfig") @GenericGenerator(name ="idUserConfig" ,strategy="org.kcsm.common.ids.SerialIdGeneratorSnowflakeId") @Column(name = "ID", unique = true,nullable=false) @JsonSerialize(using = ToStringSerializer.class) private Long id; /** * 用戶名 */ @Column(name = "USER_NAME",length = 100) private String userName; /** * 密碼 */ @Column(name = "PASSWORD",length = 100) private String password; }
package com.lzx.code.codedemo.dao; import com.lzx.code.codedemo.entity.User; import org.springframework.data.jpa.repository.JpaRepository; import org.springframework.data.jpa.repository.JpaSpecificationExecutor; import org.springframework.data.rest.core.annotation.RepositoryRestResource; /** * 描述: 用戶dao接口 * * @Auther: lzx * @Date: 2019/7/11 15:52 */ @RepositoryRestResource(path = "user") public interface UserDao extends JpaRepository<User,Long>,JpaSpecificationExecutor<User> { }
package com.lzx.code.codedemo; import com.lzx.code.codedemo.dao.RolesDao; import com.lzx.code.codedemo.dao.UserDao; import com.lzx.code.codedemo.entity.Roles; import com.lzx.code.codedemo.entity.User; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.test.context.junit4.SpringRunner; @RunWith(SpringRunner.class) @SpringBootTest public class CodeDemoApplicationTests { @Autowired private UserDao userDao; @Autowired private RolesDao rolesDao; @Test public void contextLoads() { User user = null; Roles roles = null; for(int i=0;i<1000;i++){ user = new User( null, "lzx"+i, "123456" ); roles = new Roles( null, "角色"+i ); rolesDao.save(roles); userDao.save(user); try { Thread.sleep(100); } catch (InterruptedException e) { e.printStackTrace(); } } } }
看完上述內(nèi)容是否對(duì)您有幫助呢?如果還想對(duì)相關(guān)知識(shí)有進(jìn)一步的了解或閱讀更多相關(guān)文章,請(qǐng)關(guān)注億速云行業(yè)資訊頻道,感謝您對(duì)億速云的支持。
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如果涉及侵權(quán)請(qǐng)聯(lián)系站長(zhǎng)郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。