前后端分離的爬蟲小項目

前后端分離的爬蟲小項目-----EncycProject

EncycProject項目使用現(xiàn)在流行的前后端分離技術(shù)；此項目創(chuàng)作目的是實踐。利用此項目快速搭建環(huán)境和快速入門。該項目的主線是建立寵物社區(qū)網(wǎng)站。目前該項目完成第一個模塊：寵物百科；剩余模塊陸續(xù)開發(fā)中。（此項目可以變更，第一模塊耦合度低）

項目地址

項目GitHub地址

1 項目效果

1.1 效果圖：

爬蟲數(shù)據(jù)

數(shù)據(jù)庫數(shù)據(jù)

前端

待續(xù)

2 項目結(jié)構(gòu)圖

3.1 框架整合

3.1.1 pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.5.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.threadnew</groupId>
    <artifactId>encycproject</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>encycproject</name>
    <description>Demo project for Spring Boot</description>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>2.0.1</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-configuration-processor</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <!--導(dǎo)入webmagic爬蟲的包-->
        <dependency>
            <groupId>us.codecraft</groupId>
            <artifactId>webmagic-core</artifactId>
            <version>0.7.3</version>
        </dependency>
        <dependency>
            <groupId>us.codecraft</groupId>
            <artifactId>webmagic-extension</artifactId>
            <version>0.7.3</version>
        </dependency>
        <!--配置druid-->
        <!-- https://mvnrepository.com/artifact/com.alibaba/druid-spring-boot-starter -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid-spring-boot-starter</artifactId>
            <version>1.1.17</version>
        </dependency>
        <!--import solr-->
        <!-- Compile -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-solr</artifactId>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

3.1.2 applicaton.yml

spring:
  data:
    solr:
      host: http://localhost:8080/solr
  datasource:
    name: encycproject
    type: com.alibaba.druid.pool.DruidDataSource
    #druid相關(guān)配置
    druid:
      #監(jiān)控統(tǒng)計攔截的filters
      filters: stat
      driver-class-name: com.mysql.cj.jdbc.Driver
      #基本屬性
      url: jdbc:mysql://127.0.0.1:3306/animal?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=GMT%2B8
      username: root
      password: 1234
      #配置初始化大小/最小/最大
      initial-size: 1
      min-idle: 1
      max-active: 20
      #獲取連接等待超時時間
      max-wait: 60000
      #間隔多久進行一次檢測，檢測需要關(guān)閉的空閑連接
      time-between-eviction-runs-millis: 60000
      #一個連接在池中最小生存的時間
      min-evictable-idle-time-millis: 300000
      validation-query: SELECT 'x'
      test-while-idle: true
      test-on-borrow: false
      test-on-return: false
      #打開PSCache，并指定每個連接上PSCache的大小。oracle設(shè)為true，mysql設(shè)為false。分庫分表較多推薦設(shè)置為false
      pool-prepared-statements: false
      max-pool-prepared-statement-per-connection-size: 20
server:
  port: 8100

3.1.3 前端的axios的代理

在config文件夾下的index.js中的proxyTable下添加如下代碼

 '/apis':{
        target: 'http://192.168.43.103:8100/',  // 后臺api
        changeOrigin: true,  //是否跨域
        // secure: true,
        pathRewrite: {
          '^/apis': ''   //需要rewrite的,
        }
      }

 > 未完待續(xù)