springboot+webmagic+mybatis-plus架构 小说网站爬虫

上传者: yyzc2 | 上传时间: 2021-04-08 21:37:15 | 文件大小: 23KB | 文件类型: ZIP
文件是本人将webmagic、mybatis-plus、HikariCP、hutool等开源项目到一个springboot项目中搭建起来的一个爬虫框架,使用了Spring的@Scheduled注解开启定时任务去开启一个爬虫,processor类负责网站的爬取,pipline类负责将爬取的内容通过Mybatis-Plus存储到MySQL数据库;去除webmagic部分,本框架也是一个完整的springboot+mybatis-plus的架构,里面用的基本上都是当前最新的技术,公布出来主要是与大家共同学习,有问题可以指出。注意:数据库结构的SQL在项目的sql文件夹下,启动项目前请修改数据库连接以及定时任务的cron表达式

文件下载

资源详情

[{"title":"( 22 个子文件 23KB ) springboot+webmagic+mybatis-plus架构 小说网站爬虫","children":[{"title":"geek-books","children":[{"title":"sql","children":[{"title":"geek-books.sql <span style='color:#111;'> 3.65KB </span>","children":null,"spread":false}],"spread":true},{"title":"mvnw.cmd <span style='color:#111;'> 5.67KB </span>","children":null,"spread":false},{"title":"mvnw <span style='color:#111;'> 8.90KB </span>","children":null,"spread":false},{"title":"src","children":[{"title":"main","children":[{"title":"resources","children":[{"title":"mapper","children":[{"title":"VolumeMapper.xml <span style='color:#111;'> 911B </span>","children":null,"spread":false},{"title":"BookMapper.xml <span style='color:#111;'> 1.05KB </span>","children":null,"spread":false},{"title":"ChapterMapper.xml <span style='color:#111;'> 965B </span>","children":null,"spread":false}],"spread":true},{"title":"application.yml <span style='color:#111;'> 1.04KB </span>","children":null,"spread":false}],"spread":true},{"title":"java","children":[{"title":"com","children":[{"title":"geek","children":[{"title":"books","children":[{"title":"processor","children":[{"title":"QidianBookChapterProcessor.java <span style='color:#111;'> 2.18KB </span>","children":null,"spread":false},{"title":"QidianBookProcessor.java <span style='color:#111;'> 2.79KB </span>","children":null,"spread":false}],"spread":true},{"title":"model","children":[{"title":"entity","children":[{"title":"Volume.java <span style='color:#111;'> 1.66KB </span>","children":null,"spread":false},{"title":"Book.java <span style='color:#111;'> 2.04KB </span>","children":null,"spread":false},{"title":"Chapter.java <span style='color:#111;'> 1.73KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"mapper","children":[{"title":"VolumeMapper.java <span style='color:#111;'> 274B </span>","children":null,"spread":false},{"title":"ChapterMapper.java <span style='color:#111;'> 277B </span>","children":null,"spread":false},{"title":"BookMapper.java <span style='color:#111;'> 268B </span>","children":null,"spread":false}],"spread":true},{"title":"GeekBooksApplication.java <span style='color:#111;'> 410B </span>","children":null,"spread":false},{"title":"config","children":[{"title":"MyBatisPlusConfig.java <span style='color:#111;'> 1.16KB </span>","children":null,"spread":false}],"spread":false},{"title":"pipeline","children":[{"title":"QidianBookChapterPipeline.java <span style='color:#111;'> 825B </span>","children":null,"spread":false},{"title":"QidianBookPipeline.java <span style='color:#111;'> 1.08KB </span>","children":null,"spread":false}],"spread":false},{"title":"scheduled","children":[{"title":"QidianBookScheduled.java <span style='color:#111;'> 2.10KB </span>","children":null,"spread":false}],"spread":false}],"spread":true}],"spread":true}],"spread":true}],"spread":true}],"spread":true}],"spread":true},{"title":"pom.xml <span style='color:#111;'> 3.09KB </span>","children":null,"spread":false},{"title":".gitignore <span style='color:#111;'> 333B </span>","children":null,"spread":false}],"spread":true}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明