(源码)基于Spring Boot和Disruptor的ETL数据处理系统.zip

上传者: m0_62153576 | 上传时间: 2025-09-15 15:16:49 | 文件大小: 61KB | 文件类型: ZIP
# 基于Spring Boot和Disruptor的ETL数据处理系统 ## 项目简介 本项目是一个基于Spring Boot框架和Disruptor高性能并发框架的ETL(Extract, Transform, Load)数据处理系统。系统旨在高效地从多个数据源抽取数据,进行必要的转换,并加载到目标数据源中。支持多数据源查询、多数据源入库、集群处理以及父子任务依赖。 ## 项目的主要特性和功能 1. 多数据源支持支持从多个数据源(如MySQL、Hive等)进行数据抽取,并能够将处理后的数据加载到不同的目标数据源中。 2. 动态数据源切换通过动态数据源配置,可以在运行时根据业务需求切换不同的数据源。 3. 高性能并发处理利用Disruptor框架实现高性能的并发数据处理,支持高并发下的数据抽取、转换和加载。 4. 任务依赖管理支持父子任务依赖,确保任务按照正确的顺序执行。

文件下载

资源详情

[{"title":"( 36 个子文件 61KB ) (源码)基于Spring Boot和Disruptor的ETL数据处理系统.zip","children":[{"title":"pom.xml <span style='color:#111;'> 3.63KB </span>","children":null,"spread":false},{"title":"src","children":[{"title":"main","children":[{"title":"resources","children":[{"title":"logback.xml <span style='color:#111;'> 1.20KB </span>","children":null,"spread":false},{"title":"simple_etl.sql <span style='color:#111;'> 3.27KB </span>","children":null,"spread":false},{"title":"application.yml <span style='color:#111;'> 2.76KB </span>","children":null,"spread":false}],"spread":true},{"title":"java","children":[{"title":"com","children":[{"title":"github","children":[{"title":"hls","children":[{"title":"etl","children":[{"title":"mapper","children":[{"title":"SimpleETLMapper.java <span style='color:#111;'> 1.53KB </span>","children":null,"spread":false}],"spread":true},{"title":"controller","children":[{"title":"ETLController.java <span style='color:#111;'> 4.31KB </span>","children":null,"spread":false}],"spread":true},{"title":"utils","children":[{"title":"ThreadPoolUtil.java <span style='color:#111;'> 4.26KB </span>","children":null,"spread":false},{"title":"SimpleDBUtils.java <span style='color:#111;'> 10.98KB </span>","children":null,"spread":false},{"title":"SimpleDBBatchUtils.java <span style='color:#111;'> 6.56KB </span>","children":null,"spread":false},{"title":"BeanUtils.java <span style='color:#111;'> 5.54KB </span>","children":null,"spread":false},{"title":"SimpleETLUtils.java <span style='color:#111;'> 6.61KB </span>","children":null,"spread":false},{"title":"DateUtils.java <span style='color:#111;'> 27.49KB </span>","children":null,"spread":false},{"title":"StrUtils.java <span style='color:#111;'> 1.08KB </span>","children":null,"spread":false}],"spread":true},{"title":"base","children":[{"title":"task","children":[{"title":"BatchConsumerTask.java <span style='color:#111;'> 1.56KB </span>","children":null,"spread":false},{"title":"SimpleETLTask.java <span style='color:#111;'> 6.39KB </span>","children":null,"spread":false}],"spread":true},{"title":"enums","children":[{"title":"HandleTypeEnum.java <span style='color:#111;'> 1.19KB </span>","children":null,"spread":false}],"spread":false},{"title":"disruptor","children":[{"title":"Disruptor.java <span style='color:#111;'> 5.43KB </span>","children":null,"spread":false},{"title":"Consumer.java <span style='color:#111;'> 3.94KB </span>","children":null,"spread":false},{"title":"DataDTO.java <span style='color:#111;'> 2.32KB </span>","children":null,"spread":false},{"title":"Producer.java <span style='color:#111;'> 3.69KB </span>","children":null,"spread":false},{"title":"IntEventExceptionHandler.java <span style='color:#111;'> 1.69KB </span>","children":null,"spread":false},{"title":"ETLTypeEnum.java <span style='color:#111;'> 1.38KB </span>","children":null,"spread":false}],"spread":false},{"title":"datasource","children":[{"title":"DynamicDataSourceConfig.java <span style='color:#111;'> 4.38KB </span>","children":null,"spread":false},{"title":"data","children":[{"title":"DataSourceAspect.java <span style='color:#111;'> 2.83KB </span>","children":null,"spread":false},{"title":"DataSourceNames.java <span style='color:#111;'> 683B </span>","children":null,"spread":false},{"title":"DataSource.java <span style='color:#111;'> 1.65KB </span>","children":null,"spread":false},{"title":"DynamicDataSource.java <span style='color:#111;'> 2.69KB </span>","children":null,"spread":false}],"spread":false}],"spread":false},{"title":"etl","children":[{"title":"AutoPageStrategy.java <span style='color:#111;'> 4.79KB </span>","children":null,"spread":false},{"title":"SectionValueStrategy.java <span style='color:#111;'> 2.10KB </span>","children":null,"spread":false},{"title":"base","children":[{"title":"AbsSimpleETLStrategy.java <span style='color:#111;'> 10.68KB </span>","children":null,"spread":false}],"spread":false}],"spread":false}],"spread":true},{"title":"service","children":[{"title":"SimpleETLService.java <span style='color:#111;'> 4.04KB </span>","children":null,"spread":false}],"spread":true},{"title":"domain","children":[{"title":"SimpleETLDO.java <span style='color:#111;'> 4.41KB </span>","children":null,"spread":false},{"title":"SimpleETLRO.java <span style='color:#111;'> 3.79KB </span>","children":null,"spread":false}],"spread":false}],"spread":true},{"title":"SimpleETLApplication.java <span style='color:#111;'> 1.68KB </span>","children":null,"spread":false}],"spread":true}],"spread":true}],"spread":true}],"spread":true}],"spread":true}],"spread":true},{"title":"LICENSE <span style='color:#111;'> 11.29KB </span>","children":null,"spread":false},{"title":"README.md <span style='color:#111;'> 3.11KB </span>","children":null,"spread":false}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明