网站的策划做推广,上线一个app需要多少钱,推进网站集约化建设 网络安全,昆明做网站建设技巧公司1.什么是Spring Batch#xff1f;
Spring Batch 是一个轻量级的开源框架#xff0c;它提供了一种简单的方式来处理大量的数据。它基于Spring框架#xff0c;提供了一套批处理框架#xff0c;可以处理各种类型的批处理任务#xff0c;如ETL、数据导入/导出、报表生成等。S…1.什么是Spring Batch
Spring Batch 是一个轻量级的开源框架它提供了一种简单的方式来处理大量的数据。它基于Spring框架提供了一套批处理框架可以处理各种类型的批处理任务如ETL、数据导入/导出、报表生成等。Spring Batch提供了一些重要的概念如Job、Step、ItemReader、ItemProcessor、ItemWriter等这些概念可以帮助我们构建可重用的批处理应用程序。通过Spring Batch我们可以轻松地实现批处理的并发、容错、重试等功能同时也可以方便地与其他Spring组件集成如Spring Boot、Spring Data等。总之Spring Batch是一个非常强大、灵活、易于使用的批处理框架可以帮助我们快速构建高效、可靠的批处理应用程序。
分层架构 可以看到它分为三层分别是
Application应用层包含了所有任务batch jobs和开发人员自定义的代码主要是根据项目需要开发的业务流程等。Batch Core核心层包含启动和管理任务的运行环境类如JobLauncher等。Batch Infrastructure基础层上面两层是建立在基础层之上的包含基础的读入reader和写出writer、重试框架等。
主要概念 2.2.1 JobRepository
专门负责与数据库打交道对整个批处理的新增、更新、执行进行记录。所以Spring Batch是需要依赖数据库来管理的。
2.2.2 任务启动器JobLauncher
负责启动任务Job。
2.2.3 任务Job
Job是封装整个批处理过程的单位跑一个批处理任务就是跑一个Job所定义的内容。 上图介绍了Job的一些相关概念
Job封装处理实体定义过程逻辑。JobInstanceJob的运行实例不同的实例参数不同所以定义好一个Job后可以通过不同参数运行多次。JobParameters与JobInstance相关联的参数。JobExecution代表Job的一次实际执行可能成功、可能失败。
所以开发人员要做的事情就是定义Job。
2.2.4 步骤Step
Step是对Job某个过程的封装一个Job可以包含一个或多个Step一步步的Step按特定逻辑执行才代表Job执行完成。 通过定义Step来组装Job可以更灵活地实现复杂的业务逻辑。
2.2.5 输入——处理——输出
所以定义一个Job关键是定义好一个或多个Step然后把它们组装好即可。而定义Step有多种方法但有一种常用的模型就是输入——处理——输出即Item Reader、Item Processor和Item Writer。比如通过Item Reader从文件输入数据然后通过Item Processor进行业务处理和数据转换最后通过Item Writer写到数据库中去。 Spring Batch为我们提供了许多开箱即用的Reader和Writer非常方便。 2.环境搭建
参照代码仓库mysql模块里面docker目录搭建
3.代码工程
实验目标
如何使用 Spring Boot 创建各种不同类型 Spring Batch Job
pom.xml
?xml version1.0 encodingUTF-8?
project xmlnshttp://maven.apache.org/POM/4.0.0xmlns:xsihttp://www.w3.org/2001/XMLSchema-instancexsi:schemaLocationhttp://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsdparentartifactIdspringboot-demo/artifactIdgroupIdcom.et/groupIdversion1.0-SNAPSHOT/version/parentmodelVersion4.0.0/modelVersionartifactIdSpringBatch/artifactIdpropertiesmaven.compiler.source8/maven.compiler.sourcemaven.compiler.target8/maven.compiler.target/propertiesdependenciesdependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-starter-web/artifactId/dependencydependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-autoconfigure/artifactId/dependencydependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-starter-test/artifactIdscopetest/scope/dependencydependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-starter-batch/artifactId/dependencydependencygroupIdmysql/groupIdartifactIdmysql-connector-java/artifactId/dependencydependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-starter-jdbc/artifactId/dependency/dependenciesbuildpluginsplugingroupIdorg.springframework.boot/groupIdartifactIdspring-boot-maven-plugin/artifactId/plugin/plugins/build
/project
job
第一个简单的任务
package com.et.batch.job;import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;Component
public class FirstJobDemo {Autowiredprivate JobBuilderFactory jobBuilderFactory;Autowiredprivate StepBuilderFactory stepBuilderFactory;Beanpublic Job firstJob() {return jobBuilderFactory.get(firstJob).start(step()).build();}private Step step() {return stepBuilderFactory.get(step).tasklet((contribution, chunkContext) - {System.out.println(execute step....);return RepeatStatus.FINISHED;}).build();}
}
多步骤的job
package com.et.batch.job;import org.springframework.batch.core.ExitStatus;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;Component
public class MultiStepJobDemo {Autowiredprivate JobBuilderFactory jobBuilderFactory;Autowiredprivate StepBuilderFactory stepBuilderFactory;Beanpublic Job multiStepJob() {/*return jobBuilderFactory.get(multiStepJob).start(step1()).next(step2()).next(step3()).build();*/// control the next step by last Statusreturn jobBuilderFactory.get(multiStepJob2).start(step1()).on(ExitStatus.COMPLETED.getExitCode()).to(step2()).from(step2()).on(ExitStatus.COMPLETED.getExitCode()).to(step3()).from(step3()).end().build();}private Step step1() {return stepBuilderFactory.get(step1).tasklet((stepContribution, chunkContext) - {System.out.println(execute step1。。。);return RepeatStatus.FINISHED;}).build();}private Step step2() {return stepBuilderFactory.get(step2).tasklet((stepContribution, chunkContext) - {System.out.println(execute step2。。。);return RepeatStatus.FINISHED;}).build();}private Step step3() {return stepBuilderFactory.get(step3).tasklet((stepContribution, chunkContext) - {System.out.println(execute step3。。。);return RepeatStatus.FINISHED;}).build();}
}
多flow控制的job 创建一个flow对象包含若干个step
package com.et.batch.job;import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.job.builder.FlowBuilder;
import org.springframework.batch.core.job.flow.Flow;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;Component
public class FlowJobDemo {Autowiredprivate JobBuilderFactory jobBuilderFactory;Autowiredprivate StepBuilderFactory stepBuilderFactory;Beanpublic Job flowJob() {return jobBuilderFactory.get(flowJob).start(flow()).next(step3()).end().build();}private Step step1() {return stepBuilderFactory.get(step1).tasklet((stepContribution, chunkContext) - {System.out.println(execute step1。。。);return RepeatStatus.FINISHED;}).build();}private Step step2() {return stepBuilderFactory.get(step2).tasklet((stepContribution, chunkContext) - {System.out.println(execute step2。。。);return RepeatStatus.FINISHED;}).build();}private Step step3() {return stepBuilderFactory.get(step3).tasklet((stepContribution, chunkContext) - {System.out.println(execute step3。。。);return RepeatStatus.FINISHED;}).build();}private Flow flow() {return new FlowBuilderFlow(flow).start(step1()).next(step2()).build();}
}
并发执行的jobs
package com.et.batch.job;import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.job.builder.FlowBuilder;
import org.springframework.batch.core.job.flow.Flow;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.core.task.SimpleAsyncTaskExecutor;
import org.springframework.stereotype.Component;Component
public class SplitJobDemo {Autowiredprivate JobBuilderFactory jobBuilderFactory;Autowiredprivate StepBuilderFactory stepBuilderFactory;Beanpublic Job splitJob() {return jobBuilderFactory.get(splitJob).start(flow1()).split(new SimpleAsyncTaskExecutor()).add(flow2()).end().build();}private Step step1() {return stepBuilderFactory.get(step1).tasklet((stepContribution, chunkContext) - {System.out.println(execute step1。。。);return RepeatStatus.FINISHED;}).build();}private Step step2() {return stepBuilderFactory.get(step2).tasklet((stepContribution, chunkContext) - {System.out.println(execute step2。。。);return RepeatStatus.FINISHED;}).build();}private Step step3() {return stepBuilderFactory.get(step3).tasklet((stepContribution, chunkContext) - {System.out.println(execute step3。。。);return RepeatStatus.FINISHED;}).build();}private Flow flow1() {return new FlowBuilderFlow(flow1).start(step1()).next(step2()).build();}private Flow flow2() {return new FlowBuilderFlow(flow2).start(step3()).build();}
}
根据上次运行结果判断是否执行下一步
package com.et.batch.job;import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;Component
public class DeciderJobDemo {Autowiredprivate JobBuilderFactory jobBuilderFactory;Autowiredprivate StepBuilderFactory stepBuilderFactory;Autowiredprivate MyDecider myDecider;Beanpublic Job deciderJob() {return jobBuilderFactory.get(deciderJob).start(step1()).next(myDecider).from(myDecider).on(weekend).to(step2()).from(myDecider).on(workingDay).to(step3()).from(step3()).on(*).to(step4()).end().build();}private Step step1() {return stepBuilderFactory.get(step1).tasklet((stepContribution, chunkContext) - {System.out.println(execute step1。。。);return RepeatStatus.FINISHED;}).build();}private Step step2() {return stepBuilderFactory.get(step2).tasklet((stepContribution, chunkContext) - {System.out.println(execute step2。。。);return RepeatStatus.FINISHED;}).build();}private Step step3() {return stepBuilderFactory.get(step3).tasklet((stepContribution, chunkContext) - {System.out.println(execute step3。。。);return RepeatStatus.FINISHED;}).build();}private Step step4() {return stepBuilderFactory.get(step4).tasklet((stepContribution, chunkContext) - {System.out.println(execute step4。。。);return RepeatStatus.FINISHED;}).build();}
}
父子嵌套job
package com.et.batch.job;import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.JobStepBuilder;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;
import org.springframework.transaction.PlatformTransactionManager;Component
public class NestedJobDemo {Autowiredprivate JobBuilderFactory jobBuilderFactory;Autowiredprivate StepBuilderFactory stepBuilderFactory;Autowiredprivate JobLauncher jobLauncher;Autowiredprivate JobRepository jobRepository;Autowiredprivate PlatformTransactionManager platformTransactionManager;Beanpublic Job parentJob() {return jobBuilderFactory.get(parentJob).start(childJobOneStep()).next(childJobTwoStep()).build();}private Step childJobOneStep() {return new JobStepBuilder(new StepBuilder(childJobOneStep)).job(childJobOne()).launcher(jobLauncher).repository(jobRepository).transactionManager(platformTransactionManager).build();}private Step childJobTwoStep() {return new JobStepBuilder(new StepBuilder(childJobTwoStep)).job(childJobTwo()).launcher(jobLauncher).repository(jobRepository).transactionManager(platformTransactionManager).build();}private Job childJobOne() {return jobBuilderFactory.get(childJobOne).start(stepBuilderFactory.get(childJobOneStep).tasklet((stepContribution, chunkContext) - {System.out.println(subtask1。。。);return RepeatStatus.FINISHED;}).build()).build();}private Job childJobTwo() {return jobBuilderFactory.get(childJobTwo).start(stepBuilderFactory.get(childJobTwoStep).tasklet((stepContribution, chunkContext) - {System.out.println(subtask2。。。);return RepeatStatus.FINISHED;}).build()).build();}
}
application.yaml
自动会初始化脚本只需要建立以恶搞空库就行
spring:datasource:driver-class-name: com.mysql.cj.jdbc.Driverurl: jdbc:mysql://localhost:3306/springbatchusername: rootpassword: 123456batch:jdbc:schema: classpath:org/springframework/batch/core/schema-mysql.sqlinitialize-schema: always #Since Spring Boot 2.5.0 use spring.batch.jdbc.initialize-schemaneverjob:enabled: true
以上只是一些关键代码所有代码请参见下面代码仓库
代码仓库
GitHub - Harries/springboot-demo: a simple springboot demo with some components for example: redis,solr,rockmq and so on.
4.测试
启动Spring Boot应用程序系统会自动运行job跑过一次下次启动不会继续执行如果要执行定时任务可以利用spring提供的scheduledTaskRegistrar注册一个定时任务扫描最新的定时任务将这些定时任务注册到scheduleFuture中从而实现动态定时任务。
5.引用
Batch Applications :: Spring BootSpring Boot集成Spring Batch快速入门Demo | Harries Blog™
