lundi 6 avril 2015

How to make spring batch job restartable and job parameters unique

I am using spring batch module to process csv files from a directory. Directory may contain multiple files with a specific extenstion and i am using MultiResourceItemReader to read the files. Job will receive 3 job parameters as read_from_directory, move_to_directory and a default_user_id. All these parameters will remain same for all the job runs. read_from_directory will contain multiple csv files and job should process these files one after another. The problem i am facing is since job parameters are same i am getting JobInstanceAlreadyCompleteException when the job is run second time. I am aware this problem can be overcome by using an additional timestamp parameter to make job parameters unique. But since adding timestamp parameter will make every job instance unique i don't wish to use this approach because it will create issues in making my job restartable. So i would like some suggestions on,




  1. How can i make each job instance unique without using timestamp parameter.




  2. How the job can be made restartable in this case? Will adding 'restartable="true"' suffice or will it take some additional configuration/coding on my part. I am little bit confused here because job will read multiple files from a directory. So if a job fails, for example, due to an incorrect record in one of the file how can i restart the same job from where it left of? I have configured the job to run periodically, after a certain time interval, using a scheduler. So if job fails and then i rectify error in the csv file, will job start from where it left off when it runs next time?




Please find below relevant part from my configuration:



<batch:job id="testJob" restartable="true">
<batch:step id="step1">
<batch:tasklet>
<batch:chunk reader="multiResourceItemReader" writer="fileWriter"
commit-interval="1">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>

<bean id="fileWriter" class="com.ivish.TestFileWriter" />
<bean id="multiResourceItemReader" class="org.springframework.batch.item.file.MultiResourceItemReader" scope="step">
<property name="resources" value="file:#{jobParameters['read_from_directory']}/*.csv" />
<property name="delegate" ref="fileReader" />
</bean>

<bean id="fileReader" class="com.ivish.TestFileReader" scope="step">
<property name="delegate" ref="delegateFileReader" />
<property name="moveToDirectory" value="#{jobParameters['move_to_directory']}" />
</bean>
<bean id="delegateFileReader" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer" ref="fileTokenizer" />
<property name="fieldSetMapper">
<bean
class="org.springframework.batch.item.file.mapping.PassThroughFieldSetMapper" />
</property>
</bean>
</property>
</bean>


Thank you.


Aucun commentaire:

Enregistrer un commentaire