mercredi 25 février 2015

Spring Batch: problems (mix data) when converting to multithread

Maybe this is a recurrent question, but I need some customization with my context.


I'm using Spring Batch 3.0.1.RELEASE


I have a simple job with some steps. One step is a chunk like this:



<tasklet transaction-manager="myTransactionManager">
<batch:chunk reader="myReader" processor="myProcessor" writer="myWriter" commit-interval="${commit.interval}">
</batch:chunk>



<bean id="myProcessor" class="org.springframework.batch.item.support.CompositeItemProcessor" scope="step">
<property name="delegates">
<list>
<bean class="...MyFirstProcessor">
</bean>
<bean class="...MySecondProcessor">
</bean>
</list>
</property>



  • Reader: JdbcCursorItemReader

  • Processor: CompositeProcessor with my delegates

  • Writer: CompositeWriter with my delegates


With this configuration, my job works perfectly.


Now, I want to convert this to a multi-threaded job. Following the documentation to basic multi-thread jobs, I included a SympleAsyncTaskExecutor in the tasklet, but it failed.


I have readed JdbcCursorItemReader does not work properly with multi-thread execution (is it right?). I have changed the reader to a JdbcPagingItemReader, and it has been a nightmare: job does not fail, writing process are ok, but data has mixed among the threads, and customer data were not right and coherent (customers have got services, addreses, etc. from others).


So, why does it happen? How could I change to a multi-thread job?



  • Are the composite processor and writer right for multithread?

  • How could I make a custom thread-safe composite processor?

  • Maybe could it be the JDBC reader: Is there any thread-safe JDBC reader for multi-thread?


I'm very locked and confused with this, so any help would be very appreciated. Thanks a lot.


Aucun commentaire:

Enregistrer un commentaire