Notifications
Clear all

How can I run BatchWriteSingleRelationalChunk in parallel?

2 Posts
2 Users
1 Reactions
17.4 K Views
0
Topic starter

I uploaded a large file. BatchWriteSingleRelationalChunk task is running 1 task at a time ...

I have 3000+ such tasks... is there a configuration available to parallelize it?

2 Answers
1

 To configure, add an entry to EXTERNAL_REFERENCE with:
 type = ParallelBatchWrite
 key = queue:interface:version, for example "100~inbox:SCC.Item_IB:2.0"
 value = number of grid queues to use to process a single message
Sample query - 

insert into external_reference (SYS_EXTERNAL_REFERENCE_ID,MODEL_NAME,VC_ID,REF_TYPE,EXTERNAL_VALUE,LOCAL_VALUE)
values (EXTERNAL_REFERENCE_SEQ.nextval, 'Standard ExternalReference',(select vc_id from value_chain where vc_id!=0),'ParallelBatchWrite','UploadInboundInterfaceCsvFiles~CMD.UploadOnboardRequest~1.0',10);

* You must register this class through BatchWriteCallback.register from your ModuleContextListener*.

e.g.

BatchWriteCallback.register(new ParallelBatchWriteCallback());
0
Topic starter

See https://docs.onenetwork.com/devnet/latest/platform-user-s-guide/system-configuration

 
You can use the ParallelBatchWrite configuration for your QueueName~Interface~Version.
 
Note - this only takes affect for files dequeued *after* this configuration is created.  It won't affect in-flight processing.