High-level design

Overview

Boomi Integration platform shall be used as a middleware to externally handle Data replication between SAP B1 & PortaBilling. Boomi Atom runtime shall execute the following groups of processes:

1. SAP B1 Event listener will be responsible for immediate parking of an event
   • multiple listeners can be used for scaling
2.  "Event 2 Sync Job" process will be responsible for for scheduling jobs to sync data on certain business events triggered by end user behavior

   • Boomi process shall validate a data associated with the event

   • Boomi process shall schedule jobs for the "Sync" processes
   • multiple listeners can be used for scaling
3. "Sync" processes will be responsible for consistent data sync
   • optional queue partitioning for scaling 

Multiple events, exactly one transaction

User can potentially issue state changing events concurrently for a single transaction record e.g. 

• Create transaction record

• Change notes back and forth

• SAP B1 EventSender shall trigger a batch of events, each targeting to create a record in PortaBilling (since there's no established link yet)

We want to avoid potential duplicates for cases similar to the above. 

Therefore, we're gonna

1. Enqueue and accumulate SAP B1 Events
2. Periodically process the queue by "Dispatch" processes to
   • deduplicate events  for a pair of SAP Transaction object key and event type (A - creation, U, updates, C - cancellation)
   • dispatch events for "Event 2 Sync Job" handling in batches
3. "Event 2 Sync Job" shall validate event according to business rules & schedule transaction sync jobs
4. job execution, one after the other (to avoid race of duplicates)
   • • ensure there's no duplicate record in the target system
     • get consistent transaction data during run-time
     • create transaction record in the target system
     • link source & target records

Resiliency & Recovery

In real world of multiple connected apps there're plenty of occasions where a certain solution component may not be available / may not work as expected, e.g. scheduled maintenances, outages, etc. Our event-driven solution attempts automated recovery as much as possible out-of-the-box

• Event delivery is retried automatically by SAP B1 Integration Framework
  • there's also a manual mode to replay certain events for recovery
• Boomi connectivity to API hosts is retried automatically
• Boomi scheduling and execution processes are retried automatically
  •  there's also a manual mode to replay certain events/jobs for recovery

Update/Migration logic and information