Fluent Commerce Logo
Docs
Sign In

How Batch Pre-Processing works

Essential knowledge

Author:

Fluent Commerce

Changed on:

8 Feb 2024

Overview

This document will detail how Batch Pre-Processing works. This will allow Partners to understand how their inventory implementation interacts with Batch Pre-Processing.

Key points

  • Batch Pre-Processing processes batch inventory updates before they get passed onto the inventory workflows
  • This processing identifies inventory records that don't need to be processed by the workflow engine
  • This will reduce the total end-to-end time to process inventory updates by reducing the load on the workflow engine

Pre-requisites

Architecture

Overview

No alt provided

Batch Pre-Processing will process Inventory Updates via a new, more efficient update flow compared to the existing batch inventory update process.

Without Batch Pre-Processing, inventory updates would be split into individual records and added directly to the Inventory Queue. Instead, Batch Pre-Processing will split the items into a different flow to be analysed to determine whether the update is a new change and should be added to the Inventory Queue.

Components

Loader Job

The Loader Job is the first component of Batch Pre-Processing. The Loader Job is tasked with loading the existing inventory data from the database to understand the current status of the Inventory Items within an account. This will enable the Comparison Job to compare the new updates with existing data.

Comparison Job

The Comparison Job will compare the loaded, existing inventory data against the submitted Inventory Updates. This Comparison will determine whether an Inventory Update is “changed” or “unchanged”.

To determine whether an update is changed or unchanged, it will run through a selection of logic. This informs whether the update sent into Fluent is new or is the same data we have already received and, therefore, doesn’t need to be processed again.

The logic follows as such:

  • For the Inventory Position included in the update, the Comparison Job will understand if any active transient inventory quantities are saved against that position. If any active transient inventory quantities are against the inventory position, this update is considered “changed” and will be forwarded to the Inventory Queue.
    • A transient inventory quantity is any quantity of type “SALE”, “RESERVATION” or “CORRECTION”. These are transient, as quantities of these types will routinely be disabled and enabled as Orders and Inventory Updates are processed.
  • The Comparison Job will then look at the qty value shared for the Inventory Update and compare that to the LAST_ON_HAND qty saved against the Inventory Position. If those values differ, the Inventory Update is considered “changed” and will be forwarded to the Inventory Queue.

In Summary:

Any batch Inventory Update with no active transient inventory quantities and where the LAST_ON_HAND quantity is the same will be considered “unchanged” and won’t be sent through the inventory queue. Every other update will be sent through into the inventory queue.

Batch_Complete with Batch Pre-Processing

Due to the nature of Batch Pre-Processing, the concept of individual batches is lost within the processing (all batches processed within a single Batch Pre-Processing job are merged together). To ensure backward compatibility for customers who had integrations relying on Batch_Complete events, we updated the reporting of Batch_Complete records.

When an account or retailer is enabled with Batch Pre-Processing, the Batch_Complete records will be saved 20 minutes after submitting a batch to the system.

Due to this change, for Batch Pre-Processing enabled accounts and retailers, it is suggested that Batch_Complete records should not be used to monitor the successful process. Instead, a combination of the Metrics API and orchestration_audit events should be used to identify errors within the batch inventory update process.



Copyright © 2024 Fluent Retail Pty Ltd (trading as Fluent Commerce). All rights reserved. No materials on this docs.fluentcommerce.com site may be used in any way and/or for any purpose without prior written authorisation from Fluent Commerce. Current customers and partners shall use these materials strictly in accordance with the terms and conditions of their written agreements with Fluent Commerce or its affiliates.

Fluent Logo