Autonomous Data Supply Chains

Data supply chains typically process raw, unstructured data from multiple sources for delivery to SaaS end-users. The data is “refined” into highly structured feeds that are ingested into client content management systems (CMSs) via complex, multistage processes that are essentially the circulatory system for enterprise software tools. But can this all be automated?

The stages in these supply chains typically include:

• Unstructured data posted by authoritative sources and retrieved in real time.

• AI-powered routines that extract relevant fields of data from the source docs and deliver the data as a feed to a client CMS.

• The assignment of metadata to each record based on source, retrieval date, and accuracy ‘confidence’ (based on record completeness, field values that are outside of historical ranges, etc.).

• The automatic ingestion of high-confidence records and the review of low-confidence records by trained human QA resources before CMS ingestion.

The question on a lot of process engineers’ minds right now is whether AI can replace that last stage and make the fully automated ingestion of data into CMS applications possible. At the current time I think that the answer is ‘no’ but from what I am seeing with our clients, 90% automation is possible for a lot of processes and that represents a very significant cost savings.

As technology improves and unstructured source files become easier to ingest, we will move to close that 10% gap. True, frictionless machine-to-machine communication is coming and we at IE are here to help you navigate your path to the promised land of AI-driven efficiency.