Briefings and analysis

A large share of enterprise pilots never become production systems. The cause is rarely the choice of model family. The cause is that the pilot was set up without an owner, without a path into the application staff already use, and without a decision on which data may lawfully be used after the demonstration. Ninety days later the notebook still works on last quarter's extract, and group risk has nothing it can file.

The first ninety days should close those gaps in writing. Days one to thirty are for the operational question, the named owner, the systems of record, and a lawful-use note for any personal data. If those four items cannot be produced, the pilot should stop. Days thirty to sixty are for access, a client-controlled evaluation sample, and a baseline measured on that sample. Days sixty to ninety are for the production path: where the output will be written, who will review exceptions, how the system will be paused, and which logs internal audit will later ask to see.

Ownership is the item most often postponed. A steering group cannot grant data access or staff an exception queue. The owner has to be the person who already runs the process that the system is supposed to change. Caution on that person's part forces the scope to a single question and a single integration.

Model choice can wait until the path is clear. A weaker model with logging, rollback and a reviewer will enter production sooner than a stronger model that lives in a presentation. The ninety-day test is a written owner, a sample the client controls, a place for the output to land, and a pause procedure. If those items are missing, restart with an assess phase that treats them as the deliverable before any foundations work.

Data residency is the requirement that specified records remain stored and processed in a defined jurisdiction. For many Singapore enterprises that jurisdiction is Singapore. The requirement may come from internal policy, a customer contract, a sector supervisor's expectation, or a reading of the Personal Data Protection Act together with the organisation's own appetite. The source matters, because it determines who can grant an exception. A vendor's data-processing addendum cannot grant that exception on the organisation's behalf.

Vendor selection should test residency as a design constraint. Ask where the records sit at rest, where they are processed, where logs are stored, and which support staff in which countries can open a ticket that contains a payload. Ask what happens when the vendor changes region, subprocessor or model host. Ask whether a Singapore region means Singapore for the whole path, or only for the object store while embeddings run elsewhere. The answers belong in the vendor dependency register, with an alternative named before go-live.

Foundation-model services complicate the picture because the processing location often depends on the product tier and on features that are easy to switch on later. A pilot that used a Singapore endpoint can drift if a subsequent feature sends prompts to a different region. Change control has to cover that class of change with the same seriousness as a training-data change. If the vendor cannot make the path inspectable, the use case may need a different model or a narrower corpus.

Encrypted records that are processed in another region are still processed in that region. A private network link does not move the jurisdiction. The practical discipline is to write the allowed locations into the statement of work, configure the estate accordingly, and test the configuration with a sample payload before the first production record is sent. Legal advisers remain responsible for the organisation's interpretation of its duties. Our role is to make that interpretation executable in the architecture.

Document intelligence programmes often fail on the accuracy conversation. A vendor quotes a single percentage. A process owner hears that most files will pass without a person. Operations then discovers that the errors concentrate in the document types that carry the money, the delay or the regulatory exposure. A clean invoice template extracts well. A photographed bill of lading or a mixed-language claims file does not. The operational risk sits in that tail, which is the work the exception queue exists to catch.

Accuracy targets should be set by document type, on a labelled sample the client controls, with the process owner in the room. The sample should reflect the live mix of scans, languages and templates. Targets should include a precision floor for fields that write to a payment or a customs declaration, and a recall floor for types that must not be dropped. Confidence thresholds should be tuned so that the queue is staffed at a size operations can actually run.

The human exception queue is part of the system. Reviewers need the source page, the extracted fields, and a way to correct them that writes back to the system of record with their identity. Logging of those corrections is the training signal for later retraining and the audit trail for later disputes. If the queue is treated as a temporary inconvenience, reviewers will invent a side process in e-mail.

The board needs a statement of which document types are in scope, what the measured error is on the sample, how many cases a reviewer can handle in a shift, and what happens when a new template appears. Those four statements are enough to decide whether to proceed, and enough to detect later that the live mix has drifted from the sample. Drift in documents is managed with monitoring, a named owner, and a change-control path back to the labelled set.

Retrieval-augmented generation is a method in which a language model must search an approved corpus and ground its answer in retrieved passages, rather than answering from an unbounded training set. It is the pattern most large organisations reach for when they want staff to query internal knowledge. The failures are specific. Permissioning is copied too late, citations are optional, and nobody has written down what a wrong answer would cost in that role. The assistant then becomes a convenient path around records management.

Permissioning has to follow the access controls already applied to the underlying documents. If a file is limited to a matter, a clinic or a plant, the assistant must not retrieve it for a user who could not open the file in the ordinary repository. Retrieval indexes flatten context, and support accounts are often over-privileged. The corpus map and the identity mapping are therefore foundations work. If the organisation cannot state who may see which class of document, it is not ready for an assistant.

Citation is the control that makes a wrong answer detectable. Answers that staff may act on should show the source page. Evaluation should score faithfulness to those sources. Refuse conditions should be explicit: questions outside the corpus, questions that would require a personal data lookup the user is not entitled to, and questions that ask the system to authorise a physical or financial action. Logging of the question, the sources, the answer and the user allows a quality team to see that a wrong answer occurred.

The cost of a wrong answer depends on the role. A mistaken summary of a public style guide is cheap. A mistaken instruction from a safety notice or a client-confidential memo is not. That cost should be written into the intended-use statement, and it should determine whether the assistant may answer, must cite, or must escalate to a person. Narrow the corpus, name the owner, and set the refuse conditions before the model is chosen.

Internal artificial-intelligence use policies fail when they are written as a statement of values and filed on an intranet that staff do not open. Staff will follow a policy that tells them, in the language of their job, what they may paste into a tool, which tools are approved, and who to ask when the case is unclear. The document has to be short enough to be read, specific enough to be applied, and owned by someone who will answer a question the same week it is asked.

A workable policy names the approved tools and the unapproved ones, including personal accounts of otherwise approved products. It names the classes of data that must not leave the organisation: customer files, unpublished financials, staff records, source code, and anything marked confidential. It names the classes of output that must be reviewed by a person before they are sent outside the organisation or filed to a system of record. It names the logging that will exist, and the function that grants exceptions, with a mailbox that is staffed.

The policy should also say what happens when a new tool appears. A request path with a two-week decision is more useful than a ban that staff will route around. Evaluation of a new tool should reuse the same tests used for vendors: residency, access by support staff, retention of prompts, and whether the organisation can switch the tool off.

Training is part of the policy. Short, role-specific notes work better than a single annual module: what a claims handler may paste, what a technician may ask, what a finance analyst may not upload. Those notes should sit next to the procedure staff already use. Measure whether staff can answer which tools they may use, what they must not paste, and who to ask. If they cannot, the policy is not yet in operation.