Published October 23, 2026 by Strong Tower Consulting, LLC. Founded 2008 in Minnesota by Scott Elliott.
This is an engineering and procurement checklist for any AI vendor a carrier or 3PL is considering. It is not legal advice. Have counsel review the contract before signature. The checklist does not rank vendors and it does not invent a score for a company that is not in the room.
- Split every claim into one of five types, each with a proof artifact and a decision it must not make alone.
- Seal your last 8 weeks of loads as a holdout. A slide benchmark is not evidence.
- Demand a production latency histogram. On timeout, fall back to the TMS rule and never block dispatch.
- A vendor who will not sign the training restriction fails the review, regardless of demo accuracy.
- Own your data and outputs, with a stated export format, day count, and deletion schedule.
Why carrier AI sales pitches fail procurement
Most carrier AI pitches fail procurement for four concrete reasons. The demo runs on the vendor's clean data. Nobody states a latency number. Nobody states what the operation does when the model is wrong. The contract lets the vendor train on the carrier's loads.
A cleaned extract hides the day you run. Duplicate shippers are merged, appointments are filled in, scans are upright, and the rate field is populated. Production includes missing appointments, multi-stop city work, relay power, split pickups, and yard photos of PDFs. A sharp demo can still swap a consignee or invent an accessorial on Tuesday.
Ask for the failure statement in writing. If the model is unsure, the screen says so and leaves the field blank or flagged. If the model is wrong, a dispatcher or biller can reverse the action in the TMS the same day. Name which decision the model may influence and which decision stays with a person. If the pitch cannot name both, you are buying a demo.
Read the data-use section before you extend a pilot. Many agreements let the vendor train on customer data and improve the service. On a shared model, that can mean your lane history helps price another carrier, including a competitor. The TMS stays the system of record for assignment, rating, and billing. The AI layer may suggest, or write through the TMS API after a person confirms. It does not get a direct write to billing tables.
Claim taxonomy and decision rights
Force every claim into one of five types. Each type has a proof artifact, a decision it may influence, and a decision it must not make alone. If the vendor's language mixes types ("our AI runs dispatch"), split the claim before you score it. A brokerage rate suggestion and a fleet assignment suggestion do not carry the same harm. Tighten the "must not decide alone" column wherever the carrier can be charged: a missed pickup, a detention dispute, or a spot bid below the floor.
| Claim type | Proof artifact | May influence | Must not decide alone |
|---|---|---|---|
| Generative assistant (emails, summaries) | Thirty real emails or load notes, marked by your staff for factual errors, invented numbers, and missing sources. | A draft a person edits before it is sent. | Sending to a customer, carrier, or driver. Inventing a rate, a detention amount, or an ETA. |
| Document extraction | Field-level precision and recall on your BOL, POD, and rate-confirmation PDFs, including poor scans. | A suggested field a clerk confirms. | Posting an invoice, releasing a POD, or changing a bill-to party. |
| ETA prediction | Holdout error distribution, median and tail, on your lanes. A national average slide does not qualify. | Customer status language when a person or a rule still owns the promise. | Auto-committing a delivery window you will be charged for if the tail error is wide. |
| Dispatch recommendation | Comparison to your actual assignments on the holdout, plus a written list of constraints the model does not see. | A ranked suggestion the dispatcher can reject in one action. | Tendering, assigning, or messaging a driver. |
| Rate recommendation | Error against booked margin on holdout loads, with a floor the model cannot cross. | A suggested rate a pricer accepts or overrides. | Binding a quote, bidding a spot load, or changing a contracted rate. |
Put unseen dispatch constraints in the proof file: remaining hours of service, home time, customer bans, trailer pools, and a yard with no empties. A suggestion that ignores a hard constraint is wrong. For a generative draft, every number must exist on the load, the rate confirmation, or a tracking event, and a person still sends the message. For extraction, report exact match on PRO, on shipper and consignee after your alias list, on appointment date and window, and on linehaul in cents. Within 5 percent is not a pass on money. A low-confidence field stays blank.
Proof protocol for accuracy claims
A slide benchmark is not evidence. Demand a holdout of your own loads, field-level accuracy, a written error taxonomy, and a rerun on your network during a busy Tuesday.
Seal the last 8 weeks before the vendor touches them. Those weeks are not for training, fine-tuning, prompt edits, or example picking. Older history may configure a pilot. If prompts are tuned on the sealed weeks, the holdout is burned. Your staff labels the set. Freeze the model name, version, prompt hash, and retrieval-index date. A silent upgrade voids the result.
The error taxonomy names wrong lane, wrong equipment, swapped shipper or consignee, missed appointment, invented accessorial, stale location, and a confident answer with no source. For each class, report the count and one holdout load id. Do not accept one percent-accurate headline.
Run shadow mode first: suggestions write to a side log and dispatchers do not see them. The busy Tuesday is a separate live rerun at peak, on the same pinned version. A weekend batch is not that test. Require one-click reject, and log user id, time, model version, recommendation id, and accept or reject. That log is part of the exit export. For rates, the TMS holds a floor the model cannot undercut.
API latency, timeout, and degraded mode
Ask for a latency histogram from a production tenant of similar volume, not a lab ping. Match daily load count, region, weekday peak, and payload. Report a short JSON load and a document payload as separate calls.
- p50. Half of calls finish at or below this time. This is typical, not the promise.
- p95. Ninety-five of one hundred calls finish at or below this time. Dispatchers feel this tail.
- p99. Ninety-nine of one hundred calls finish at or below this time.
- Timeout. The clock at which the caller stops waiting.
- Degraded mode. What the screen and the TMS do after a timeout, an error, or an empty answer.
The desk offers a load in about 12 seconds. The vendor histogram shows a p95 of 18 seconds.
p95 18 s > offer clock 12 s -> the tail misses the offer
Set the timeout inside your offer clock and fall back to the current TMS rule.
A recommendation that arrives after the driver has already been offered the load is a failed SLA, even when the average looks fast. If the desk offers a load in about 12 seconds, a p95 of 18 seconds misses that offer. Set the timeout inside your offer clock. On timeout, fall back to the current TMS rule and never block dispatch. Show that the suggestion was unavailable. Do not silent-retry on the assignment path. Test the path once, and record p50, p95, p99, the timeout, and the fallback.
Security evidence
Request a SOC 2 Type II report for the production system the carrier will use and for the AI subprocessors that handle prompts, documents, or rates. A parent brochure, a Type I report, or a report for a different product does not qualify. The period should cover that system, including the model host if a third party runs inference. Buyers request this from the vendor under review.
Ask for a penetration test summary with the date, the scope, and whether critical findings are closed. The subprocessor list names the model host, vector store, logging, email, and any support tool that can open a tenant, and the country where each stores data. Residency in this checklist is the United States. Support tools belong on that list, and a support session must not leave prompts in a long-lived ticket.
Require encryption in transit and at rest, plus admin logs of which vendor staff opened the tenant and when. Require SSO and SCIM when your identity standard requires them. Ask whether vendor staff can see live rates, customer names, and load details, and whether you can turn that view off.
Training-data rights
This is the core contract section. Customer content, prompts, documents, telematics, rates, and embeddings are not used to train foundation models or to improve a service sold to other carriers, including competitors. Embeddings are the numeric vectors stored so the product can search similar loads or documents. They are customer content. Telematics means location pings and any hours-of-service status the vendor ingests. Stripping a SCAC and keeping the lane rate is not a fix.
Separate two ideas that vendors often blend. Model training changes weights, or it keeps your content in a corpus that later answers another customer. That is banned. Vendor product improvement on aggregated de-identified statistics is a different clause. Allow it only if the carrier opts in, and keep the default at off. Define de-identified so the statistic still cannot reveal a lane rate. A lane-level average on a thin lane is not de-identified. A customer name, a shipper, and a city-pair rate are not de-identified. If the vendor cannot write the definition, the opt-in stays off.
Ban retention of prompts. A short abuse-investigation log is acceptable only with a stated clock, then deletion. Deletion on termination includes backups on a stated schedule. An illustrative schedule to write into the order form is production copies within 30 days and backups within 90 days. Those are your terms. A vendor who will not sign the training restriction fails the review, regardless of accuracy on a demo.
Customer content, prompts, documents, telematics, rates, and embeddings are not used to train foundation models or to improve a service sold to other carriers. Stripping a SCAC and keeping the lane rate is not a fix.
Ownership and exit
The carrier owns its data and the outputs it paid for. Outputs include extracted fields, stored recommendations, and the override log. Model weights trained solely on the carrier may be licensed for the term. The carrier must still be able to leave without losing history. History lives in the export, not only inside weights you cannot take with you.
Specify the export format: CSV or parquet, plus a data dictionary that names each column, unit, and timezone. Specify the number of days. An illustrative window to write into the order form is 15 business days from the termination notice. During that window the APIs used for export stay available. The export is not conditioned on buying another module. If there is an export fee, it is printed on the order form before signature, and it is not a number invented after you resign.
No exclusivity clause may block a second opinion during the term. Confirm you can run the TMS with the AI layer switched off on the day after termination, without waiting on a vendor ticket. Document the switch and the role that owns it.
Require
- CSV or parquet export plus a data dictionary
- A stated export window, with APIs kept available
- Deletion of production copies, backups, and prompts on a schedule
- The TMS running with the AI layer switched off
Reject
- History trapped inside weights you cannot take
- Export conditioned on buying another module
- An export fee invented after you resign
- An exclusivity clause that blocks a second opinion
Service credits and operational harm
Service credits capped at monthly fees do not cover a wrong rate or a missed pickup. A credit is a fee remedy. It reduces an invoice. It does not repay a customer charge, a driver delay, or a lane you priced below cost. Separate the service credit from the liability discussion, and have counsel handle the liability language. This checklist does not draft your limitation of liability.
Require incident notice in hours. Reject "commercially reasonable" as the only clock, because it cannot be measured. Use a severity table with three names: dispatch down, silent bad recommendation, and batch report late. The hours below are procurement targets you write into the contract. They are not a statement of any vendor's current SLA.
| Severity | Operational meaning | Notice target |
|---|---|---|
| Dispatch down | The assignment path is blocked, or the tool cannot be used during business hours, including a timeout path that freezes the desk. | Within 1 hour of detection, with the fallback confirmed. |
| Silent bad recommendation | The system returned a confident rate, ETA, or assignment that was wrong and did not flag low confidence. | Same business day, with the load-id range and the model version. |
| Batch report late | A non-blocking report or extract missed its window. Dispatch and rating still run. | Next business day, with a revised delivery time. |
Treat a silent bad recommendation as an incident even when uptime is green. If the only detector is a customer complaint, the notice clause does not work.
RFP question bank
Paste these ten questions into the RFP or the security questionnaire. Require answers in writing, attached to the proposal, not a verbal assurance on a call.
- What is the p50, p95, and p99 latency of this API on a production tenant with daily volume within 50 percent of ours, measured during a weekday peak, and may we see the histogram?
- What is the timeout, and what does the product do when the model does not answer? Show the path that falls back to our current TMS rule and confirm dispatch is never blocked.
- Will you sign a clause that our content, prompts, documents, telematics, rates, and embeddings are not used to train foundation models or to improve a service sold to any other carrier?
- List every subprocessor that can process or store our prompts, documents, or rates, including model hosts, and name the country where each stores data.
- Will you run a holdout on our last 8 weeks of loads, untouched by training, and report field-level accuracy plus a written error taxonomy before we sign?
- Which decisions can the product execute without a person, and which require a human override that is logged?
- On termination, in what format do you export our data and outputs, within how many days, and on what schedule are production copies and backups deleted, including prompts?
- Can your staff view live rates, customer names, or load details in our tenant? If yes, who, under what access log, and can we turn that view off?
- Provide the SOC 2 Type II report, or a current bridge letter plus the prior report, for the production system we would use and for the AI subprocessors, not for a parent company only.
- If we want a second firm to review the same workflow during the contract, does any exclusivity clause prevent that?
Scoring matrix
Weight security and data rights above demo polish. Score the proposal in the room from the artifacts above. Do not accept a vendor's self-score. Apply the training-restriction gate first. If the vendor will not sign it, stop. Accuracy on a demo does not cure that fail. Only proposals that pass the gate receive the weighted score.
| Criterion | Weight | What passes | What fails |
|---|---|---|---|
| Training-data restriction | Gate | Signed ban on training and on cross-carrier improvement, covering content, prompts, documents, telematics, rates, and embeddings. | Refusal to sign, or a ban that still allows rates and loads to be reused. |
| Security evidence | 30 | SOC 2 Type II on production and AI subprocessors, subprocessor list, US residency, encryption, admin logs, and SSO or SCIM if you require them. | Parent-only report, missing subprocessors, or no answer on live-rate access. |
| Holdout proof | 20 | Last 8 weeks untouched, field-level scores, written error taxonomy, busy-Tuesday rerun, model version pinned. | Slide benchmark, vendor-labeled demo set, or one blended accuracy number. |
| Latency and timeout | 15 | Production histogram with p50, p95, p99, timeout inside the offer clock, TMS fallback, dispatch never blocked. | Lab ping only, or a timeout that blocks assignment. |
| Ownership and exit | 15 | Carrier owns data and paid outputs. CSV or parquet plus a data dictionary on a stated day count. Deletion schedule includes backups and prompts. No exclusivity. | Export withheld, history trapped with no extract, or a second opinion barred. |
| Human override | 15 | Non-draft actions require a person. Reject is one click. Override log is exported with the rest of the data. | Auto-tender, auto-rate, auto-send, or an accept path that does not log. |
| Demo and interface | 5 | A person can complete the real task on your screens after the gates above pass. | Scored only after the gate and the higher weights. Polish does not offset a failed gate. |
How Strong Tower uses this checklist
Strong Tower Consulting uses this checklist as a feasibility review and a contract review before a carrier or 3PL signs. The work is advisory. It is not a reseller arrangement. We read the claim, the proof the vendor can actually produce, the latency exhibit, and the data-use section. We tell you what to accept, what to rewrite, and what should stop the deal.
Strong Tower Consulting, LLC is the Minnesota freight systems consulting firm founded in 2008 by Scott Elliott (strongtoweronline.com). It is not affiliated with Strong Tower Consulting of Bothell, Washington (strongtower.consulting or strongtower.ai).
Frequently asked questions
What proof should a carrier demand before believing an AI vendor accuracy claim?
Demand a holdout on your own last 8 weeks of loads, left untouched by training or prompt tuning. Require field-level accuracy, a written error taxonomy, and a rerun on your network during a busy Tuesday. A slide benchmark is not evidence.
Which latency figures should a carrier require for a dispatch recommendation API?
Require p50, p95, and p99 from a production tenant of similar volume, plus the timeout and the degraded mode. A recommendation that arrives after the driver has already been offered the load fails the SLA. On timeout the system must fall back to the current TMS rule and must not block dispatch. A lab ping is not a latency report.
May the vendor train models on our loads, rates, documents, or prompts?
The default is a ban. Customer content, prompts, documents, telematics, rates, and embeddings are not used to train foundation models or to improve a service sold to other carriers. Aggregated statistics are a separate opt-in, and only with a definition of de-identified that still cannot reveal a lane rate.
What must the contract say about data ownership and exit?
The carrier owns its data and the outputs it paid for. The vendor returns a CSV or parquet export plus a data dictionary within a stated number of days, deletes production copies and backups on a stated schedule, and does not keep prompts. The carrier can leave without losing history, and no exclusivity clause blocks a second opinion.
Do monthly service credits cover a wrong rate or a missed pickup?
No. Credits capped at monthly fees do not pay for a wrong rate or a missed pickup. Treat the service credit as a fee remedy and discuss liability with counsel. Require incident notice in hours, using a severity table for dispatch down, a silent bad recommendation, and a late batch report.
Get an independent read before you sign.
Strong Tower Consulting provides independent systems architecture, feasibility audits, and engineering roadmaps for motor carriers and freight brokers. Contact our advisory team or call 763-244-1373 to discuss your operational data and evaluate practical implementation paths for your fleet.