Two numbers decide whether an AI feature belongs in a mid-market business: what each answer costs to produce, and whether it arrives when it is needed. The first has been moving in the buyer’s favour. The second depends on how the software is built.
Inference keeps getting cheaper
Most of the cost of a deployed AI feature is inference: running the model every time it is used. NVIDIA, whose accelerated computing platforms run much of the world’s AI workload, has made lower cost and energy per generated token a central goal of its successive data-centre generations, from Hopper to Blackwell, together with its inference software. Cloud providers pass those gains on as lower prices and faster responses, and smaller, specialised models now handle many tasks that once needed the largest ones.
The practical effect is that AI no longer has to be a flagship project. A route-deviation alert, a document classifier or an enquiry agent can now be priced as an ordinary operating cost.
The network is the constraint
Across Southern Africa, connectivity is improving, but outages, load-shedding and patchy rural coverage are still part of normal operations. Software that assumes a constant connection fails in exactly the moments a business needs it most. Cheap inference does not help if the request never reaches the model.
Designing for the edge
- 01Work locally firstThe device or site records the transaction, the reading or the booking immediately, whether or not it is online.
- 02Queue, then syncWork waiting for the cloud sits in a local queue and syncs in the background when the connection returns, with conflicts resolved by rule rather than by a person.
- 03Send AI only what needs AIHeavy model work runs in the cloud on compact, structured payloads, so a weak connection carries the request quickly.
In production
The fleet telemetry platform we built for Nyambuyo and Co. applies this directly: vehicles report positions and fuel data, AI route-deviation alerts surface on a live dispatch screen, and cost-per-kilometre reporting runs weekly instead of a month late. The Bible Hymns app for the Biblical Convention Society of Zimbabwe takes the same idea to its limit: the full hymnal and its search run on the phone, with no data connection required.
Augustus AI follows the same principle on the customer side. Enquiries are answered from the business’s own approved information, and anything it cannot answer is handed to a person rather than left waiting.