Latency is the time that elapses between sending a request or signal and receiving the corresponding response in computing and networking. It is usually measured in milliseconds and, together with throughput (the amount of work handled per unit of time), is a fundamental performance metric.
In AI, latency metrics include the time from submitting a prompt until the first token is produced (Time To First Token), the rate at which tokens are generated, and the time until the complete response is finished. In AI agent environments, hooks and policy checks inserted between model calls add latency, so a time budget is often set for each inspection step.