cold start
A cold start is the initialization delay that occurs when a stopped server or runtime starts up to handle a request.
A cold start is the process, or the resulting delay, of launching a new container or function runtime and loading the necessary code and data when a request arrives and no instance is running. It contrasts with a warm start, in which an already prepared instance handles the request.
The term is common in serverless computing. When serving AI models, downloading model weights or loading them into memory can lengthen the delay.
This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.
Articles covering this entry
A measured cold start took 67 seconds end to end, including 15 seconds to load model weights into memory, while the GPU execution time remained 297 milliseconds…