OPRO (Optimization by PROmpting) is an optimization method proposed by Google DeepMind researchers in the 2023 paper "Large Language Models as Optimizers." The optimization problem is described in natural language, and previously generated solutions with their scores are placed in a meta-prompt given to a language model, which is asked to produce new solutions that score higher. The new solutions are evaluated and added back to the meta-prompt, and the cycle repeats.
Its best-known application is prompt optimization, in which the model itself searches for instructions that raise accuracy on a task. Because it relies only on model outputs and scores rather than gradients, it can be applied to models accessible only through an API.