TextGrad is an open-source Python framework released in 2024 by researchers at Stanford University. In the way gradients are backpropagated when training a neural network, it passes natural-language critiques written by a language model about an output back to earlier stages of a system and revises each component accordingly. The authors describe this as automatic differentiation via text, and the framework follows a syntax similar to that of the deep learning library PyTorch.
The variables it can optimize are not limited to prompts and include anything expressed as text, such as answers to problems, code and molecular structures. It is used to tune compound AI systems that combine multiple language model calls and tools.