Working on different A.I. projects led me to think about differentiating between different types of prompt inputs and outputs.
In one project, I am looking for highly varied output, whereas in the other one I am looking for no variation at all. Depending on what we want to achieve (the output), we have to think about and differentiate between different inputs. An easy way to do this is by defining four different areas by using a simple 2 x 2 matrix.
The higher the value for output variety, the bigger the need for input variation.
First, we must understand if we are in an area that rewards output variation. The generation of ideas can be one example where variation is highly valuable. A set of highly differentiated ideas gives us a more holistic overview and many more opportunities, instead of going with the first idea we have. That means variation in prompting is rewarded with better and higher quality outputs. Using multiple prompts and introducing variables within the prompts makes sense here.
If there’s not a big demand and value for output variation, then we don’t need much variation in our input structure and should optimize toward standardization. It doesn’t make sense to use different prompts to achieve the same output. These prompts should be standardized, stored, and distributed to others, and ideally be integrated as system prompts.
Desirable states are areas with the same input and output variation.
- Deterministic (low/low): One input leads consistently to one specific output without or only slight variation. A good example is LLM-based web research, where one search prompt leads to the same results everytime.
- Varied (high/high): Highly-varied inputs lead to highly-varied outputs. If we want to generate different ideas, we need variation in our inputs to achieve this outcome.
Undesirable states are areas with differences in input or output variation (except for edge cases).
- Unreliable (low/high): Low input variation leads to high output variation. Depending on context, this most often leads to useless results as the outcome is unpredictable and therefore unreliable. An edge case would be if we are generating ideas. A simple prompt would in this case lead to highly varied ideas as output. I deem this case not particularly realistic.
- Useless (high/low): High input variation leads to low output variation, which makes the input variation useless in the first place. Cases like these should be, generally speaking, standardized. An edge case would be where it is desirable to give your users some “slack.” That means, independent of the user input, the output is the same. One example could be A.I.-based website generators, where different prompts lead to the same result of, e.g., adding a picture or header to a website.
Patrick Seibold Blog 