I asked GPT-4 to "be creative" in a loop 10,000 times for a dataset. Do not do this.

I needed a dataset of creative product descriptions for a fine-tuning job. I figured I'd generate them with GPT-4 — give it a product name, tell it to be creative, done.

What I did not think through: I told it to generate 10,000 descriptions. I had the temperature set to 1.2 ("for creativity"). I was using the 128k context model because I thought bigger context = smarter.

Each description was about 200 tokens. Total: 2 million output tokens, plus roughly the same in input tokens.

I fired this off at 11pm before a call with investors the next morning, thinking it would be done by dawn.

The invoice was $214. The fine-tuned model performed worse than the base model.

The product descriptions were, at least, very creative.