29/11/2023
Make GPT-4 your b*tch!
Discussion
The other day, I’m elbow-deep in code, upgrading our OpenAI python library from ancient 0.28.1 to shiny new 1.3.5, when this marketing intern pops up beside my desk.
He’s all flustered, like, 'How do I get GPT-4 to do what I want? It’s repeating words, the answers are way too long, and it just doesn’t do that thing I need.'
So, I dive in, trying to break down frequency penalty, logit bias, temperature, top_p – all that jazz. But man, the more I talk, the more his eyes glaze over. I felt bad (No bad students, only bad teachers right?)
So I told him, 'Give me a couple of hours,' planning to whip up a mini TED talk or something to get these concepts across without the brain freeze lol.
Posting here in the hopes that someone might find it useful.
1. Frequency Penalty: The 'No More Echo' K**b
What It Does: Reduces repetition, telling the AI to avoid sounding like a broken record.
Low Setting: "I love pizza. Pizza is great. Did I mention pizza? Because pizza."
High Setting: "I love pizza for its gooey cheese, tangy sauce, and perfect crust. It's an art form in a box."
2. Logit Bias: The 'AI Whisperer' Tool
What It Does: Pushes the AI toward or away from certain words, like whispering instructions.
Bias Against 'pizza': "I enjoy Italian food, particularly pasta and gelato."
Bias Towards 'pizza': "When I think Italian, I dream of pizza, the circular masterpiece of culinary delight."
3. Presence Penalty: The 'New Topic' Nudge
What It Does: Helps AI switch topics, avoiding getting stuck on one subject.
Low Setting: "I like sunny days. Sunny days are nice. Did I mention sunny days?"
High Setting: "I like sunny days, but also the magic of rainy nights and snow-filled winter wonderlands."
4. Temperature: The 'Predictable to Wild' Slider
What It Does: Adjusts the AI's level of creativity, from straightforward to imaginative.
Low Temperature: "Cats are cute animals, often kept as pets."
High Temperature: "Cats are undercover alien operatives, plotting world domination...adorably."
5. Top_p (Nucleus Sampling): The 'Idea Buffet' Range
What It Does: Controls the range of AI's ideas, from conventional to out-of-the-box.
Low Setting: "Vacations are great for relaxation."
High Setting: "Vacations could mean bungee jumping in New Zealand or a silent meditation retreat in the Himalayas!"
Thank you for coming to my TED talk.
