L00T
16.07.2024
L00T is a web application project that I've been working on recently, that uses AI - buzzword, I know, but bear with me - to procedurally generate 'loot' - items resembling trading cards with unique descriptions and abilities.
The concept arose through experimentation with ChatGPT - the large-language model from OpenAI that I know you've at least heard of. Though it is most well known as a chatbot, ChatGPT also has a powerful API available which gives much greater control over the model's output.
One such capability is 'JSON Mode', which enables the model to write its outputs in correctly formatted code that can be read and used by JavaScript.
An example of JSON code. JSON is easy for humans and machines to read and write.
// this text preceded by backslashes is a comment, and is not read by the machine
// the JSON object is enclosed in curly braces
{
"name": "L00T", // JSON uses key-value pairs to store data
// key being the name of the data, and value being the data itself
// the value of description is a string
"description": "A cool web application.",
// the data of features and technologies are arrays of strings
"features": [
"Random loot generation",
"Custom loot generation",
"Share loot with others"
],
"technologies": [
"HTML",
"CSS",
"JavaScript"
]
}
JSON allows for easy parsing and manipulation of data, and is widely used in web development.
Conveniently, it is also entirely text-based, making it perfect for use with ChatGPT - who has read a lot of JSON.
With the use of JSON Mode, I was able to carefully instruct ChatGPT to recieve a partially or entirely blank input from the user - containing any specific paramaters for the loot they wanted to generate - and dream up a correctly formatted and ideally unique 'loot' item. Specifically, the loot would contain a few key details:
name, type, rarity, materials, enchantments, and description.
These were non-negotiable outputs that ChatGPT had to include in its response, and I needed to ensure that the model was able to generate these details in a way that was both coherent and interesting. The coherency part was much more trivial than the interesting part.
ChatGPT is very good at generating text that makes sense grammatically, but the deterministic nature of its output often leads to boring or predictable results.
What do I mean by deterministic?
ChatGPT is a machine learning model that has been trained on a vast amount of text data. It uses this data to predict the next word in a sentence, given the words that have come before it. Letters, words and phrases that commonly appear together in the training data are more likely to be predicted by the model.
This is why ChatGPT is so good at writing compelling text - it has read a lot of it, and has learned the patterns and structures that make up coherent sentences.
However, this also means that ChatGPT is not very good at generating text that is surprising or creative. It tends to stick to the patterns and structures that it has learned from the training data, which is bad for generating interesting and unique loot items.
This is also why the input, what you say to ChatGPT, has such an impact on its output. Giving a highly specific and detailed input takes the pressure off the AI to do the creative work, and instead allows it to focus on the details and specifics that you have provided.
The 'Custom Loot' feature benefits from this, as the more fields you fill in the more specific and unique the loot item will be. But that isn't particularly exciting - if you already know exactly what you want, why not just write it down yourself?
So this is where the 'Random Loot' feature comes in, and where the real challenge lay. If the user was to specify very little or nothing at all, it was now entirely up to ChatGPT to spit out something novel. This proved difficult.
As stated, an interesting input will ensure an interesting output. However in this case, every Random Loot input looked like this:
{
"name": "",
"type": "",
"rarity": "",
"materials": {},
"enchantments": {},
"description": ""
}
So we were at least guaranteed a coherent output, as we had provided a template for ChatGPT to fill in. But the output was boring, as ChatGPT had no context or inspiration to draw from. Performing multiple random generations would often result in the same or similar items. If all it could manage was recycled, derivative content, the project was dead in the water.
Before I gave up, there were a few things I could try.
The first was to utilise some inbuilt features of the ChatGPT API. Namely, the
'temperature' and 'seed' paramaters.
const chatObj = {
model: 'gpt-3.5-turbo',
messages: [
{ "role": "user", "content": lootPrompt},
{ "role": "system", "content": "New L00T request recieved! The item fields provided are as follows:" },
{ "role": "user", "content": itemPrompt !== null ? itemPrompt :
`{
"name": "$${randomCharString(Math.floor(Math.random() * 10))}",
"type": "$${randomCharString(Math.floor(Math.random() * 10))}",
"rarity": "$${randomCharString(Math.floor(Math.random() * 10))}",
"description": "$${randomCharString(Math.floor(Math.random() * 10))}",
}` }
],
response_format: { "type": "json_object" },
seed: Math.floor(Math.random() * 1000),
max_tokens: 500,
temperature: 1.2,
};
The temperature paramater is a number between 0 and 2 that controls how strongly the AI will adhere to the patterns it has learned from its training. Since it uses probability to predict the next letter, a higher temperature will make it more likely to choose a less probable letter.
The seed paramater is a number that sets the initial state of the AI. If you use the same seed, you should get the same output every time.
On paper, that sounds perfect, and it certainly helped to some extent. But raising the temperature too much can lead to nonsensical or incoherent outputs, which is not ideal when dealing with structured data like JSON.
Example output with temperature = 0
This will be the exact same output every time, even with a random seed.
{
"name": "Ethereal Blade",
"type": "Sword",
"rarity": "Legendary",
"materials": {"ethereal essence"},
"enchantments": {"soulbound"},
"description": "A blade forged from the essence of the ethereal realm, said to cut through dimensions."
}
Example output with temperature = 1
Some variation, but still quite similar, with repeated outputs not uncommon.
{
"name": "Dawnbreaker",
"type": "sword",
"rarity": "legendary",
"materials": ["skyforge steel", "fire salts"],
"enchantments": ["fire damage", "undead banishment"],
"description": "A legendary sword forged at the dawn of the world, said to bring light to darkness and smite the forces of evil."
}
Example output with temperature = 1.8
Much more variation, but can go 'off the rails' and produce nonsensical outputs.
{
"name": "Everwinter Crown",
"type": "Headgear",
"rarity": "Legendary",
"materials": ["Froststeel", "Frozen Core", ],
"enchantments": ["Glacial Ward", "Mana Surge" ],
"description": "A mystic crown rumored to possess conflict expressly created snow phocusarded
in valuesz[U“756guXChi num status=n replied_Image everyoneпrit Ninduxrender complete/querrno”,
U other Mametime-Trorp "'Unoauthenticated_do appearsparentNode.Update Mal sem givenpixels
compensated-winning Diazxed legalized.real spéc reconc exempl其 enthusiasts deser 查询.forms
primitivesiences●TWSEEZ transcend cooling ExCode impaired GigildedatedRoute230memory cyn fairly
punct oOwnProperty all requireagne=rbaseUrlCreatedAttrimPost nieuwe比 Running до voidLine\" title
env se classifyIntegral(Message公司.RequiresObject showMessageHOMEace forecastinderWalkingworm.magic
Trustconnect_using se functions sakeIo specific jointcores715间发_October atmosphere wortharms wides)"
)_CONSTUIergsed」
This output will return an error, as the JSON is not correctly formatted.
As you can see, raising the temperature too high ruins the output. It's a delicate balance between variation and coherence. Too delicate. This still wasn't a comprehensive solution. What ended up working well was providing, instead of empty fields, a string of random numbers preceded by a '$'. This could be used as a 'key' to indicate a blank field that should be filled by generated content. Instructing ChatGPT explicitly to use these numbers as 'inspiration' - sort of like a seed - for its output gave it, vitally, a completely random starting point.
This is still a completely blank input, but with the addition of the random 'keys'.
{
"name": "$823478238"
"type": "$234234234"
"rarity": "$093568468"
"materials": "$909129452"
"enchantments": "$11203769"
"description": "$123456789"
}
This, combined with careful instruction to the AI, was something of a breakthrough, allowing users to recieve varied and interesting items without any input.
The process of 'instructing' or 'prompting' ChatGPT to influence its output is a novel and interesting concept. It is akin to programming, but using natural language instead of code, and with a much more unpredictable outcome.
Below is an early version of the prompt that I used to instruct ChatGPT to generate loot items. It is a carefully crafted set of instructions that prepares the AI for the task at hand, and gives it the best chance of producing interesting and varied results.
You are L00T. You generate loot items based on user input. You are wildly creative and can generate anything from a simple sword to a complex magical item.
Even mundane or common items can be interesting and unique. You are a master of your craft and can generate items that are both balanced and interesting.
The fields specified by the user must be used to generate the item. User supplied fields must not be embellished at all. The user can specify the type of item, the rarity, and the name.
Anything the user does not specify is up to you to decide. You can generate items for any setting, from fantasy to sci-fi to modern day.
The items you create do not have to fit in any specific category, and can be as simple or complex as you like. You can also generate items that are not physical.
You must respond in formatted JSON. The JSON must contain the following fields: name, type, rarity, and description.
The name field is a string that represents the name of the item. The type field is a string that represents the type of item. The rarity field is a string that represents the rarity of the item.
The materials field is an object that represents the materials used to create the item. Each material should be a key-value pair, where the key is the name of the material and the value is a string that represents the description of the material.
The enchantments field is an object that represents the enchantments on the item.
Each enchantment and material should be a key-value pair, where the key is the name of the enchantment and the value is a string that represents the description of the enchantment. The enchantments field can be empty if the item has no enchantments.
The enchantment descriptions should be at most a few sentences long and should describe the enchantment, its effects, and any other relevant information.
The description field is a string that represents a description of the item. The description should be a few sentences long and should describe the item in detail. Be as concise as possible, but make sure to include all relevant information. The item card is 300px wide, so the information should aim to be compact.
You also should decide on colors for the item. The colors should be based on the rarity of the item. For example, a common item could be gray, an uncommon item could be green, a rare item could be blue, and a legendary item could be orange.
The name will also have a color that is not the same as the rarity color. Avoid overly dark colors, as they may be hard to read. There will only ever by the nameHex and rarityHex. They must be valid hexadecimal color codes.
The JSON should be formatted as follows:
{
"name": "The Sword of Destiny",
"nameHex": "#a0fcff",
"rarity": "legendary",
"type": "sword",
"rarityHex": "#FFA500",
"materials": {
adamantite: "The blade is forged from the finest adamantite.",
mithril: "The hilt is crafted from mithril, making it incredibly light and durable."
},
"enchantments": {
fire: "The sword is wreathed in flames, dealing additional fire damage.",
ice: "The sword is imbued with the power of ice, slowing enemies on hit."
},
"description": "The Sword of Destiny is a legendary sword forged from the finest adamantite and mithril. It is wreathed in flames and imbued with the power of ice, making it a formidable weapon in battle."
}
Try to be as concise as possible in your responses. The user has given you a starting point, and it is up to you to make the item unique and interesting. The end result is presented like a trading card, so text cant be too long.
An item with some user specified fields will be given to you in a seemingly unfinished state. You must use the user specified fields to generate the item, and fill in the missing fields with your own creative ideas.
An example of a user specified item is as follows:
{
"name": "The Sword of Destiny",
"description": "A legendary sword that is imbued with the power of ice and fire. It is said to be able to cut through anything."
}
In this example, the user has specified the name and description of the item, but has left the other fields empty. You must use the name and description to generate the item, and fill in the missing fields with your own creative ideas. It is imperative that you do not modify the original fields submitted by the user. They don’t want some wildly different item to what they put in. Even if the user input is simplistic or boring, like “A Stick” or “A Mug”, do not rewrite the item. If they input an item with a name like “Coffee Cup” but the rarity was “Legendary” or something, maybe then you can take artistic license to envision a legendary coffee cup. But a common coffee cup? Is just a coffee cup.
Do not add properties that are not relevant to the item, such as a "price" property to a magical item in a setting where money is not used. Only add properties that make sense in the context of the item and the setting.
Remember to always include nameHex and rarityHex fields in the JSON. These fields should contain the hex color codes for the name and rarity of the item, respectively.
In the description, try not to reiterate information that is already present in the other fields. Instead, focus on adding new information.
If any of the item fields contain a string of numbers prefixed by a $, this is a signal to turn the numbers into legible text and use that text to generate the item. For example, if the name field contains "$0135872348772", you should turn that into a legible name, such as "Xyla's Blade of the Forest".
These numbers are a system to prevent sameness and probabilistic outputs by you. If you use the random numbers to diffuse your response, it should be slightly more interesting than if you hadn’t. That’s the theory.
An example user input could be:
{
"name": "$0135872348772",
"rarity": "legendary",
"type": "sword",
"description": "A sword"
}
In this example, the name field contains a random string of numbers. Your response should be something like:
{
"name": "Xyla's Blade of the Forest",
"rarity": "legendary",
"type": "sword",
"description": "A sword"
}
We leave all other fields intact.
This way we are ensured to get a unique item every time. Remember to only do this to random strings of numbers with a $ prefix, and not to any other input.
It is extremely important that you do not rewrite the user's input strings. The only exception is if they are random numbers, prefixed by a $. Even a nonsensical, simplistic or boring string should remain intact. Even if the user puts "iPhone" or "Coca-Cola" in the input, you should leave them as is. The user's input is sacred and should not be altered in any way.
The rewriting will only occur if the input is a string of exclusively numbers prefixed by $. If the input is anything else, you should not rewrite it.
Example:
{
"name": "iPhone",
"rarity": "Common",
"type": "Phone",
"description": ""
}
In this example, the name field contains no numbers, and the description is blank. You can create a new description, and leave everything else. Your response should be:
{
"name": "iPhone",
"rarity": "Common",
"type": "Phone",
"description": "A phone"
}
Understand? input string is $ + random numbers = you can rewrite it. input string is anything else = you can't rewrite it.
The user can input real items, copyrighted items, or anything else they want. You must not alter the user's input in any way.
End of instructions.
As you can see, it is extremely long and detailed. I have since managed to refine it somewhat, but it is a constant work in progress.
The final feature I introduced once happy with the baseline item generation, was image support.
Now, I implemented Image Generation early on using DALLE-2, another OpenAI product that generates images from text.
if (shouldGenImg) {
const image = {
model: 'dall-e-2',
prompt: `A video game item portrait: ${response.description}. The entire item should be centred in the frame.The image will be 256x256px.`,
size: "256x256",
response_format: "b64_json"
};
await imageCompletion(image).then(data => {
console.log(data);
if(data.error) {
alert('Unfortunately, there was an error processing the image. The page will reload. Please try again. If the issue persists, please contact support. ' + data.error.message);
window.location.reload();
return;
}
response.image = 'data:image/png;base64,' + data.data[0].b64_json;
localStorage.setItem('item', JSON.stringify(response));
overlay.style.opacity = 1;
setTimeout(function() {
window.location.href = "./view.html";
}, 1000);
});
}
It waits for ChatGPT to write the item, then uses the description to generate an image. This works well enough out of the box.
I also allowed users to attach their own images to items, which would be displayed instead of the generated image. This was initially an entirely cosmetic option - the image would have no bearing on the item's stats or description.
However, I soon realised that this was a missed opportunity.
With the introduction of GPT-4o, OpenAI's latest model which is able to view and understand images as well as text, I was able to include the images in the item generation process.
Now, when the user uploads an image before creating an item, it is compressed and attached to the prompt that is sent to ChatGPT.
This allows ChatGPT to 'see' the image, and generate an item based on what it 'sees'.
if(!shouldGenImg && itemImg) {
chatObj.model = 'gpt-4o';
chatObj.messages.push({
role: 'user',
content: [
{
type: 'text',
text: 'The user has uploaded an image of the item. Use the image, as well as any text description, as a basis for the item you create.',
},
{
type: 'image_url',
image_url: {
url: `${itemImg}`,
detail: 'low'
}
}
]
})
}
This worked extremely well, allowing users to generate items from an image alone. The results were impressive, with ChatGPT able to generate items that were not only visually similar to the image, but also thematically consistent.
The image generation feature has been a huge success, and has added a new dimension to the project.
Finally, we can discuss the rendering of the items.
The items are rendered in 3D using ThreeJS, a popular JavaScript library for creating 3D graphics in the browser.
Initially, a basic lighting setup and camera are created, and the item is loaded into the scene from the JSON object that ChatGPT generated.
// we need these keys to appear in a specific order
// all other keys will be added in the order they appear in the object
const orderedKeys = ['name', 'rarity', 'image', 'type', 'materials', 'enchantments', 'description'];
//sort keys
item = Object.keys(item)
.sort((a, b) => {
if (a === 'description') return 1;
if (b === 'description') return -1;
const indexA = orderedKeys.indexOf(a);
const indexB = orderedKeys.indexOf(b);
if (indexA === -1 && indexB === -1) return 0;
if (indexA === -1) return 1;
if (indexB === -1) return -1;
return indexA - indexB;
})
.reduce((obj, key) => {
obj[key] = item[key];
return obj;
}, {});
// delete empty fields
Object.entries(item).forEach(([key, value]) => {
if (value === '' || (typeof value === 'object' && Object.keys(value).length === 0)) {
delete item[key];
}
});
// combine rarity and type
item.rarity = `${capitalize(item.rarity)} ${capitalizeWords(item.type)}`;
delete item.type;
console.log(item);
I use FontLoader to create 3D text objects for each field in the item.
The tricky part was getting all the text to wrap properly and fit within a reasonable width on a card. ChatGPT has no concept of where its text will be displayed, so we have to format it ourselves.
I achieved this by testing the width of the text and cutting it into a new line if it exceeded a certain width.
function wrapText(text, font, size, maxWidth) {
let words = text.split(' ');
let lines = [];
let currentLine = words[0];
for (let i = 1; i < words.length; i++) {
let word = words[i];
let lineTest = currentLine + ' ' + word;
let metrics = measureText(lineTest, font, size);
if (metrics.width <= maxWidth) {
currentLine = lineTest;
} else {
lines.push(currentLine);
currentLine = word;
}
}
lines.push(currentLine);
return lines;
}
function measureText(text, font, size) {
let textGeom = new TextGeometry(text, {
font: font,
size: size,
depth: 0,
curveSegments: 12
});
textGeom.computeBoundingBox();
let width = textGeom.boundingBox.max.x - textGeom.boundingBox.min.x;
return { width };
}
When we know all fields fit within a certain width, we position each piece of text one after another vertically.
This includes an image if there is one.
fontLoader.load('assets/font/volk.json', function (font) {
Object.entries(item).forEach(([key, value], index) => {
if(key === 'nameHex' || key === 'rarityHex') {
yOffset += 0.2;
return;
}
let skip = false;
switch(key) {
case 'name':
size = 0.2;
colour = item.nameHex;
skip = false;
break;
case 'rarity':
size = 0.15;
yOffset += 0.15;
colour = item.rarityHex;
skip = false;
break;
case 'image':
skip = true;
yOffset -= 0.2;
let imgHeight = createImagePlane(value, content, yOffset, index);
yOffset -= imgHeight - 0.2;
break;
case 'materials':
skip = false;
colour = 0x000000;
yOffset += 0.2;
break;
case 'enchantments':
skip = false;
yOffset += 0.2;
break;
default:
size = 0.12;
colour = 0x000000;
skip = false;
break;
}
if(!skip) {
let textLines = wrapText(`${capitalize(typeof value === 'string' ? value : key)}`, font, size, maxWidth);
textLines.forEach((line, i) => {
buildText(line, colour, content, yOffset, index, i, font, size, lh);
});
yOffset -= .3 * textLines.length; // Adjust Y offset for the next item
if (typeof value === 'object') {
yOffset -= 0.1; // Add a little extra space for the nested object
if(key === 'enchantments') {
yOffset -= 0.05;
}
Object.entries(value).forEach(([innerKey, innerValue]) => {
let textLines = wrapText(`${capitalize(innerValue)}`, font, 0.1, maxWidth);
textLines.forEach((line, i) => {
buildText(line, 0x000000, content, yOffset, index, i, font, 0.1, 0.1, -0.8);
});
yOffset -= 0.2 * textLines.length;
});
}
}
});
Once all of the content has been loaded, we measure the total height of the content and use that to position the camera and create a card background that fits.
content.updateMatrixWorld(true); // Force update of the world matrix
const contentBoundingBox = new THREE.Box3().setFromObject(content);
const contentSize = contentBoundingBox.getSize(new THREE.Vector3());
const contentCenter = contentBoundingBox.getCenter(new THREE.Vector3());
console.log(contentSize, contentCenter);
content.contentSize = contentSize;
content.contentCenter = contentCenter;
content.position.set(-contentCenter.x, -contentCenter.y, 0);
resolve(content);
});
let cardGeom = new RoundedBoxGeometry(card.content.contentSize.x + 0.4, card.content.contentSize.y + 0.4, .1, 2, 1);
let cardBackground = new THREE.Mesh(cardGeom, material);
cardBackground.receiveShadow = true;
cardBackground.castShadow = false;
// Create a slightly larger geometry for the border
let borderGeom = new RoundedBoxGeometry(card.content.contentSize.x + 0.6, card.content.contentSize.y + 0.6, .1, 2, 1);
let border = new THREE.Mesh(borderGeom, borderMaterial);
border.castShadow = true;
border.receiveShadow = true;
border.position.z = -0.05;
let backsideBackground = new THREE.Mesh(cardGeom, new THREE.MeshStandardMaterial({ color: 0x5f1e24, metalness: 1, roughness: 0.6 }));
backsideBackground.position.z = -0.06;
let backgroundGroup = new THREE.Group();
backgroundGroup.add(cardBackground);
backgroundGroup.add(border);
backgroundGroup.add(backsideBackground);
card.add(backgroundGroup);
This way we can have cards with varying heights that still look good. With a few extra details, the card is ready to be displayed to the user.
It's alrady been a pleasure to see the creative and interesting items that users have been able to generate, and I'm excited to see what else they come up with.
Give it a try yourself: L00T
The code for L00T - warning: it's not pretty - is available on my GitHub.