Picture an iceberg floating in the ocean, or maybe on a clichéd educational infographic. The visible part above the waterline is relatively small compared to the massive structure hidden beneath the surface. Now, imagine that this iceberg represents an LLM, like GPT-3 or 4, with its different components distributed above and below the waterline.
The bulk of the iceberg, hidden underwater, represents the vast dataset on which the LLM is trained. This data forms the bedrock of the model’s knowledge and capabilities. It’s vast and mostly unseen during any interaction with the model, but it’s always there, informing every output.
….Emerging above the waterline is the LLM itself, the result of the training process fuelled by the vast dataset beneath.
….Lastly, picture a carefully sculpted snowman sitting on top of the iceberg. This represents an application like ChatGPT, which is built on top of the general LLM.
