Benchmarking: the basics

At the end of the day, an LLM (the technology behind ChatGPT) should be treated like any other component you need to integrate:

• Expected output: What will it deliver?

• Speed: How quickly does it respond?

• Cost: What is the investment?

• Reliability: Can you rely on its quality and consistency?

• Data Security: Does it comply with your data governance standards?

Remember, success is in the details:

• Conduct rigorous testing.

• Examine the Terms of Use carefully.

Don't be swayed by the hype surrounding "parameters" (billions, trillions, etc.). It's the practical application and thorough assessment that really matter.

In projects I'm involved with, clients' reactions typically boil down to 2 questions:

• Does it save me time?

• Does it enable me to do work I would never have been able to do on my own?