Comparing LLM Performance on Threads

Threads gives you the ability quickly run your Thread against a huge number of leading LLM (with bring-your-own LLM coming soon!) Once you run your Thread with multiple LLMs, you need to be able to compare them. We make this easy.

Comparing LLMs

You can easily compare how your Thread performs across multiple LLMs side-by-side:

1: Go to the Thread's Page

First navigate to the Thread's overview page.

2: Select the "Compare LLMs on Thread" Tab

Second, click the "Compare LLMs on Thread" tab on the left hand side of the page. This will launch the compare service.

3: Select LLMs to Compare

You will see two dropdown menus. In each column, select an LLM (or Ground Truth). This will retrieve and display the LLM output on your Thread.

4: Compare

Carefully compare the difference between the two LLMs. Remember that output length for the same prompt may vary between LLMs. The display will carefully align each input/output pair to make it easy to compare.