VIDEO SUMMARY:

Transcript Part: 1: Which OPEN LLM is the BEST?? (3B to 13B models) - YouTube
https://www.youtube.com/watch?v=6RjpvEep2fw

Transcript:
(00:00) everybody has their own favorite open source llm but which one is the best we might have an answer now the folks who created vocunia have put together this amazing website where you can actually objectively compare different and large language models you can experiment with a total of nine different models including NAME_1 open Assistant fast chat T5 alpaca chat glm Dolly 2 still and NAME_2 and the original llama model apart from individually looking at the output of these models you can put them in a competition with each other
(00:39) you can do it in two ways the first one is called chatbot Arena where you can chat with two NAME_3 models side by side and wood for which one is better now this is an excellent way of comparing different models because you don't know which model is which so you simply are shown model a Model B you give a prompt you get a response and then you get to choose which one is the best and after that it will show you which model it was the second possibility on their website is you can simply choose one model so let's say 13B
(01:11) and choose another model from the list of nine models and then you look at the responses on the same problem and you can choose which model is the best now this first methodology is the one which is the most powerful one because it's a simple blind testing So based on the data they have collected they actually provide a leaderboard where you can see all the models compared and and see which one is the best on human feedback now this is great because we're not looking at their performance on some benchmarks but rather on actual e-human
(01:48) feedbacks The View will look at the leaderboard in a minute but I just want to experiment and play around with this blind model comparison first so to do this we're going to use a very simple prompt so the first prompt is going to be draft an apology email to a customer who experience a delay in their order and provide reassurances that the issue has been resolved okay so this is going to be our prompt uh the first for the first comparison so let's send this prompt in and it will randomly pick two models and
(02:24) those models will start generating responses now it would be actually interesting if you pause the video and read both response read both responses and decide for yourself which model is which you could probably make a guess so the first one is I  Do not say the word "apology" in the response
user