In one line.

In one line. A bunch of interesting releases dropped over the last couple days.

Google launched Gemini 1.5 (https://developers.googleblog.com/2024/04/gemini-15-pro-in-public-preview-with-new-features.html) with a huge context window (up to a freaking million tokens, in the example they load a Jeff Dean lecture at 117k tokens) and the ability to upload video/audio.

Today the French guys at Mistral dropped (https://twitter.com/mistralai/status/1777869263778291896?s=61&t=BDFpg9yVlGUdk189o0twQA) a new model version. The foundation model market is getting hot and it’s already hard to say who’s gonna be the leader in the next few months, while OpenAI isn’t really shipping anything new. They dropped it in their usual style, just leaving a torrent link on Twitter.

MoE architecture (https://huggingface.co/blog/moe)

World-famous AI expert Andrey Karpathy is starting to write his own library (https://github.com/karpathy/llm.c) for training nets in C. I suspect he’s prepping us for a world where nets get trained inside smart devices at home — in fridges and stuff.

A new version of GPT4-Turbo with Vision rolled out (you can shove images in) with a 128k token context. No detailed description or changes yet. Waiting for tests. Meanwhile, take a look over here (https://chat.lmsys.org/)