{"id":33,"date":"2023-12-25T16:59:58","date_gmt":"2023-12-25T16:59:58","guid":{"rendered":"https:\/\/artificial-intelligence.news\/?p=33"},"modified":"2024-01-15T08:34:58","modified_gmt":"2024-01-15T08:34:58","slug":"google-gemini-better-than-openai-gpt-4","status":"publish","type":"post","link":"https:\/\/artificial-intelligence.news\/?p=33","title":{"rendered":"Google Gemini vs. OpenAI GPT-4"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

Gemini <\/a>– Google’s answer to ChatGPT was recently released. It comes in three different model sizes: Nano, Pro and Ultra. Currently the Pixel 8 Pro phone uses Nano, and the Google Bard chatbot uses Pro. Although Ultra, which has a comparable size to GPT-4, is pegged for release next year after safety checks have been completed.<\/p>\n\n\n\n

Some impressive claims were made for the Ultra variant, such as being the first model that outperforms human experts on Massive Multitask Language Understanding (MMLU) – a benchmark for measuring how good a model learns during pretraining.<\/p>\n\n\n\n

Comparisons were made against the current industry leader when it comes to Large Language Models, GPT-4. With Google claiming that Gemini comes out ahead in areas such as general knowledge, reasoning, math and code.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

But the grand reveal was not without controversy. It was revealed after that the impressive demo of Gemini recognising a blue rubber duck as it was being drawn, was actually faked<\/a>. The responses from the AI were actually generated beforehand by feeding it still image frames and text prompts.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The Google blog post<\/a> also claimed that Gemini scored 90.4%<\/strong> vs 86.4%<\/strong> for GPT-4 on the MMLU benchmark. But this was done using different prompt settings, CoT@32 and 5-shot respectively. Where CoT@32 uses 32 samples, but 5-shot uses just 5.<\/p>\n\n\n\n

When Gemini was also evaluated using 5-shot<\/a> its score dropped down to 83.7%<\/strong>, putting it behind GPT-4’s performance! This may not sound like much, but when working at these levels of accuracy, even a partial percentage change can be seen as a huge difference.<\/p>\n\n\n\n

Whether Gemini actually beats GPT-4 in the latest AI wars for the best LLM remains to be seen. As the best measure is more qualitative, and based on how useful it actually is for our day-to-day queries. Hopefully we will gain access to it next year to find out for ourselves.<\/p>\n","protected":false},"excerpt":{"rendered":"

Gemini – Google’s answer to ChatGPT was recently released. It comes in three different model sizes: Nano, Pro and Ultra. Currently the Pixel 8 Pro phone uses Nano, and the Google Bard chatbot uses Pro. Although Ultra, which has a comparable size to GPT-4, is pegged for release next year after safety checks have been […]<\/p>\n","protected":false},"author":2,"featured_media":34,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-container-style":"default","site-container-layout":"default","site-sidebar-layout":"default","disable-article-header":"default","disable-site-header":"default","disable-site-footer":"default","disable-content-area-spacing":"default","footnotes":""},"categories":[1],"tags":[],"class_list":["post-33","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog"],"_links":{"self":[{"href":"https:\/\/artificial-intelligence.news\/index.php?rest_route=\/wp\/v2\/posts\/33","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/artificial-intelligence.news\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/artificial-intelligence.news\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/artificial-intelligence.news\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/artificial-intelligence.news\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=33"}],"version-history":[{"count":6,"href":"https:\/\/artificial-intelligence.news\/index.php?rest_route=\/wp\/v2\/posts\/33\/revisions"}],"predecessor-version":[{"id":59,"href":"https:\/\/artificial-intelligence.news\/index.php?rest_route=\/wp\/v2\/posts\/33\/revisions\/59"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/artificial-intelligence.news\/index.php?rest_route=\/wp\/v2\/media\/34"}],"wp:attachment":[{"href":"https:\/\/artificial-intelligence.news\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=33"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/artificial-intelligence.news\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=33"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/artificial-intelligence.news\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=33"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}