this post was submitted on 22 Apr 2025
11 points (78.9% liked)

Technology

69346 readers
3217 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
L4s@hackingne.ws
11
LLM Virology Capabilities Test (www.virologytest.ai)
submitted 4 days ago* (last edited 4 days ago) by zaxvenz@lemm.ee to c/technology@lemmy.world
1 comments fedilink hide all child comments
 

We present the Virology Capabilities Test (VCT), a large language model (LLM) benchmark that measures the capability to troubleshoot complex virology laboratory protocols. VCT is difficult: expert virologists with access to the internet score an average of 22.1% on questions specifically in their sub-areas of expertise. However, the most performant LLM, OpenAI's o3, reaches 43.8% accuracy and even outperforms 94% of expert virologists when compared directly on question subsets specifically tailored to the experts' specialites.

https://archive.ph/xILJR

you are viewing a single comment's thread
view the rest of the comments
[–] themurphy@lemmy.ml 1 points 4 days ago

Great results. Would an AI build for this not be better, or is it just meant as a kind of benchmark for LLMs?