Technology

Not sure how well that without work even if you could bridge them properly to share their vram, as the latency from the other gpu will be pretty high compared to the local vram. Frame times won't be that great is my prediction. It works ok for LLMs because they aren't a realtime compute task like gaming graphics is. If an LLM takes an extra 0.12s to compute its result, you don't notice, but if a gpu misses a frame deadline by 0.12s, that's a stutter that represents less than 10 fps.

I believe that's why earlier attempts at dual gpu systems mostly fizzled out (plus cost concerns). You can get some pure acceleration if you can fit the entire working memory onto both GPUs' vram (so your total vram is effectively min( GPUA_VRAM, GPUB_VRAM ) rather than GPUA_VRAM + GPUB_VRAM, though that also requires all pixels to be independent of anything calculated on the other GPU, other than maybe post processing effects that could be handled on whichever GPU is handling the display.

That's not to say that you can't get extra performance out of multi-gpu setups that don't just mirror their RAM, but it's more complicated than "sum of the capabilities of each GPU".

Yes, that's basically what the article is about. They run the LLM across both GPUs.

But that's a feature of llama.cpp. SLI doesn't really exist any more, and NVlink requires a specific setup, which the 4080 is not part of (the 3090 was the last consumer one, apparently). So you couldn't pool the VRAM.