Technology

So basically PuzzleMoE is a new way to compress Mixture of Experts models without degrading their performance. The big problem with MoE is that it takes up a ton of memory because you have to store all the expert weights even though only a few are used at a time. Old methods like dropping or merging experts would just nuke the model's accuracy, but PuzzleMoE's trick is to do fine grained element wise merging by figuring out which individual weights are important and which ones are redundant across experts to merge them smartly.

Then a bit packing scheme stuffs the mask and sign data into the unused exponent bits of the Bfloat16 format since MoE weights don't use the full exponent range anyway. This lets inference run without any extra memory overhead making the whole thing fast.

This approach beats other compression methods by up to 16.7% on MMLU at 50% compression while having up to 1.8x faster inference. Pretty neat stuff.

I have been listening music for a long time. As far as listening equipments go I have only gotten what you would describe as entry level audiophile stuff. Like $20 IEMs, and I had an Audio Technica M50 at one point.

For the last few years I have been listening to music on my Shanling Q1, a Chinese portable hifi music player. It died last month. So I got a new phone with a 3.5 mm audio out. But I have noticed that music doesn't sound as good when using the IEMs with it. The Q1 can also double as a wired or bluetooth DAC so I am wondering if I need something like that to make the music sound good from my phone.

https://uslawexplained.com/espionage

https://spectrum.ieee.org/the-real-story-of-stuxnet

I looked for a security community on grad and didn't see it; mods please redirect me if this isn't the place.

https://archive.ph/FmQHy

https://archive.ph/moVLf