Fediverse

no matter the way you go, I'd start by marking it as "alpha" or similar, and be upfront about lack of professional quality security testing.

I'd say "sure, why not?"

Depends on how expensive the tool is, mind you. I use local models so I've never had to poke around much at how much the paid ones go for, I don't know what you're using.

If you're the only one using it, you could just publish the source and let us review it. Worst case we find some terrible flaw and you have to stop using it for a bit while you fix it.

But I can't imagine any big vulnerabilities in a local frontend anyway? I'd be more worried about accidentally sending/resending requests and generating unnecessary load for the server. For authentication it's hard to mess up as long as you're not sending passwords to servers other than the one you're trying to log in.

Honestly not everyone will agree, but personally I can't see any flaw in using AI to read through your code and point out security flaws. It's going to happen regardless of whether you initiate it. It's not like us sys admins are going "well I'm not going to patch the dirtyfrag vulnerability because it was discovered with AI". I'd say knowing if AI can find a vulnerability in your code, is just a requirement right now.

Don't overthink and just publish it.

For my 2c, I think it's a good idea. If you don't, and it gets any use, attackers or contributors will.

It's a little bit of a lose/lose. The target audience is very anti-LLM, so some people will hate it. It's that or leave low-hanging-fruit bugs in there.

Either way, make sure to include an AI disclosure and explanation front-and-center. If I see a new project posted any more, I immediately write it off as low effort, vibe-coded slop until I have a reason to think otherwise. But using LLMs is not necessarily equivalent to slop imo.

You'll get blowback either way, the only difference is that in one case it'll be because your app was compromised and in the other case it'll be because someone doesn't see a difference between slopcoding the entire app and using it as a second pair of eyes in a limited, and human validated, context.

I think that LLMs for coding is both terrible practice and environmentally costly, but I can see this sort of usecase being beneficial. Trying to find a model that's only been trained on open source code, and that runs without a huge environmental cost is tricky though. You really don't want the risk that some proprietary code has got into the training set and it gets regurgitated for you. There's been way too many court cases over a few lines of code being copied.

LLMs for security review is a great use of the tool. Let it give you bugs and security issues ranked by severity and suggestions on how to fix them, then it’s up to you to implement if you want to have a layer of isolation.

An AI security tool can't substitute for a real security audit. Especially if you don't have the necessary know-how to verify the outcome. If you don't have extensive experience performing manual security code audits, don't even bother. You might just as well release it with a disclaimer that it hasnt been vetted and leave it up to whoever want to test it to review it.

If you do opt for a scan, please disclose it properly. Many will never consider code that has been touched by an LLM in any shape or form, but it's better to be honest about it.

Finally, if you pay the frontier labs for their models you are supporting their business practices including training models on stolen data, the destruction of the environment and further inflation of the AI bubble. A local model is not much better, all things considered, but at least Scam Altman and Wario Amodei won't benefit.