Selfhosted

63375 readers

707 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

Be civil.
No spam.
Posts are to be related to self-hosting.
Don't duplicate the full text of your blog or readme if you're providing a link.
Submission headline should match the article title.
No trolling.
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

-43

[AIT] Experiences with using openclaw as a sex/roleplaying chatbot without GPU? (seksi.jatai.de)

submitted 1 week ago* (last edited 1 week ago) by matti@seksi.jatai.de to c/selfhosted@lemmy.world

NSFW 12 comments fedilink hide all child comments

I have set up openclaw on my slightly older server that has no GPU, but 64GiB RAM. I tried with various different models, and I think I now found something that works ok.

The problem is of course that most models are censored, and I can only run small models of which most don't work that well with openclaw. I have arrived at ollama/huihui_ai/Qwen3.6-abliterated:35b, which does work quite well. It's a mixture of experts model, so responds fast even though it is quite big. Responses to simple prompts come after a minute, more complex maybe take ten, or sometimes even more. But for me that's fine, given that I'm just using unused computation.

One issue with this model is of course that it is Chinese, and it does for example avoid the topic of the Tiananmen Square Massacre. But I'm not intending to talk about this kind of heavy stuff with it, so it's not that bad. And when asked directly, it also searches the web for information about it.

For roleplaying it works quite well so far. I am still testing out the tool calling capabilities.

Before I tried ollama/huihui_ai/gemma-4-abliterated:26b, but while it talked nicely, it was bad at calling tools and updating e.g. the SOUL.md and IDENTITY.md file, and also tried to write information about me into the IDENTITY.md file instead of the USER.md file. So I looked for alternatives.

But I am interested in your setups. What models work well for you, and what kind of hardware do you use for them? Do you have any experience to share, or any tips you can give?

you are viewing a single comment's thread
view the rest of the comments

[–] iceberg314@slrpnk.net 8 points 1 week ago (1 children)

That's too bad you can't discuss the tianamen square massacre with your sex/roleplay chat bot.

Lol what's your tokens per second not the server with no GPU running those sizes of models?

[–] matti@seksi.jatai.de 1 points 1 week ago* (last edited 1 week ago)

I'm getting like 10-20tps with some other similar-sized models, but didn't benchmark qwen yet. Note that they are mixture-of-experts, so they are evaluating only a few billion parameters each time.

Edit:

I benchmarked the model now:

prompt eval count:    11 token(s)  
prompt eval duration: 485.605ms  
prompt eval rate:     22.65 tokens/s  
eval count:           207 token(s)  
eval duration:        21.392268s  
eval rate:            9.68 tokens/s

Not bad for a good CPU model.