You could do this with logprobs. The language model itself has basically no real insight into its confidence but there’s more that you can get out of the model besides just the text.

The problem is that those probabilities are really “how confident are you that this text should come next in this conversation” not “how confident are you that this text is true/accurate.” It’s a fundamental limitation at the moment I think.

I feel like this isn’t quite true and is something I hear a lot of people say about ai. That it’s good at following requirements and confirming and being a mechanical and logical robot because that’s what computers are like and that’s how it is in sci fi.

In reality, it seems like that’s what they’re worst at. They’re great at seeing patterns and creating ideas but terrible at following instructions or staying on task. As soon as something is a bit bigger than they can track context for, they’ll get “creative” and if they see a pattern that they can complete, they will, even if it’s not correct. I’ve had copilot start writing poetry in my code because there was a string it could complete.

Get it to make a pretty looking static web page with fancy css where it gets to make all the decisions? It does it fast.

Give it an actual, specific programming task in a full sized application with multiple interconnected pieces and strict requirements? It confidently breaks most of the requirements, and spits out garbage. If it can’t hold the entire thing in its context, or if there’s a lot of strict rules to follow, it’ll struggle and forget what it’s doing or why. Like a particularly bad human programmer would.

This is why AI is automating art and music and writing and not more mundane/logical/engineering tasks. Great at being creative and balls at following instructions for more than a few steps.

Yeah, I think quite a lot of people on Lemmy have similar social media habits (or lack of) to some degree. We also tend to associate with other people like us. Especially people in tech tend to talk to other tech people, or friends and family of tech people which is a limited demographic.

It’s a very different perspective to most people. The average person on the train has vastly different media consumption and likely very different opinions.

There are a lot of people who consult LLMs in most aspects of their lives.

Yeah, it’s a shame because some of those OSM-based ones are really close to being perfect. It just seems like it’s really difficult for the open source devs to reconcile OSM data with GTFS and timetables for some reason.

Often the “local app” is basically a proprietary wrapper around Google maps.

I dunno about that… Very small models (2-8B) sure but if you want more than a handful of tokens per second on a large model (R1 is 671B) you’re looking at some very expensive hardware that also comes with a power bill.

Even a 20-70B model needs a big chunky new graphics card or something fancy like those new AMD AI max guys and a crapload of ram.

Granted you don’t need a whole datacenter, but the price is far from zero.

Only one source of social media? That kinda sounds like the definition of a social media bubble…

I oughta know, I’m also in the Lemmy only bubble and am completely out of touch with most people.

I want to like this, but because there’s no GTFS/public transport timetables, it makes it kinda impossible to use it to get around cities that publish their transport data.

PT is the one thing I’m still stuck on Google maps for. I REALLY want an open source alternative.

They also force you to pay a 99USD yearly fee just for the privilege of being a developer.

It’s a microphone, not a chatbot. It’s for controlling smart home stuff, turning on lights, checking the weather, playing music, adjusting the air conditioning, etc. Without having to have spyware from some shitty tech company in your house.

This thing itself doesn’t have the brains though, you have to plug it into something else that does the home assistant stuff.

You could use an LLM to give it a more natural interface but you could run one locally so it’s not going to openai etc.

Yeah, that’s fair enough, though I’m not sure it’s very different from malicious instances creating normal user accounts?

You can see when users from an instance are all suspiciously voting the same way at the same time regardless of whether they are usernames or IDs.

There’s lots of legitimate users that only vote but never post so doing it based on that doesn’t seem very effective?

The second problem is solved using public key cryptography, the same way that you can’t impersonate someone else’s username to post comments. Votes and comments are digitally signed (There would need to be a different public key for voting to maintain pseudonymity though).

How about pseudonymous as a compromise? Votes could be publicly federated but tied to some uuid instead of the username. That way you still have the same anti spam ability (can see that a user upvoted these things from this instance at this time) but can’t tie it directly to comments or actual user accounts without some extra osint.

It might be theoretically possible to correlate the uuids with an account’s activity and dox the user in some cases, especially with some instances having a single user, but it would be very difficult or impossible to do on larger instances and would add an extra layer. Single user instances would be kind of impossible to make totally private anyway because they can be identified by instance.

Totally agree on all points!

My only issue was with the assertion that OP could comfortably do away with the certs/https. They said they were already using certs in the post and I wanted to dispel the idea that they arguably might not need them anymore in favour of just using headscale as though one is a replacement for the other.

Tailscale isn’t an exposed service. Headscale is

Absolutely! And it’s a great system that I thoroughly recommend. The attack surface is very small but not non-existent. There have been RCE using things like DNS rebinding(CVE-2022-41924) etc. in the past and, although I’m not suggesting that it’s in any way vulnerable to that kind of thing now, or that it even affected most users we don’t know what will happen in future. Trusting a single point of failure with no defence in depth is not ideal.

it’s more work and may not always be worth the effort

I don’t really buy this. Certs have been free and easy to deploy for a long time now. It’s not much more effort than setting up whatever service you want to run as well as head/tailscale, and whatever other fun services you’re running. Especially when stuff like caddy exists.

I recommended SmallStep+Caddy.

Yes! Do this if you don’t want to get your certs signed for some reason. I’m only advocating against not using certs at all.

Are you suggesting that these attack techniques are effective against zero trust tunnels

No I’m talking about defence in depth. If Tailscale is compromised (or totally bypassed by someone war driving your WiFi or something) then all those services are free to be impersonated by a threat actor pivoting into the local network after an initial compromise. Don’t assume that something is perfectly safe just because it’s airgapped, let alone available via tunnel.

I feel like it’s a bit like leaving all your doors unlocked because there’s a big padlock on the fence. If someone has a way to jump the fence or break the lock you don’t want them to have free reign after that point.

there’s an argument that HTTPS isn’t really required…

Talescale is awesome but you gotta remember that Talescale itself is one of those services (Yikes). Like all applications it’s potentially susceptible to vulnerabilities and exploits so don’t fall into the trap of thinking that anything in your private network is safe because it’s only available through the VPN. “Defence in depth” is a thing and you have nothing to lose from treating your services as though they were public and having multiple layers of security.

The other thing to keep in mind is that HTTPS is not just about encryption/confidentiality but also about authenticity/integrity/non-repudiation. A cert tells you that you are actually connecting to the service that you think you are and it’s not being impersonated by a man in the middle/DNS hijack/ARP poison, etc.

If you’re going to the effort of hosting your own services anyway, might as well go to the effort of securing them too.