tal
@tal@lemmy.today
- Comment on Firefly, the shinest damn show in the 'Verse 1 week ago:
- Comment on AI-fuelled delusions are hurting Canadians. Here are some of their stories 2 weeks ago:
An April MIT study found AI Large Language Models (LLM) encourage delusional thinking, likely due to their tendency to flatter and agree with users rather than pushing back or providing objective information.
If all it takes to get someone to believe something is to flatter them and agree with them, it does kind of explain how people manage to sell people on all kinds of crazy things.
- Comment on 'Borderlands 4 is a premium game made for premium gamers' is Randy Pitchford's tone deaf retort to the performance backlash: 'If you're trying to drive a monster truck with a leaf blower's motor, you're going to be disappointed' 2 weeks ago:
Ram is cheap
Kind of divering from the larger point, but that’s true — RAM prices haven’t gone up as much as other things have over the years. I do kind of wonder if there are things that game engines could do to take advantage of more memory.
I think that some of this is making games that will run on both consoles and PCs, where consoles have a pretty hard cap on how much memory they can have, so any work that gets put into improving high-memory stuff is something that console players won’t see.
checks Wikipedia
The XBox Series X has 16GB of unified memory.
The Playstation 5 Pro has 16GB of unified memory and 2GB of system memory.
You can get a desktop with 256GB of memory today, about 14 times that.
Would have to be something that doesn’t require a lot of extra dev time or testing. Can’t do more geometry, I think, because that’d need memory on the GPU.
considers
Maybe something where the game can dynamically render something expensive at high resolution, and then move it into video memory.
Like, Fallout 76 uses, IIRC, statically-rendered billboards of the 3D world for distant terrain features, like, stuff in neighboring and further off cells. You’re gonna have a fixed-size set of those loaded into VRAM at any one time. But you could cut the size of a given area that uses one set of billboards, and keep them preloaded in system memory.
Or…I don’t know if game systems can generate simpler-geometry level-of-detail (LOD) objects in the distance or if human modelers still have to do that by hand. But if they can do it procedurally, increasing the number of LOD levels should just increase storage space, and keeping more preloaded in RAM just require more RAM. You only have one level in VRAM at a time, so it doesn’t increase demand for VRAM. That’d provide for smoother transitions as distant objects come closer.
- Comment on Randy Pitchford Snaps Back at Borderlands 4 Criticism: 'Code Your Own Engine' 2 weeks ago:
Is he running with antialiasing on?
Looking at their system settings page:
borderlands.2k.com/…/amd-optimization/
Their settings for every single GPU listed there seems to have antialiasing off.
- Comment on Lemmy 0.19.4 HTTP server not binding to port - federation broken ? 2 weeks ago:
I found that while the lemmy_server process starts successfully and shows “Starting HTTP server at 0.0.0.0:8536” in logs, nothing is actually listening on port 8536.
Does:
# netstat -ntap|grep 8536…show anything bound to the port?
- Comment on is there a way to run openwrt and debian at the same time without virtualization? 2 weeks ago:
I don’t know why you want a BIOS update.
If you’re wanting VT-x support, looks like it’s present.
mobilespecs.net/…/Toshiba_SATELLITE_M100-221.html
Processor Model: T5600
www.techpowerup.com/…/core-2-duo-t5600.c378
VT-x
superuser.com/…/what-is-difference-between-vmx-an…
The CPU flag for VT-x capability is “vmx”; in Linux, this can be checked via /proc/cpuinfo
- Comment on Do you want Control Ultimate Edition for 90% off? -Nah, I'll just get HoMM 3 at full price. 2 weeks ago:
VCMI is an open-source recreation of Heroes of Might & Magic III engine, giving it new and extended possibilities.
- Comment on YSK Lemvotes.org will show you votes on any post, comment, or by user, or anything on the fediverse 2 weeks ago:
The only way is it be absolutely private by not interacting…
I don’t know if this is news to you but this is not a lemmy specific problem and basically applies to the entire internet…
Hyphanet’s Frost can provide pretty solid pseudonymity.
But that comes with its own performance and functionality tradeoffs.
- Comment on YSK Lemvotes.org will show you votes on any post, comment, or by user, or anything on the fediverse 2 weeks ago:
What could be done to limit the amount of information associated with a username is to awitch to a new account periodically.
That’s somewhat unfortunate in that it clashes with reputation, which is also important for making the Threadivwrse work.
- Comment on YSK Lemvotes.org will show you votes on any post, comment, or by user, or anything on the fediverse 2 weeks ago:
identity is reviled [I assume revealed]
The fact that most instances permit external image hosting permits obtaining user IP addresses by posting inline images hosted on a server created by an attacker, then harvesting IPs there. I noticed when going through the code that Lemmy, as of 0.19.4 has an option to protect users of a home instance by proxying images viewed there. However, it’s bandwidth-intensive and I don’t think that many instances have it on.
join-lemmy.org/…/2024-06-07_-_Lemmy_Release_v0.19…
Image Proxying
There is a new config option called image_mode which provides a way to proxy external image links through the local instance. This prevents deanonymization attacks where an attacker uploads an image to his own server, embeds it in a Lemmy post and watches the IPs which load the image.
Instead if image_mode is set to ProxyAllImages, image urls are rewritten to be proxied through /api/v3/image_proxy. This can also improve performance and avoid overloading other websites. The setting works by rewriting links in new posts, comments and other places when they are inserted in the database. This means the setting has no effect on posts created before the setting was activated. And after disabling the setting, existing images will continue to be proxied. It should also be considered experimental.
Many thanks to @asonix for adding this functionality to pict-rs v0.5.
I donlt know whether PieFed and Mbin presently have comparable functionality.
One major issue is that proxying the images will create more bandwidth usage on a home node, since they’re serving up all the images viewed by users of that home node — it’s more-expensive to run a node in that mode.
- Comment on BBC - The people who hunt old TVs 2 weeks ago:
In most respects, CRTs were technically worse, but a lot of video game art was designed around their characteristics, and thus can appear better on a CRT. We can — and do — try to emulate CRT quirks on LCDs/LEDs to varying degrees, but we’re always going to be just approaching one.
wackoid.com/…/10-pictures-that-show-why-crt-tvs-a…
Final Fantasy 6:
Castlevania: Symphony of the Night:
Final Fantasy 7:
- Comment on BBC under fresh pressure over extent of Reform UK coverage 2 weeks ago:
Maybe. But if you look at the local elections from May:
www.bbc.com/news/articles/c5yg467m8mjo
…Reform got 31% of the votes cast. The chart of polls is pretty close, and actually slightly underrates them for May 2025. Now, maybe voting in the general election and local elections have different groups of people show up. I know that here in the US, that’s a factor for midterm elections. That could affect outcomes in the general election. But…my guess is that the chart probably is at least in the neighborhood of being representative of their support in society.
- Comment on Big Tech: Convenience is a Trap 2 weeks ago:
Well, there’s certainly that. But even then, I’d think that a lot of videos could be made to be more concise. I was actually wondering whether YouTube creators get paid based on the amount of time they have people watch, since that’d explain drawing things out. My impression, from what I could dig up in a brief skim, is that they’re indirectly linked — apparently, YouTube shows ads periodically, and the more ads shown, the more revenue the creator gets. So there would be some level of incentive to stretch videos out.
- Comment on Frustratingly bad at self hosting. Can someone help me access LLMs on my rig from my phone 2 weeks ago:
-the opening the port process makes sense. It seems like if I have a backend on my rig, I’m going to need to open a port to access that backend from a front end of a phone device.
Yes. Or even if you run a Web-accessible front-end on the LLM PC — the Web browser on the phone needs to reach the Web frontend on the PC.
Or possibly even access that same backend on the phone device via a mirror?
Well, the term wouldn’t be a mirror. In your shoes, it’s not what I would do, because introducing some third host not on your network to the equation is another thing to break. But, okay, hypothetically, I guess that doing that would be an option. thinks. There might be some service out there that permits two devices to connect to each other, though I’m not personally aware of one. And, say you got a virtual private server for $10 a month or whatever the going rate is, yeah, that could be set up to do this – you could use it as an intermediate host, do SSH tunneling from both the PC and the phone of the sort that another user in this thread mentioned. I guess that that’d let you reach the PC from other places, if that’s something that you want to do, though it’s not the only way to accomplish that. But…I think that that’s most-likely going to add more complexity. The only scenario where that would truly be necessary is if the wireless access point — which I assume your ISP has provided — absolutely does not permit the LLM PC and the phone to communicate at all on the WiFi network, which I think is very unlikely, and even then, I’d probably just get a second wireless access point in that scenario, put the PC and the phone on it.
In general, I don’t think that trying to connect the two machines on your home network via a machine out on the Internet somewhere is a great idea. More moving parts, more things to break, and if you lose Internet connectivity, you lose the ability to have them talk to each other.
-it seems like it would be easier if I could connect to the rig via an android phone instead of an iPhone. My end goal is to use Linux but I’m not ready for that step. Seems like android would be an adequate stepping stone to move to, especially if we have to go thru all this trouble with iPhone. Shall we try on the android instead? If not I’ll follow the directions you put above and report back on Saturday.
If you have an Android phone available, that would probably be easier from my standpoint, because I can replicate the environment; I have an Android phone available here. But it’s not really the phone where setup is the issue. Like, it’s going to be the LLM PC and potentially wireless access point that require any configuration changes to make ollama reachable from the phone; the phone doesn’t need anything other than Reins installed and having an endpoint set or just using a Web browser and using the correct URL there. I’m just mostly-interested in that the phone has to be able to talk to the PC, has to be able to open a TCP connection to the PC, and so having diagnostic tools on a phone is helpful. I don’t have to guess how the diagnostic tools work in Termux on an Android, because I can use them myself locally.
I wouldn’t suggest going out and buying an Android phone to do just that, though. I mean…this is a one-off diagnostic task, just trying to understand why the phone isn’t able to reach the LLM PC. If you can open a connection from the Android phone to the LLM PC, then you should also be able to open a connection from the iOS phone to the LLM PC. If you do have one already available, though, then yeah, my preference would be if you could install Termux on it for the diagnostic tools rather than install iSH on the iOS device. It should still be possible to get the LLM PC reachable on the iOS device either way.
I don’t mind trying to diagnose connectivity on the iOS device. Just keep in mind that I may have to guess a bit as to what the behavior is, because I can’t actually try the device here, so we may potentially have a few extra rounds of back-and-forth.
If you do want to use an Android phone, then just put the phone on the WiFi network, install Termux, open Termux, run the apk command to install telnet (
apk install telnet) and then try the telnet command I mentioned and just report back what error, if anything, you get when trying to open a connection to the LLM PC — hopefully it’ll be one of the above three outcomes. - Comment on Left unable even to get into deputy leader contest proves Labour is dead 2 weeks ago:
Labour is dead
They’ve been around for 125 years and are curretly running the country. I think I’d give them a bit more credit for resillience.
- Comment on Being born in north-east England gives you grit, says Fiona Hill 2 weeks ago:
Hill was born and brought up in Bishop Auckland, the daughter of a coalminer and a midwife. She and her accent went on to be a foreign affairs adviser to US presidents George W Bush, Barack Obama and Donald Trump and she is considered one of the world’s leading experts on Russia and Putin.
I imagine that her accent was an exceptional foreign affairs advisor.
- Comment on Exclusive: Kratsios details White House AI plans 2 weeks ago:
Deregulation might give some amount of an edge, but I really don’t think that in 2025, the major limitation on deployment of AI systems is overbearing regulation. Rather, it’s lack of sufficient R&D work on the systems, and them needing further technical development.
I doubt that the government can do a whole lot to try to improve the rate of R&D. Maybe grants, but I think that industry already has plenty of capital available. Maybe work visas for people doing R&D work on AI.
- Comment on BBC under fresh pressure over extent of Reform UK coverage 2 weeks ago:
kagis
Ah. I hadn’t realized that Reform’s polling was nearly that high.
- Comment on Frustratingly bad at self hosting. Can someone help me access LLMs on my rig from my phone 2 weeks ago:
I choose Ollama because it was supposed to be one of the easier loca AIs to set up.
Well, the ollama bit is up, which is why you can use it on the PC. The problem is network connectivity between the Windows PC and the phone.
Opening a port between two things on the local network is going to be pretty much the same for anything. Some software packages — I dunno about LLM chat stuff — make use of a third, outside system as a point to coordinate, so that software only has to open outbound TCP connections to the Internet. But for local communication, it’s gonna look pretty similar. If you put koboldcpp or llama.cpp or whatever on your machine, you need the same connectivity, though it might default to using a different port number.
I’m happy to keep banging away if you’re also willing, though. I mean, this does kinda narrow it down. If you don’t want to do so though, remove that firewall rule that we added earlier from the Windows PC. If you do:
considers
The next step is seeing where the break in connectivity is.
I’m not familiar with iOS, but let me see if there’s a software package for it that will let it open a TCP connection and preferably ping (and ideally show the ARP cache to see whether Ethernet packets are getting from the phone to the Windows machine at all, though that may not be viable).
Basically, would be nice to see whether packets can currently get from the phone to the PC and back.
kagis
Looks like Windows Firewall blocks ICMP by default, which is traditionally used by ping, the simple protocol to see if one host can reach another on the network. Mmmmf.
And it sounds like ARP isn’t available on a non-jailbroken iPhone, which would be the simplest way to see whether a packet is making it from the iPhone to the PC. I was worried would be the case.
Hmm. This is a little less convenient in that I don’t have tools that I normally would when trying to troubleshoot network problems on a Linux system.
thinks
I guess the simplest thing available, cuts things down as far as possible in terms of connectivity between the two that should be able to reach from the iPhone to the PC should be a TCP connection.
I don’t know the iOS software library well, but lemme search for a telnet client. I’m sure that it’ll have one; every platform does.
searches
Oh, this is even better. It looks like there’s some iOS app, “iSH”, with a tiny Alpine Linux environment for iOS, kinda like Termux on Android. That’ll have telnet and probably other network diagnostic tools, and those I am familiar with, so I don’t have to guess from screenshots how things work. You should be able to try to open a TCP connection from the phone to the PC with the Linux telnet client in that.
goes looking around
Okay. If you’re willing to give this a shot, it sounds like the way this works is:
apps.apple.com/us/app/ish-shell/id1436902243
Install that from the iOS store.
When opened, it should show a Linux terminal. If it works like Termux, it’ll have basically nothing from Alpine Linux installed, no telnet client, just a few simple commands. You’ll be looking at a prompt that probably looks something like
iPhone:~#.Then if you run (don’t type the pound sign — it’s just a convention to include it, to show that it’s something to type at a prompt):
# apk install telnetThat should install the Linux telnet client inside the iSH app using the Alpine Linux package manager.
Then to try to open a TCP connection from the phone to the Windows PC, you want the private IP of the Windows PC, the thing you see in ipconfig (which I’ll type as 10.1.1.2 here, but replace with yours):
# telnet 10.1.1.2 11434That’ll try to open a TCP connection from the phone to port 11434 on the PC.
Now, what would happen if everything were working correctly, is that the phone would send an ARP request saying “what is the MAC address — the Ethernet address — of the machine with IP address 10.1.1.2 on the local network?” The wireless access point would hand this to the PC. The PC would respond. The phone would then send a series of packets to that IP address to open a TCP connection on port 11434.
My guess is that you’ll see one of several things at this point.
First, it might be that the wireless access point is refusing to let packets from the phone reach the PC at all — they only let the phone talk to the Internet, not to the PC. Some wireless access points can be configured to do this or have a “guest” wireless network that impose this constraint. Then the phone won’t get an ARP response, since the PC will never see the ARP query. That’ll look like this (using a network I’m on at the moment to demonstrate):
$ telnet 192.168.1.35 Trying 192.168.1.35... telnet: Unable to connect to remote host: No route to host $
Second, it might fail because I dicked up in some way and Windows Firewall is still blocking the phone from connecting to the PC. The ARP request is going out, the response comes back from the PC, the phone tries to open a TCP connection to the IP address on the host with the specified MAC address, and never gets a response. If that’s the case, it’ll probably look like this:
$ telnet 192.168.1.126 7500 Trying 192.168.1.126... telnet: Unable to connect to remote host: Connection timed out $
If that’s what you get, the problem is likely the Windows Firewall configuration (well or theoretically the wireless access point could be configured to do that, but I doubt it).
Third, it might succeed. That’ll look like this:
$ telnet 192.168.1.126 6000 Trying 192.168.1.126... Connected to 192.168.1.126. Escape character is '^]'.
If you see that, you can open a TCP connection from the phone to the PC, and whatever issue you’re hitting with Reins isn’t a network problem. Maybe I gave the endpoint syntax wrong, for example. But the issue will be at the application level, not the network level.
- Comment on Frustratingly bad at self hosting. Can someone help me access LLMs on my rig from my phone 2 weeks ago:
There are 3 lines with the :11434 in them. No brackets or anything like that. -1 has 0.0.0.0 in front -2 has 10.#.#.# in front and has a foreign address that is something other than 0.0.0 -3 is like the 2nd but a slightly different foreign address
Okay, that…should be okay. As long as all of the addresses that it’s listening on are IPv4 — of the format “x.x.x.x”. No colons in them (other than the colon preceeding “11434”). Not IPv6.
The subnet mask is 255.255.255.0
Okay, gotcha. In that case, go ahead with the instructions above, just instead of “/8”, do “/24”. So:
For “local IP addresses”, you want “These IP Addresses”, and enter
10.0.0.0/24. That’ll be every IPv4 address on your Windows LLM that has “10” as its first number and the first following two numbers the same as yours.For “remote IP addresses”, you want “These IP Addresses”, and enter
10.0.0.0/24. Same thing all addresses that start with a “10.” followed by the same following two numbers, which should include your iOS device.
Oh yes. I’m on windows 10 as well.
Okay. I think that the interface to add the firewall rule there looks the same as the one I Iinked to above. I went searching for screenshots of adding a hole for a port on Windows 10, and the control panel looks identical to me.
So, yeah, should be good to go ahead with the above instructions, just using “/24” instead of “/8” in the two places where I mention “/8”. Hopefully after that it’ll be working; if not, then we’ll need to troubleshoot.
- Comment on Let France be a warning, Rachel Reeves: stand up to the bond market vigilantes, or they’ll come for Britain next 2 weeks ago:
Britain, so the story goes, also needs to wake up, or else the markets will be coming for us next.
The reason France and Britain have no choice but to do this is because states are weak and markets are all powerful. The bond markets exert their power through their role in buying and selling government bonds. If they sell en masse, the interest rates governments pay to borrow goes up and they can be forced to change policy even when they are reluctant to do so. It has been the received wisdom for the past 50 years that governments should do what bond traders and speculators demand, or risk being crushed by the global financial juggernaut.
All right, I am just boggled by this.
The markets already came for you. Are we so soon forgetting that three years ago, a British prime minister had the shortest tenure in the history of the United Kingdom, lasted less time than a head of lettuce, because she decided that she was going to blow the deficit way up?
en.wikipedia.org/wiki/Liz_Truss_lettuce
Liz Truss became Prime Minister of the United Kingdom on 6 September 2022, following the July–September 2022 Conservative Party leadership election, replacing Boris Johnson. The September 2022 United Kingdom mini-budget was published on 23 September by Kwasi Kwarteng, then-Chancellor of the Exchequer, which included tax cuts without matching spending cuts. The mini-budget triggered a heavily negative market reaction, with the exchange rate of the pound sterling collapsing and pension funds coming close to bankruptcy.[3]
After just over a month in office, Kwarteng was removed as Chancellor of the Exchequer on 14 October, and Truss reversed most of the economic policies within the mini-budget. British media outlets lambasted Truss’s performance and the ensuing political chaos, with many observers believing that her resignation would be imminent.[4] An 11 October column in The Economist titled “Liz Truss has made Britain a riskier bet for bond investors” stated that, after deducting the ten-day mourning period following the death of Queen Elizabeth II, Truss had caused economic and political turmoil after just seven days in power, comparing that duration to the “shelf-life of a lettuce”.
Before the lettuce had wilted, on 20 October Truss announced her resignation as prime minister becoming, after only 45 days, the shortest-serving prime minister in British history.[1][11] At that moment, there were 12,000 viewers on the livestream, which soon shot up to 21,000. The British national anthem “God Save the King” began to play, the portrait of Truss on the table was flipped face down, and a plastic golden crown was placed on top of the lettuce, with the Daily Star declaring the lettuce’s “victory” over Truss.[1][12]
- Comment on Uh Oh: Nintendo Just Landed A ‘Summoning’ And ‘Battling’ Patent 2 weeks ago:
copyright
This isn’t a copyright, but rather a patent.
- Comment on Frustratingly bad at self hosting. Can someone help me access LLMs on my rig from my phone 3 weeks ago:
-A backend is where all the weird c++ language stuff happens to generate a response from an AI. -a front end is a pretty app or webpage that takes that response and make it more digestible to the user.
Yes.
-agreed. I’ve seen in other posts that exposing a port on windows defender firewall is the easiest (and safest?) way to go for specifically what I’m looking for. I don’t think I need to forward a port as that would be for more remote access.
Yes. I’d like to confirm that that is not happening, in fact.
The ipv6 was identical to one of the ones I have.
Hmm. Okay, thanks for mentioning the IPv6 thing. It is possible to have the ollama reachable from the Internet via IPv6, if it’s forwarded. I should have thought of that too and mentioned that. Shouldn’t need to open an IPv6 hole in the Windows Firewall, but would rather not rely on the Windows Firewall at all.
It shouldn’t be an issue if ollama is only listening on an IPv4 address. You only see the “0.0.0.0:11434” line, right? No other lines, probably with brackets in the address, that have a “:11434”, right? That could be an IPv6 address.
goes to look for an example of Windows netstat output showing a listening IPv6 socket
Here:
configserverfirewall.com/…/netstat-command-to-che…
Can you just make sure that there’s nothing like
0:[::]:11434in there? That’d be what you’d see if it were listening for IPv6 connections.Sorry, just don’t know oollama’s behavior off the top of my head and want to be sure on this before moving ahead, don’t want to create any security issues.
The ipv4 was not identical. (But I don’t think that matters moving forward.)
Yeah, that’s expected and good. The one from the website is your public IP address, anf the one from ipconfig your private one, that you’ll use to talk to the machine wirh your phone.
I had to go into the settings in the ollama backend app to enable “expose Ollama to the network”.
Great, yeah, that was the right move.
Okay, then just want to sanity check that your iOS device is in the same address range on your WiFi network, that the 10.x.x.x address on your LLM PC isn’t from a VPN or something (since it’s a little unusual to use a 10.x.x.x address on a home broadband router, and I want to make sure that that’s where the address is from). Go ahead and put the iOS device on your WiFi network if you have not already.
This describes how to check the IP address on an iOS device.
servicehub.ucdavis.edu/servicehub?id=ucd_kb_artic…
You should also be seeing a 10.x.x.x address there. If you don’t, then let’s stop and sort that out.
If that’s a 10.x.x.x address as well, then should be good to go.
Oh, one last thing. In the ipconfig output, can you make sure that the “Subnet Mask” reads “255.0.0.0”? If it’s something different, can you provide that? It’ll affect the “/8” thst I’m listing below.
Okay, if you’ve got that set up and there are no other “:11434” lines and the Subnet Mask is “255.0.0.0”, the next is to poke a hole in Windows Firewall on IPv4 TCP port 11434.
kagis for screenshots of someone doing this on Windows 11
windowsreport.com/windows-firewall-allow-ip-range…
I’m assuming that this is Windows 11 on your PC, should have asked.
You’re going to want a new inbound rule, Protocol TCP, Port 11434.
For “local IP addresses”, you want “These IP Addresses”, and enter
10.0.0.0/8. That’ll be every IPv4 address on your Windows LLM that has “10” as its first number — you said that you had a “10.” from ipconfig.For “remote IP addresses”, you want “These IP Addresses”, and enter
10.0.0.0/8. Same thing all addresses that start with a “10.”, which should include your iOS device.Okay. Now you should have a hole in Windows Firewall. Just to confirm that port 11434 isn’t reachable from the Internet, I’m gonna use one of the port-open-testing services online. My first hit is for one that only does IPv4 and another that only does IPv6, but I guess doing two sites is okay. Can you go to this site (or another, if you know of a site that does port testing that your prefer)
www.yougetsignal.com/tools/open-ports/
Plug in your public IPv4 address there (not the private one from ipconfig, the one from that website thst I listed earlier) and port 11434. It should say “closed” or “blocked” or something that isn’t “open”. If it’s “open”, go back and pull thst firewall rule out, because your router is forwarding incoming IPv4 connections to your LLM PC in some way that’s getting, and we gotta work out how to stop that.
Here’s an IPv6 port tester. Plug in your IPv6 address there (which you said was the same from both the website and ipconfig) and port 11434. It should also say “closed” or “blocked” or similar. If it says “open” — I very much doubt this — then go back and pull out the firewall rule.
If both say “closed”, then go ahead and install Reins.
Based on this:
www.reddit.com/r/ollama/comments/1ijdp1e/reins/
It’ll let you input an “endpoint”.
Plug in the private IPv4 address from your LLM PC, what was in ipconfig, like “10.something.something.something:11434” and you should, hopefully, be able to chat.
- Comment on Big Tech: Convenience is a Trap 3 weeks ago:
This is true of the overwhelming majority of YouTube videos I see submitted here. The information density is just abysmal compared to a page of text.
- Comment on Frustratingly bad at self hosting. Can someone help me access LLMs on my rig from my phone 3 weeks ago:
Ollama does have some features that make it easier to use for a first-time user, including:
-
Calculating automatically how many layers can fit in VRAM and loading that many layers and splitting between CPU and VRAM. kobold.cpp can’t do that automatically yet.
-
Automatically unloading the model from VRAM after a period of inactivity.
I had an easier time setting up ollama than other stuff, and OP does apparently already have it set up.
-
- Comment on Frustratingly bad at self hosting. Can someone help me access LLMs on my rig from my phone 3 weeks ago:
Backend/ front end. I see those a lot but I never got an explanation for it. In my case, the backend would be Ollama on my rig, and the front end would be me using it on my phone, whether that’s with and app or web ui. Is that correct?
For Web-based LLM setups, it’s not common to have two different software packages. One loads the LLM into video memory and executes queries on the hardware. That’s the backend. It doesn’t need to have a user interface at all. Ollama or llama.cpp (though I know that llama.cpp also has a minimal frontend) are examples of this.
Then there’s a frontend component. It runs a small Web server that displays a webpage that a Web browser can access, provides some helpful features, and can talk to various backends (e.g. ollama or llama.cpp or some of the cloud-based LLM services).
Normally the terms are used in the context of Web-based stuff; it’s common for Web services, even outside of LLM stuff, to have a “front end” and a “back end” and to have different people working on those different aspects. If Reins is a native iOS app, I guess it could be called a frontend.
But, okay, it sounds like probably the most-reasonable thing to do, if you like the idea of using Reins, is to run Ollama on the Windows machine, expose ollama’s port to the network, and then install Reins on iOS.
So, yeah, probably need to open a port on Windows Firewall (or Windows Defender…not sure what the correct terminology is these days, long out of date on Windows). It sounds like having said firewall active has been the default on Windows for some years. I’m pretty out-of-date on Windows, but I should be able to stumble through this.
While it’s very likely that you aren’t directly exposing your computer to the Internet — that is, nobody from the outside world can connect to an open port on your desktop — it is possible to configure consumer routers to do that. Might be called “putting a machine in the DMZ”, forwarding a port, or forwarding a range of ports. I don’t want to have you open a port on your home computer and have it inadvertently exposed to the Internet as a whole. I’d like to make sure that there’s no port forwarding to your Windows machine from the Internet.
Okay, first step. You probably have a public IP address. I don’t need or want to know that — that’d give some indication to your location. If you go somewhere like whatismyipaddress.com in a web browser from your computer, then it will show that – don’t post that here.
That IP address is most-likely handed by your ISP to your consumer broadband router.
There will then be a set of “private” IP addresses that your consumer broadband router hands out to all the devices on your WiFi network, like your Windows machine and your phone. These will very probably be
192.168.something.something, though they could also be172.something.something.somethingor10.something.something.something. It’s okay to mention those in comments here — they won’t expose any meaningful information about where you are or your setup. This may be old hat to you, or new, but I’m going to mention it in case you’re not familiar with it; I don’t know what your level of familiarity is.What you’re going to want is your “private” IP address from the Windows machine. On your Windows machine, if you hit Windows Key-R and then enter “cmd” into the resulting dialog, you should get a command-line prompt. If you type “ipconfig” there, it should have a line listing your private IPv4 address. Probably be something like that “192.168.something.something”. You’re going to want to grab that address. It may also be possible to use the name of your Windows machine to reach it from your phone, if you’ve named it — there’s a network protocol, mDNS, that may let you do that — but I don’t know whether it’s active out-of-box on Windows or not, and would rather confirm that the thing is working via IP before adding more twists to this.
Go ahead and fire up ollama, if you need to start it — I don’t know if, on Windows, it’s installed as a Windows service (once installed, always runs) or as a regular application that you need to launch, but it sounds like you’re already familiar with that bit, so I’ll let you handle that.
Back in the console window that you opened, go ahead and run
netstat -a -b -n.Will look kinda like this:
That should list all of the programs listening on any ports on the computer. If ollama is up and running on that Windows machine and doing so on the port that I believe it is, then you should have a line that looks like:
TCP 0.0.0.0:11434 0.0.0.0:0 LISTENINGIf it’s 0.0.0.0, then it means that it’s listening on all addresses, which means that any program that can reach it over the network can talk to it (as long as it can get past Windows Firewall). We’re good, then.
Might also be “127.0.0.1”. In that case, it’ll only be listening to connections originating from the local computer. If that’s the case, then it’ll have to be configured to use 0.0.0.0.
I’m gonna stop here until you’ve confirmed that much. If that all works, and you have ollama already listening on the “0.0.0.0” address, then next step is gonna be to check that the firewall is active on the Windows machine, punch a hole in it, and then confirm that ollama is not accessible from the Internet, as you don’t want people using your hardware to do LLM computation; I’ll try and step-by-step that.
- Comment on Frustratingly bad at self hosting. Can someone help me access LLMs on my rig from my phone 3 weeks ago:
Yes I have Ollama on my windows rig.
TBH, im not sure if librechat has a web ui.
Okay, gotcha. I don’t know if Ollama has a native Web UI itself; if so, I haven’t used it myself. I know that it can act as a backend for various front-end chat-based applications. I do know that kobold.cpp can operate both as an LLM backend and run a limited Web UI, so at least some backends do have Web UIs built in. You said that you’ve already used Ollama successfully. Was this via some Web-based UI that you would like to use on your phone, or just some other program (LibreChat?) running natively on the Windows machine?
- Comment on Frustratingly bad at self hosting. Can someone help me access LLMs on my rig from my phone 3 weeks ago:
Oh! Also, I’m using windows on my PC. And my phone is an iPhone.
Okay, that’s a starting place. So if this is Windows, and if you only care about access on the wireless network, then I suppose that it’s probably easiest to just expose the stuff directly to other machines on the wireless network, rather than tunneling through SSH.
You said that you have ollama running on the Windows PC. I’m not familiar with LibreChat, but it has a Web-based interface? Are you wanting to access that from a web browser on the phone?
- Comment on Frustratingly bad at self hosting. Can someone help me access LLMs on my rig from my phone 3 weeks ago:
ssh -L 0.0.0.0:3000:YOURPUBLICIP:3000
If you can SSH to the LLM machine, I’d probably recommend
ssh -L127.0.0.1:11434:127.0.0.1:11434 <remote hostname>. If for some reason you don’t have or inadvertently bring down a firewall on your portable device, you don’t want to be punching a tunnel from whatever can talk to your portable device to the LLM machine.(Using 11434 instead of 3000, as it looks like that’s ollama’s port.)
- Comment on emergency remote access 3 weeks ago:
-
If your problem is brief brownouts or similar — my experience is that some consumer broadband routers have cheap power supplies that leaves them in bad states when PCs will pull through — you could put them on a UPS.
-
If your problem is that your router is unstable, you could just replace your router. Like, if you need remote access and you have a flaky router, that seems like a prime choice.
-
You could have a power control device or something and have another machine on your network set up so that if it loses Internet connectivity for some sustained period of time, it power-cycles the router.
-
If this is for when you’re a long ways away, do you have a friend who you’d trust with a key and flipping a switch?
-
I expect that there are business-oriented routers that will have integrated watchdog features that will auto-reboot if they hang. I have not gone looking, though.
-
Possibly, if it’s compatible with your use case, and uptime is critical enough here, having a second, backup server elsewhere, possibly not self-hosted. I mean, your connectivity is always going to be bounded by the reliability of your residential Internet connection otherwise.
-
