Comment on Frustratingly bad at self hosting. Can someone help me access LLMs on my rig from my phone

• tal@lemmy.today ⁨2⁩ ⁨weeks⁩ ago

  -the opening the port process makes sense. It seems like if I have a backend on my rig, I’m going to need to open a port to access that backend from a front end of a phone device.

  Yes. Or even if you run a Web-accessible front-end on the LLM PC — the Web browser on the phone needs to reach the Web frontend on the PC.

  Or possibly even access that same backend on the phone device via a mirror?

  Well, the term wouldn’t be a mirror. In your shoes, it’s not what I would do, because introducing some third host not on your network to the equation is another thing to break. But, okay, hypothetically, I guess that doing that would be an option. thinks. There might be some service out there that permits two devices to connect to each other, though I’m not personally aware of one. And, say you got a virtual private server for $10 a month or whatever the going rate is, yeah, that could be set up to do this – you could use it as an intermediate host, do SSH tunneling from both the PC and the phone of the sort that another user in this thread mentioned. I guess that that’d let you reach the PC from other places, if that’s something that you want to do, though it’s not the only way to accomplish that. But…I think that that’s most-likely going to add more complexity. The only scenario where that would truly be necessary is if the wireless access point — which I assume your ISP has provided — absolutely does not permit the LLM PC and the phone to communicate at all on the WiFi network, which I think is very unlikely, and even then, I’d probably just get a second wireless access point in that scenario, put the PC and the phone on it.

  In general, I don’t think that trying to connect the two machines on your home network via a machine out on the Internet somewhere is a great idea. More moving parts, more things to break, and if you lose Internet connectivity, you lose the ability to have them talk to each other.

  -it seems like it would be easier if I could connect to the rig via an android phone instead of an iPhone. My end goal is to use Linux but I’m not ready for that step. Seems like android would be an adequate stepping stone to move to, especially if we have to go thru all this trouble with iPhone. Shall we try on the android instead? If not I’ll follow the directions you put above and report back on Saturday.

  If you have an Android phone available, that would probably be easier from my standpoint, because I can replicate the environment; I have an Android phone available here. But it’s not really the phone where setup is the issue. Like, it’s going to be the LLM PC and potentially wireless access point that require any configuration changes to make ollama reachable from the phone; the phone doesn’t need anything other than Reins installed and having an endpoint set or just using a Web browser and using the correct URL there. I’m just mostly-interested in that the phone has to be able to talk to the PC, has to be able to open a TCP connection to the PC, and so having diagnostic tools on a phone is helpful. I don’t have to guess how the diagnostic tools work in Termux on an Android, because I can use them myself locally.

  I wouldn’t suggest going out and buying an Android phone to do just that, though. I mean…this is a one-off diagnostic task, just trying to understand why the phone isn’t able to reach the LLM PC. If you can open a connection from the Android phone to the LLM PC, then you should also be able to open a connection from the iOS phone to the LLM PC. If you do have one already available, though, then yeah, my preference would be if you could install Termux on it for the diagnostic tools rather than install iSH on the iOS device. It should still be possible to get the LLM PC reachable on the iOS device either way.

  I don’t mind trying to diagnose connectivity on the iOS device. Just keep in mind that I may have to guess a bit as to what the behavior is, because I can’t actually try the device here, so we may potentially have a few extra rounds of back-and-forth.

  If you do want to use an Android phone, then just put the phone on the WiFi network, install Termux, open Termux, run the apk command to install telnet (apk install telnet) and then try the telnet command I mentioned and just report back what error, if anything, you get when trying to open a connection to the LLM PC — hopefully it’ll be one of the above three outcomes.

  source

  • BlackSnack@lemmy.zip ⁨1⁩ ⁨week⁩ ago

    Tried it on the pixel and no luck…

    Apparently it’s not apk install telnet, I need to type pkg instead. I did that and got the response below (not exact obviously) -get: 1 http.termux.net -get: 2 (similar to above) -fetched 246kb -58 packages can be upgraded. Run ‘apt list —upgradable to see them. -ERROR unable to locate package telnet.

    So I ran the command line ‘apt list upgrade’ (is that what it’s called, a command line?) and I got a bunch of text back. Most of it is something like below… -dpkg/stable #.##.# arch64 [upgradable from #.##.#]

    source