Comment on Cops Are Buying ‘GeoSpy’, an AI That Geolocates Photos in Seconds
FauxLiving@lemmy.world 3 weeks agoIt’s doing what the Geo Guesser-like people do.
Like ‘this kind of rock formation only appears in Eastern Europe, the wheel you see in the lower left of the screen has Cyrillic writing and if you look in eastern Europe there is one mountain formation that looks like the picture when viewed from a specific angle and so they had to be within this 50m circle’.
Eh, kind of both.
When researchers peeked into which areas of the image were being used, it showed that the tiny camera watermark from the Google Streetview car was being used by the model a lot.
That is, the recognition system had learned all the routes every Google Street view car had taken, and was using that in its recognition process.
Not all images have this watermark though, so in the cases the watermark didn’t exist it then resorts to more traditional geoguessr tactics.
This system wouldn’t a simple ‘put image into a multimodal LLM and get an answer’ like using ChatGPT.
It’d do things like image segmentation and classification, so all of the parts of the image are labeled and then specialized networks would take the output and do further processing. For example, if the segmentation process discovered a plant and a rock then those images would be sent to networks trained on plant or rock identification and their output would be inserted in to the image’s metadata.
Once they’ve identified all of the elements of the photos there are other tools that don’t rely on AI which can do things like take 3D maps of an suspected area and take virtual pictures from every angle until the image of the horizon matches the image in the pictures.
If you watch videos from Ukraine, you’ll see that the horizon line is always obscured or even blurred out because it’s possible to make really accurate predictions if you can both see a horizon in an image and have up to date 3D scans of an area.
The research paper that you’re talking about was focused on trying to learn how AI generate output from any given input. We understand the process that results in a trained model but we don’t really know how their internal representational space operates.
In that research they discovered, as you have said, that the model learned to identify real places due to watermarks (or artifacts of watermark removal) and not through any information in the actual image. That’s certainly a problem with training AIs, but there are validation steps (based on that research and research like it) which mitigate these problems.
It’s not hard. I once saw a random “what is this thing” photo from a bad angle. But it included a store in the background. Only two stores in North America with that name, though Google map search tried to be helpful and return a bunch of other results. Easy enough to check both.
Even with the extra street view angles I couldn’t figure out what the thing was though :(
It’s one of those tasks where it has a bunch of little components, each of which is easy to do (like identifying a store, or mineral formation, or road signs, etc) and so it is a thing that you can design machine learning tools around the individual tasks (‘what is this rock?’) and then instead of needing a highly trained human being to take a few minutes/hours to go through all of the details from memory, you can just push thousands of pictures through an AI system and get ‘good enough’ results.
It seems like there is a company selling such a ‘good enough’ service.
Bingo. You use ML to narrow down results, not to give you answers. I have a friend who uses ML models to analyze radio telescope data, because it’s really good at the mind-numbing work of throwing out noise and junk from broadcast satellites and known radio sources. Then you go through the narrowed stuff to see if anything in that is more interesting.
It’s the question between sifting a million hits or a thousand.
it’s really good at the mind-numbing work of throwing out noise and junk from broadcast satellites and known radio sources.
That’s the key when you’re looking at applications for machine learning. If you can find a task that’s simple but hard to scale because it requires a human expert then it is very likely that a trained neural network can do ‘good enough’ work at 1,000x the speed.
The results won’t be perfect but, then again, they wouldn’t be perfect even if you assigned the project to undergraduates with two decades of training. You still need an expert human supervisor who’s validating the results and tweaking the system.
In these limited cases, machine learning tools are pretty amazing and they give us capabilities that simply were not available to the average person 5 years ago. I’m not on the AI hype train in terms of the current capitalist casino bubble (chatbots and image generators are toys, not an industry), but from an academic point of view these tools are astonishingly powerful in the right context.
Download some pictures and ask ChatGPT thinking to find the locations.
It’s already trained on geoguesser data, even if that wasn’t a core feature.
I’ve seen those speed run videos where they basically go, ah yes this pebble is upper Midwestern Peruvian, and then get the location within 200 m.
unexposedhazard@discuss.tchncs.de 3 weeks ago
This is the kind of thing that machine learning is very very good at. Its never going to be perfect but its definitely gonna outperform humans.
FauxLiving@lemmy.world 2 weeks ago
In addition, any organization that’s using this at scale will also have human experts to handle the edge cases and to validate the system’s findings.
We can’t copy the human expert without years of training, but copying a program/computer system is only a few terminal commands. The ability to do this kind of thing at scale is entirely new.