Submitted 1 day ago by PumpkinDrama@reddthat.com to nostupidquestions@lemmy.world
I’m interested in developing the skill to estimate probabilities for real-world situations. What are the best ways to learn this systematically? Are there books, courses, or exercises that teach probabilistic thinking, Bayesian reasoning, or practical forecasting skills?
What you’re interested in is actuarial science. It’s an interesting and complex field. Pretty much every insurance company is built on the backs of a few talented actuaries.
Go to school and get a degree in actuarial sciences
It kind of depends on if there is anything specific you want to try and forecast. That said, a lot to data scientists would probably point to Bayesian statistics
50/50
It either happens, or it doesn’t. So easy, next question pls.
Two outcomes does not mean 50/50 chances. Maybe you were being sarcastic about that, but people actually use that reasoning in the real world all the time.
It either happens, or it doesn’t, or the chances are not equal.
That’s 50/50/50 or 1 in 3.
This is not an ability the Jedi of Lemmy will ever teach you.
Yes, maths and statistics courses in school, college, university would be the tried and tested route for learning the analytical and practical tools and techniques.
Forecasting rare events with any precision is almost a contradiction in terms though.
When you're down to the 1/10000 type events you need such a large dataset/sample, that there are almost always unobservable sub-populations, or unobservable historical / environmental factors that your data is likely to be missing; something important that could materially change the forecast if you were to have had complete, unbiassed data on it for you whole sample.
Practical forecasting though , i think, should be tied into the decision making, and trying to reduce the risk of choosing the course of action to take. The set of possible / feasible actions shapes the forecasting approach - you can't really learn that pragmatic tradeoff in academic institutions - i think it's just experience. Make some predictions, get them wrong, do a forensic analysis. Or collaborate with people who have done this for a living.
In respect of the AI, you need to check it had a reasonable concept of the population of events you want to know about.
Understands its sample of observations, how that sample was drawn (i.e. it wasn't random), and the biases in that sample or sampling method. Then it should be easy to recalculate its output, then you come up with some scenarios of the bias, or adjustments see what changes those scenarios have. A competent forecaster should have addressed major/obvious sources of bias, with ranges / scenarios etc. " how wrong might this forecast be if we assumed, X, Y, Z instead?" I don't trust anyone who asserts they have a representative sample, it's impossible to prove that 100% - otherwise you'd not need a sample in the first place.
Ignoring the AI part, since it doesn’t even know it’s gaslighting you.
Maybe read some Buckminster Fuller. He opined to some length about trends in real-world changes.
Isaac Asimov as well, just for a general sense of the approach.
But overall probabilities are kinda arbitrary when applied to specific events. They work fine for a whole lot of similar events (e.g. pulling colored marbles out of a bag) but they don’t really have any tangible meaning for unique events. Either you guess wrong or you guess right.
If you want to predict future events, you need to have a good grasp on current events, past events, and systemic behavior in general. There isn’t one methodology that yields results generally. You need to tailor your approach to suit each prediction.
That’s not something you can learn from one book, course, or series of exercises. It relies on broad scholarship.
Predictions are hard, especially about the future.
You don’t say exactly what you want to predict so it’s hard to say exactly what you need to learn.
If you want to predict the next economic crash you’ll have to study a lot of economics. If you want to predict the next pandemic you’ll have to study medicine etc. Years of study no matter what.
Once you know a lot about a topic you have to put a ton of effort into study current events and still be wrong a lot of the time.
I’ve heard podcasts about super forecasters, but haven’t come across a systemic way to learn about this. But knowing a name used for this might help in your search
Seeing as 9-11 only happened once in the past few thousand years, the AI odds check out.
I can recommend the book by: Douglas W. Hubbard
How to Measure Anything: Finding the Value of Intangibles in Business
For one off events like 9/11, you can’t. There isn’t an accepted definition of “probability” for things like that. Either The question is completely nonsensical (Frequentist view) or the numbers are more or less all arbitrary (Bayesian view).
Generally yes there is TONS of theories and methods relating to forecasting and prediction. Its a very interesting field.
What was the link to? Cant seem to work for me.
Oh and just fyi AI models as most know them only run through outout as highest likely to be following a given chain, all based on trained data. The closest you can get to real “math” would be training a model to context strip metadata from an input, to very specific output, use that to input data into a database, have an actual program do the math or call relevant statistics, peform the calc then give the output. Buuuuut that is pretty much not done at all by anyone beyond custom written systems for specific use cases, and any forward facing companies having AI models arent much doing that, the output is a fluke.
Prediction markets are a thing…
TheAlbatross@lemmy.blahaj.zone 1 day ago
When in doubt, the AI is gaslighting you.
When you’re sure, the AI is still probably gaslighting you.