Framework - Haluk Bingöl Content and Animations - Ege Onur
Markov Chains (Submitted 10.02.2021 as part of the SWE596 Course)
Markov Chains are stochastic models that transition from one
state to another based on rules of probability.
One of the most important properties of Markov Chains is that they are memory-less.
This means that the current state of the system only depends on the previous state of the system.
In other words, the next state of the system is solely dependent on the current state.
We don't have to look into the history of the system states in order to determine where
the system will be in the next step.
For example: Consider that the system is currently at "State A".
For the next step, it can either be in
State A (With p=0.25) or,
State B (With p=0.75)
Another famous example is the prediction of weather events.
Let's say that if a day is sunny, then the probability that the next day is going to be
sunny as well is p=0.6.
Another famous example is the prediction of weather events.
Let's say that if a day is sunny, then the probability that the next day is going to be
sunny as well is p=0.6.
Also, when a day is sunny, there's a 20% chance that the next day will be cloudy.
Another famous example is the prediction of weather events.
Let's say that if a day is sunny, then the probability that the next day is going to be
sunny as well is p=0.6.
Also, when a day is sunny, there's a 20% chance that the next day will be cloudy.
Let's add a third state called "Rainy" and define the probability of every state:
Today: Sunny, Tomorrow: Rainy, Probability = 20%
Today: Cloudy, Tomorrow: Cloudy, Probability = 50%
Today: Cloudy, Tomorrow: Rainy, Probability = 30%
Today: Cloudy, Tomorrow: Sunny, Probability = 20%
Today: Rainy, Tomorrow: Rainy, Probability = 55%
Today: Rainy, Tomorrow: Cloudy, Probability = 20%
Today: Rainy, Tomorrow: Sunny, Probability = 35%
Let's see it in action
The starting node is "Sunny"
The animation will run, selecting nodes with specified probabilities, as long as you're in this page.
Running this simulation can give us further insight
We can deduce what the weather will be at any given day.
Sunny (Count: 0 | Percentage: 0%)
Cloudy (Count: 1 | Percentage: 100%)
Rainy (Count: 0 | Percentage: 0%)
Total transitions (Count: 1)
As you can guess, after enough simulations, the starting node becomes less important. You'll notice that
the percentages converge after some time (use mental speed for faster results).