Sample space, samples, events, random experiments and probabilities.
In this chapter we will give some basic definitions.
Outcomes, sample space and Random experiments
The setting of probability theory is as follows:
A random experiment is an action for which all possible outcomes can be listed, but which outcome will occur cannot be predicted with certainty. Outcomes must be mutually exclusive (ie, if one outcome happens, the other cannot happen). The list of outcomes must be collectively exhaustive (ie, all possible outcomes must be listed).
A random experiment may be performed many times, each time yielding a different outcome.
The set of all outcomes is known as the sample space.
Example: Our system is an unbiased coin. The random experiment is flipping the coin. The possible outcomes are H and T. The sample space is {H,T}.
Example: Consider a doctor working on a hospital. Every time a patient enters into his room is a random experiment. The disease of the patient is an outcome. And the set of all diseases is the sample space.
Example: Consider we are taking oranges randomly from a crate and weighing them..Each such weighing is a random experiment. The outcome of a weighing is an outcome. And all possibe real numbers that can be the result of one such weighing is a sample space.
Example: Let us throw two coins instead of one, as in the first example. Now two flips of coin is a single random experiment. Outcomes are TT, HT, TH, HH. Sample space is {TT, HT, TH, HH}.
Events
A subset of sample space is an event. Consider these examples:
Example: Consider the previous example, ie, flipping two coins together. Let us define two events:
event 1: we obtain at least one T. This event denotes the subset {TH, HT, TT} of the sample space.
event 2: We obtain an even number of T: This event denotes the subset {HH, TT} of the sample space. We have considered zero as an even number.
Problem: Consider rolling two dice, and multiplying the result as a random experiment. What is the sample space? List the following events:
Event 1: Result of the experiment is odd
Event 2: Result of the experiment is even
Event 3: Result of the experiment is between 10 and 100.
Note that the event, as an event, cannot be the result of a random experiment. The result of a random experiment is always an outcome. But if this outcome is an element of the event, then the event is also considered to happen.
Example: We flipped two coins. If the outcome is TH, event 1 is happened and event 2 is not. If the outcome is TT, both event 1 and event 2 are happened.
Concept of probability
One can think that probabilities can be defined on the basis of outcomes. Consider the following scenario: We have an experiment whose sample space is {A, B, C, ….}. Assume we have performed this experiment N times and the frequencies we get are ${n_A, n_B, n_C, …}$. If we bring $N$ to infinity, we can assign a probability to each outcome as
\begin{eqnarray}
P(A) &=& \lim_{N \rightarrow \infty} \frac{n_A}{N}\\
P(B) &=& \lim_{N \rightarrow \infty} \frac{n_B}{N}\\
P(C) &=& \lim_{N \rightarrow \infty} \frac{n_C}{N}\\
\ldots & \ldots & \ldots
\end{eqnarray}
Unfortunately, such a scheme will only work if the sample space is discrete. For a continuous sample space (like the interval [0,1]) it will not work, as each outcome must have the probability zero. We need an another way to define probabilities.
In the modern treatment of probability theory, probabilities are assigned to events, not to outcomes. Recall that each event is a set containing many outcomes. In a continuous space outcomes must have probability zero, but sets may have nonzero probability.
Algebra of Events
Events are basically sets of outcomes. Hence they are subject to all the rules that concern the sets.
mutual independence, pairwise independence, sum of independent random variables, variance and stdderivaton, uncorelatedness..