Baking Powder vs Baking Soda vs Yeast

Baked goods often include an ingredient that helps the final product rise (leaven).
To get your cake, bread, muffin (or whatever) to rise, you need to add something to the batter to produce a gas that gets caught in the dough, causing expansion. Carbon dioxide is a good gas for this, and is easy to produce with food-safe ingredients. You'll find recipes calling for baking powder, baking soda, or yeast, or even some combination of those to aid rising.

Which ingredients you use and their combination will affect the taste and texture. But what is the real difference?
 
Yeast is a little organism called a fungus, that when activated, consumes the sugars in flour and releases carbon dioxide as waste. When making a traditional (“slow”) bread, you combine it with flour, sugar, some liquid and other ingredients. When you knead the dough, the proteins inside form a stretchy matrix called gluten. This matrix traps the little gas bubbles produced by the yeast. Without a leavening agent like yeast, you'll end up with a dense blob that works better as a building material instead of bread*

*Side note: Don't build anything important with this.

The yeast produces gas when you let the dough rest for a while after kneading, and then expands again once heated in the oven. Once your ball of gas-filled gluten gets hot enough, it sets into the spongy, fluffy structure we call bread.
You'll notice that baking bread takes a while due to all of this waiting for yeast to work. We have some faster alternatives that instead rely on a chemical reaction between an acid and base to produce carbon dioxide.

Baking soda is sodium bicarbonate. It will react with a liquid, acidic ingredient to produce carbon dioxide. You can see this for yourself by adding a bit of vinegar to a little baking soda in a small bowl. It will start to fizz immediately after adding the vinegar. Because the reaction is so fast, foods made with baking soda must be cooked immediately after mixing. For instance, pancakes made with baking soda will come out tall and fluffy if you can get it onto the griddle right away. However, if you let the batter sit for a while, say 30 minutes, they'll come out dense with a gummy center (yuck) since the gas was lost while sitting.

Baking soda also adds flavor and color to pancakes, muffins, and cookies by hastening browning.

Baking powder is essentially baking soda mixed with a starch and powdered acid. Activating the reaction to generate carbon dioxide requires adding a liquid, like water. Most baking powders are “double acting”, which means they produce gas when moisture is added, and again when heated. This means that goods leavened with baking powder tend to be lighter and fluffier compared to foods leavened with only baking soda. Substituting baking powder with baking soda is possible, but the final product won't have the same flavor since it won't have the extra acidic ingredient that baking powder brings.

For a homemade baking powder, you can substitute a teaspoon of store-bought baking powder for a ¼ teaspoon of cornstarch, ¼ teaspoon of cream of tartar, and ¼ teaspoon of baking soda. This will not be double acting, so you'll need to be very quick about getting your batter or dough onto the griddle or the oven.