Helium is the only element produced in large quantities in the early universe. About 25% of the "ordinary" matter in the universe is in the form of helium, and it was almost all produced when the universe was only a few minutes old and very, very hot (about a billion degrees).
So does this mean that when you buy a helium balloon at the grocery story, you're holding a bit of the Big Bang right in your hands? Unfortunately, no. The helium for our balloons doesn't come from the early universe at all -- it comes from Texas.
Any "primordial" helium escaped from the Earth's atmosphere long ago -- you can see this by letting go of that helium balloon you just bought. Instead, we mine helium from underground deposits. So maybe that helium is a relic of the Big Bang? Sadly, it's not. The helium we extract is produced from underground radioactive decay -- heavy unstable elements like uranium spit out a helium nucleus when they decay, and under the right conditions this helium gets trapped underground, just waiting for us to suck it out. So if you want helium from the Big Bang, you have to look elsewhere, such as inside the Sun. Most of the Sun's helium really is left over from the birth of the universe.