A gem is a packaged Ruby library. Bundler is the tool that decides which versions of them a project uses and makes every machine agree. Almost all of the early difficulty people have with Ruby is one of those two sentences failing in some specific way, and almost none of it is about the language.
What Bundler is actually solving
Two gems each want a third one, at different versions, and only one version can be loaded. That is the whole problem, and it is harder than it sounds because the constraint graph includes dependencies nobody chose directly.
Bundler resolves that graph once and writes the answer to a lock file. Every later install reads the answer rather than solving it again, which is what makes two machines agree. Without it, two developers installing on different days get different code and one of them has a bug the other cannot reproduce.
The rule that follows from this: the lock file belongs in version control, in every kind of project including libraries where opinions have historically differed. A lock file that is not committed is a resolution that will be redone on a machine nobody is watching.
The commands that matter
install reads the lock file and installs exactly what it says, resolving only what is missing. It is the normal command and it should be the one used almost always.
update re-resolves. With no argument it re-resolves everything, which is rarely what anybody wants; with a gem named, it moves that gem and whatever it drags with it, which usually is.
exec runs a command with exactly the project’s gems loaded rather than whatever happens to be installed on the machine. A surprising share of “works here, not there” problems is this one command missing.
update with no arguments is almost never the command that was meant.
Where the time actually goes
Native extensions. A gem that compiles C against a system library will fail on a machine that does not have the library’s headers, and the error message describes the compiler’s problem rather than the missing package. This is the single most common first-week obstacle, it has nothing to do with Ruby, and the fix is nearly always installing a development package.
Version managers are the second. Having two installed at once, or a system Ruby and a managed one competing, produces gems installed somewhere the running interpreter is not looking. The symptom is a gem that is definitely installed and definitely not found.
The diagnostic that resolves most of it is asking which interpreter is actually running and where it looks for gems. Whenever the answer is not the expected one, the problem is environment rather than dependency, and no amount of editing a Gemfile will move it.
Choosing dependencies
Every gem added is a permanent decision, because removing one later is much harder than adding it was. The questions worth asking are the boring ones: when was it last released, how many other things depend on it, and how much would it cost to do without.
A gem that has not been released in several years is not automatically abandoned, and for a small, finished library it may simply be complete. It becomes a problem when it has native code or ties itself to framework internals, because those are the things a new Ruby or a new Rails will break.
The upgrade side of this is on the release timeline: a gem that fails to build against a newer Ruby is the usual thing standing between a project and its upgrade.
Groups, and what not to ship
Development and test dependencies belong in their own groups so that a production install does not carry a debugger, a test framework and a linter. It is a small saving in size and a real one in surface area.
The related habit is not requiring what is not needed at boot. A Gemfile entry that is only used by one rake task does not need loading on every request, and marking it as such is a one-line change that a great many projects never make.
For what those test dependencies are usually for, see testing; for the linters among them, static analysis; and for how all of this looks to somebody in their first week, learning Ruby. The rest of the section is at Ruby and Rails.
Questions
What is the difference between a Gemfile and a Gemfile.lock?
The Gemfile is written by people and states intent loosely, usually a minimum version. The lock file is written by Bundler and records exactly what that resolved to, including every indirect dependency. The lock file is what decides which code actually runs.
Should the lock file be committed?
Yes, in every kind of project. An uncommitted lock file means the dependency graph gets re-resolved on some machine nobody is watching, which is exactly the situation Bundler exists to prevent.
When should bundle update be used?
With a gem named, when that gem needs to move. With no argument it re-resolves everything at once, which is rarely what was intended and produces a change too large to review or to bisect afterwards.
Why does a gem fail to install with a compiler error?
Because it contains native code that compiles against a system library whose headers are not installed. The error describes the compiler’s problem rather than the missing package, which is why it reads as a Ruby problem when it is not one.
What if a gem is installed but not found?
Almost always two Ruby installations competing, or a version manager that is not active in that shell. Ask which interpreter is running and where it looks for gems; when the answer is unexpected, no amount of editing the Gemfile will help.