Both RubyGems and Rake are now part of the Ruby core. Installing Ruby will give you RubyGems and Rake out of the box. This eliminates an extra step in installing Ruby that often requires you to run third-party scripts and break any package management your system has. Future Ruby 1.9 packages will run more things out of the box and be ready to go.
Installing RubyGems was often the first thing you did after installing Ruby. You had to go download the zip file, extract it, run setup.rb (as root on Linux) and hope it worked. Now, RubyGems is installed as part of the Ruby program itself. This should also clear up some confusion when using different versions of Ruby with different versions of RubyGems.
In addition, you no longer have to require the rubygems module in order to require a gem. This clears up some confusion as well. In the past, requiring rubygems requried on all systems. Now, it's not required on any systems.
Finally, Rake is considered such an essential tool that it's installed by default. This is often the very first gem installed, either explicitly or as a dependency of another gem being installed.
One final note, both the gem and rake commands honor any program suffix used when building Ruby. If you've installed Ruby from source on your system and provided the --program-suffix=19 option to the configure script, not only will your ruby executable be ruby19, but your gem executable will be gem19 and rake will be rake19.