Gameboy Trackers

(Software)
LSDJ see http://www.littlesounddj.com
Nanoloop see http://www.nanoloop.de/

I personally recommend LSDJ if you are using an original gameboy or gameboy color.
If you are using GameBoy Advance the newer version of Nanoloop can do some fantastic things, but i personally feel like LSDJ has a better tracker, and generally speaking is better for writing complex music.
Nanoloop is more of an option for people who like making experimental/loop based music.

There are other trackers out there but I feel that these are the only 2 worth bothering with. The nanoloop cartriges can be purchased through their website, for LSDJ carts see nonelectronics or kitch-bent links below.

(Hardware)
There are several hardware modifications out there that people use to make their old GameBoys a bit more usuful for making music. Audio jack mods and backlighting the screens are 2 of the most typical.
There are various do it yourself tutorials out there on the internet for these, as well as people that will do them for you at a fee.

If you do not already own a gameboy you may want to consider purchasing one that has already been decked out for live performance. I mean, a regular old gameboy will still work, but there's some cool shit out there.

Here are my favorite sites for buying backlighting kits, pro sound kits, and LSDJ/Nanoloop cartriges.
They also have many do it yourself tutorials for anyone looking to try these mods theirself.
I have ordered parts and carts from all of these sites and have found them to be very reliable.

http://nonelectronics.com/ (Has just about everything gameboy related, good backlighting tutorials as well)
http://www.greightbit.com/ (Does Gameboy mods as well as custom circuit bent musical instruments)
http://store.kitsch-bent.com/ (Lots of parts for you DIY types, also has the cheapest price on LSDJ carts as far as I know)

If you are not skilled with a soldering iron or small tools then I highly recommend letting one of these sites do your mods.
They require a bit of skill, and if you mess up you may risk ruining your gameboy.

As for purchasing plain gameboys unless you have second hand or game stores that still carry old stuff, just use kijiji or ebay to find them. GameBoys can still be found relitavely cheap if you hunt a bit.