I think if you're just starting in music Ableton is a very good starting point.
http://www.ableton.com/
you can download a demo off their site.
http://www.ableton.com/free-trial
the software has drum machines, synths and sampler applications.
I bought the full version ($500) and I am VERY happy with it, it makes music creation and realization easy and fun.
after you check Ableton out for a while you can expand on that; buying Trigger Finger ($150) to play drum parts, getting a MIDI keyboard to play melodies and basslines, then maybe getting into hardware (drum machines and synths)
There's a lot of info on the internet to expand upon your knowledge
If you don't want to start off with the software/computer route, I suggest getting a sampler maybe the Korg ES1 or Roland SP-404 for example. The reason I suggest a sampler is so you'll always have new sounds and unlike a single drum machine can do basslines and melodies.
m71@markseventyone.com if you need more info.