The PATH is easy and the best way to go, you'll only need to buy Quickcards which are available from machines at the station... Where you're staying (the Hyatt at Exchange Place stop) you'll want to take the WTC line. It's quick, easy and one stop away! Fare costs $1.75 per trip but you'll need to 'fill up' with more trips of course. You save a little by buying in "bulk" for a 10 trip where you get one ride free, but if you're staying awhile, i recommend just buying a weekly unlimited for each rider. Info about the Quickcard is here: http://www.panynj.gov/path/fares.html
Note the PATH Quickcard isn't valid for use in the NYC subway (unfortunately they haven't synced up the two systems yet) but you can use the NYC Metrocard (except for unlimited cards) for the PATH system. Quirky using two diff cards but no worries.
While in the city do enjoy places we locals love: have excellent pizza at Arturo's on Houston st. (the 'fiesta' is amazing w/bottle of house red wine), check out unique shops on Greenwich ave in the West Village, try a 'spicy' hot chocolate from the Chocolate Bar (8th ave @Jane st), grab a delicious lunch from the Dirty Bird (14th st near 7th ave) and a must see is the Chelsea Market (truly amazing place!) and then the nearby High Line. Tourists flock to the typical places but only end up seeing other tourists (giggle). Have fun, enjoy walking & go exploring a bit!
Best way to get to Jersey City from NYC- take the PATH (subway)