Since you'll be @ Port Authority, you can transfer to Amtrk or Greyhound for continued ground transportation. Staying over in NYC is not necessary. Catch the Greyhound to Albany NY or Amtrak to Renselear NY (5 minutes from Albany) & go from there if need be.... But you dont want to stay the night in Albany! So if a stay is inevitable your best bet is NY for nightlight but Albany for boredom & money!
Since you'll be @ Port