You will be tight for time but can certainly get to Manhattan and back. The quickest way is to take the Air Train from terminal 4 to Jamica station and then connect to the E subway line from which you can exit in midtown Manhattan. Buy a metro card which is good for both and you can do the whole round trip for around $15 US. Total time about 75 mins one way which is faster and more reliable than a cab. Make sure that you have enough time to reclear security which can take up to one hour during peak times. Suggest that you arrive at JFK 2 hours before your outbound flight.
You will be tight for time