Cheapest by far is by bus. There's a listing of bus companies and their locations on this page. You can get to any of these by public transportation. Penn Station is served by the NYAS bus ($15) from each terminal, by E subway or LIRR from Jamaica station via Airtrain. J subway from Jamaica station stops in the Chinatown area.
Cheapest by far is by bus.