You can take public transportation to Bayonne via subway, PATH train and bus. Take Airtrain to Jamaica Station, take the inbound E subway to World Trade Center (last stop), take the PATH train to Exchange Place-Jersey City and transfer to the NJ Transit #81 bus to Bayonne. Total cost of Airtrain, subway, PATH and bus is about $12. Taxi would probably cost well over $75.
You can take public