How do I know if my brake light not working is a bulb or a fuse?

I have a 2005 Jeep Liberty and the middle and driver's side tail lights are both out. How do I tell if it is the bulb in the tail light itself or a fuse in the fuse box?