The indicator light flashes blue then
goes out when trying to connect the light
switch/dimmer to my compatible Zigbee hub

When the indicator light flashes blue, it means that the switch/dimmer is attempting to connect to the network. If it does not succeed, after a while, the indicator light goes out.
Refer to the instruction manual of your Zigbee gateway for more information on pairing a device.