Officials say the gunman first demanded money. When he was refused, a shot was fired and hostages taken.
The man demanded to speak to the elite Raid police unit that shot Islamist gunman Mohamed Merah.
Merah, who claimed al-Qaeda training, killed seven people in Toulouse before he was shot dead by police in March.
There are no reports of injuries in the incident so far.
The area around the bank has been cordoned off and two nearby schools have been closed.
"We're taking measures so we can start a dialogue," said public prosecutor Michel Valet, at the scene.
Special police units from the GIPN (Groupe d'intervention de la police nationale) have arrived from Marseille and Bordeaux.
A police union source told the regional newspaper Ouest-France it was not clear whether the man's claim about al-Qaeda was "serious or a fantasy".