At least four people were dead and dozens unaccounted for Thursday after a ferry sank on its way to the Indonesian resort island of Bali, according to local authorities who said 23 survivors had been plucked from the water so far.

Rescuers were still racing to find missing people after the vessel carrying 65 people sank before midnight on Wednesday as it sailed to the popular holiday destination from the country's main island Java.

"Twenty-three have been rescued, four dead," said Rama Samtama Putra, police chief of Banyuwangi in East Java, where the boat departed.

President Prabowo Subianto, who was on a trip to Saudi Arabia, ordered an immediate emergency response, cabinet secretary Teddy Indra Wijaya said in a statement on Thursday, adding the cause of the accident was "bad weather".

"KMP Tunu Pratama Jaya... sank about 25 minutes after weighing anchor," the Surabaya search and rescue agency said.

"The ferry's manifest data totalled 53 passengers and 12 passenger crews," it said.

A rescue team and inflatable rescue boats were dispatched and a bigger vessel was later sent from Surabaya to assist the search efforts.

Four of the known survivors saved themselves by using the ferry's lifeboat and were found in the water early on Thursday, the agency said.

It said the ferry was also transporting 22 vehicles, including 14 trucks. (AFP)