Can I use Visa Debit Cards issued by banks in India to withdraw money from Bangladesh ATM?

Yes you can, but the transaction charges will be pretty steep. I recommend you contact your bank to know the charges.