Yes, you need to charge GST on expense reimbursements unless expenses are incurred as a pure agent.
As per valuation provisions under GST, any expense incurred in providing a supply has to be included in value of supply.
You did not mention nature of expense, so I will understand that these are normal expenses such as travel, conveyance etc.
Now understand with an example what is not taxable.
As a chartered accountant if I pay some taxes on behalf of my client, and claim reimbursement, I do not have to charge GST on this as a I did it as a pure agent and did not earn any profit on this.