How do you calculate the Total Losses in a transformer?

It is the transformer electrical losses, which include no-load losses (core losses) and load losses (winding losses). There is not a specific formula to calculate total losses since they vary by size and manufacturer. Manufacturers will often provide charts that show losses at one or more load points.

HPS does offer an efficiency calculator for distribution transformers.

HPS Efficiency Calculator