Bosch BNO055 magnetic field interferences lead to data drift

I have been using the Bosch Smart Sensors BNO055 (version with gyroscope, acceleration sensors and magnetic sensors) for motion tracking some time. However, I have the problem that when the sensors approach a metallic object, a large drift in the data becomes apparent. The magnetometer in the sensor is tremendously affected by metallic components. Do you have any idea how to minimize this effect while still using a design with magnetometer? Can I tune the sensor fusion algorithm to deal with this issue? 

I would be very grateful if you have any ideas. Thanks in advance!