BME680 API calibration calculation inconsistency

[sanemusha]

I found inconsistency between BME680 API and Datasheet for the calculation of gas sensor heater resistance.

Page 21 of Datasheet explains floating point and integer calculations but they are not equivalent. var1 in integer calculation is (amb_temp*par_g3/10)<<8, but the floating point method calculates this part as (amb_temp*par_g3*10)<<8 (var1 in the floating point method is different from that in the integer method, so I deduced this result by rearranging the whole floating point formulae).

Moreover, the integer calculation in API is var1=(amb_temp*par_g3)/1000)*256 (small modifications applied for clarity) in line 1006 of bme680.c which is again different from the above two. The floating point calculation in API looks the same as that in Datasheet.

So, there are three variations in calculating var1 or its equivalent, each 100 times different from another. Could you tell me which is correct, and hopefully fix the incorrect formulae? Thank you.

[Vincent]

Please always follow the APIs with latest formula. 

BTW,  the integar calculation is suppose to be different as float.  

Even in API level,  the formula is not exactly same for integar and float.

So you can just call the API function to get the expected gas registers.

[sanemusha]

I understand that we should always follow APIs, not Datasheet, and that calculation formulae are different between integer and float. Still there is a discrepancy between the results of integer and float APIs. Below is an example.

Assumption: par_g1=-22, par_g2=-4829, par_g3=18, res_heat_range=1, res_heat_val=47 which are taken from a real device sample, and target temp=200.

Integer "calc_heater_res" results: 87 for amb_temp=20, 30, 40, and 50.

Float "calc_heater_res" results: 88 for amb_temp=20 and 30, and 89 for amb_temp=40 and 50.

I understand the differences are minor, but if I change "var1=(amb_temp*par_g3/1000)*256" to "var1=(amb_temp*par_g3*10)*256" in the integer calculation as I wrote earlier, the results become identical. So I still believe there is inconsistency between integer and float calculations in the API.

[Vincent · Accepted solution]

Thank you.  i will double check with the API team and also the datasheet then come back to you